[
  {
    "id": "django_gen_01",
    "repo": "django",
    "question": "What is the full class hierarchy and architectural design behind how the Polygon geometry class supports list-like ring mutation through GEOS C bindings?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `Polygon` class in `django/contrib/gis/geos/polygon.py` is part of a multi-layered architecture that combines GEOS C library bindings with Python's list-like mutation capabilities.\n\n**Class Hierarchy:**\n`Polygon` \u2192 `GEOSGeometry` \u2192 `GEOSGeometryBase` (+ `ListMixin` via MRO) \u2192 `GEOSBase`\n\nSpecifically, `GEOSGeometry` is defined as `class GEOSGeometry(GEOSGeometryBase, ListMixin)` in geometry.py, giving Polygon all list mutation methods (append, extend, pop, sort, etc.) from `ListMixin`.\n\n**Key architectural details:**\n\n1. **_minlength = 1** on Polygon, meaning a Polygon must have at least 1 ring (the exterior ring). This is enforced by `ListMixin._rebuild()` which checks `newLen < self._minlength`.\n\n2. **_set_single and _assign_extended_slice** are explicitly set to the rebuild variants: `_set_single = GEOSGeometry._set_single_rebuild` and `_assign_extended_slice = GEOSGeometry._assign_extended_slice_rebuild`. This means any single-ring mutation triggers a full polygon rebuild via `_set_slice` \u2192 `_set_list`, which destroys the old GEOS geometry pointer and creates a new one.\n\n3. **_create_polygon** handles both empty polygons and populated ones. For empty polygons (length=0), it calls `capi.create_empty_polygon()` which maps to the GEOS C function `GEOSGeom_createEmptyPolygon`. For non-empty ones, it clones all ring pointers using `_clone()`, creates a ctypes array `(GEOM_PTR * n_holes)` for the holes, and calls `capi.create_polygon(shell, holes_param, n_holes)` which maps to `GEOSGeom_createPolygon`.\n\n4. **_clone method** handles two types of inputs: raw `GEOM_PTR` ctypes pointers (calling `capi.geom_clone(g)`) and Python GEOSGeometry wrapper objects (calling `capi.geom_clone(g.ptr)`).\n\n5. **_get_single_external** returns a full `GEOSGeometry` object constructed from a cloned pointer with the parent's SRID, while **_get_single_internal** returns a raw GEOM_PTR without cloning. The external ring uses `capi.get_extring` and interior rings use `capi.get_intring` with `index - 1` offset.\n\n6. **from_bbox** has a type-checking optimization: if any bbox value is not a float or int, it falls back to WKT string parsing via `GEOSGeometry(\"POLYGON(...)\")`. Otherwise it constructs directly from coordinate tuples.\n\n7. **LinearRing** (used as ring type) is a subclass of `LineString` with `_minlength = 4` and `_init_func = capi.create_linearring`. The `_construct_ring` method on Polygon returns an existing LinearRing directly if the input is already a LinearRing instance, otherwise tries to construct one, raising `TypeError` on failure.\n\n8. **_set_list** preserves the SRID during mutation: it saves the current SRID, replaces the pointer via `_create_polygon`, restores SRID if it was set, then destroys the old geometry pointer via `capi.destroy_geom`.\n\n9. The GEOS C function factories use **lazy loading** via `GEOSFuncFactory` (libgeos.py), which uses `@cached_property` on `func` to defer loading until first call. `GEOM_PTR` is `POINTER(GEOSGeom_t)` where `GEOSGeom_t` is an opaque ctypes `Structure`.\n\n10. The `__init__` logic detects `Polygon(shell, (hole1, hole2))` form: if there's exactly 1 hole argument that is a tuple/list, and its first element is a LinearRing, it unpacks that tuple as the hole list.",
    "rubric": [
      "Identifies the class hierarchy: Polygon \u2192 GEOSGeometry \u2192 (GEOSGeometryBase + ListMixin) \u2192 GEOSBase/CPointerBase",
      "Explains that GEOSGeometry inherits from both GEOSGeometryBase and ListMixin, giving Polygon list mutation methods like append, extend, pop, sort",
      "Notes that Polygon sets _minlength = 1 (requiring at least one ring), enforced by ListMixin._rebuild",
      "Explains that _set_single and _assign_extended_slice are set to the rebuild variants, meaning any ring mutation triggers a full polygon rebuild via _set_list \u2192 _create_polygon",
      "Describes _create_polygon: handles empty polygons via capi.create_empty_polygon, and populated ones by cloning ring pointers and creating a ctypes array (GEOM_PTR * n_holes) for holes",
      "Describes _clone method: handles both raw GEOM_PTR (calls capi.geom_clone(g)) and GEOSGeometry wrapper objects (calls capi.geom_clone(g.ptr))",
      "Explains _get_single_external returns a GEOSGeometry from a cloned pointer with SRID, while _get_single_internal returns a raw GEOM_PTR; interior rings use index-1 offset",
      "Mentions _set_list preserves SRID during mutation: saves SRID, replaces pointer, restores SRID, then destroys old geometry",
      "Notes that LinearRing (used for rings) is a LineString subclass with _minlength=4 and _init_func=capi.create_linearring",
      "Mentions GEOSFuncFactory uses @cached_property for lazy loading of GEOS C functions, and GEOM_PTR is POINTER(GEOSGeom_t) where GEOSGeom_t is an opaque Structure"
    ],
    "key_files": [
      "django/contrib/gis/geos/polygon.py",
      "django/contrib/gis/geos/geometry.py",
      "django/contrib/gis/geos/mutable_list.py",
      "django/contrib/gis/geos/linestring.py",
      "django/contrib/gis/geos/prototypes/geom.py",
      "django/contrib/gis/geos/libgeos.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/geos/polygon.py] django.contrib.gis.geos.polygon.Polygon\ndjango.contrib.gis.geos.polygon.Polygon.__init__:\n  Initialize on an exterior ring and a sequence of holes (both\n        instances may be either LinearRing instances, or a tuple/list\n        that may be constructed into a LinearRing).\n\n        Examples of initialization, where shell, hole1, and hole2 are\n        valid LinearRing geometries:\n        >>> from django.contrib.gis.geos import LinearRing, Polygon\n        >>> shell = hole1 = hole2 = LinearRing()\n        >>> poly = Polygon(shell, hole1, hole2)\n        >>> poly = Polygon(shell, (hole1, hole2))\n\n        >>> # Example where a tuple parameters are used:\n        >>> poly = Polygon(((0, 0), (0, 10), (10, 10), (10, 0), (0, 0)),\n        ...                ((4, 4), (4, 6), (6, 6), (6, 4), (4, 4)))\n\ndjango.contrib.gis.geos.polygon.Polygon.__iter__:\n  Iterate over each ring in the polygon.\n\ndjango.contrib.gis.geos.polygon.Polygon.__len__:\n  Return the number of rings in this Polygon.\n\ndjango.contrib.gis.geos.polygon.Polygon.from_bbox:\n  Construct a Polygon from a bounding box (4-tuple).\n\ndjango.contrib.gis.geos.polygon.Polygon._construct_ring:\n  Try to construct a ring from the given parameter.\n\ndjango.contrib.gis.geos.polygon.Polygon._get_single_internal:\n  Return the ring at the specified index. The first index, 0, will\n        always return the exterior ring. Indices > 0 will return the\n        interior ring at the given index (e.g., poly[1] and poly[2] would\n        return the first and second interior ring, respectively).\n\n        CAREFUL: Internal/External are not the same as Interior/Exterior!\n        Return a pointer from the existing geometries for use internally by the\n        object's methods. _get_single_external() returns a clone of the same\n        geometry for use by external code.\n\ndjango.contrib.gis.geos.polygon.Polygon.num_interior_rings:\n  Return the number of interior rings.\n\ndjango.contrib.gis.geos.polygon.Polygon._get_ext_ring:\n  Get the exterior ring of the Polygon.\n\ndjango.contrib.gis.geos.polygon.Polygon._set_ext_ring:\n  Set the exterior ring of the Polygon.\n\ndjango.contrib.gis.geos.polygon.Polygon.tuple:\n  Get the tuple for each ring in this Polygon.\n\ndjango.contrib.gis.geos.polygon.Polygon.kml:\n  Return the KML representation of this Polygon.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, internal method implementations, ctypes details, etc.) that go far beyond what the documentation provides. These cannot be verified from the documentation alone.",
      "Point 10 describes backward-compatibility logic for `Polygon(shell, (hole1, hole2))` form. The documentation shows this as a valid initialization example (`>>> poly = Polygon(shell, (hole1, hole2))`), so the claim about detecting this form is consistent with the docs, though the specific implementation details (lines 37-43, checking first element is LinearRing) cannot be verified.",
      "The class hierarchy claim (`Polygon \u2192 GEOSGeometry \u2192 GEOSGeometryBase (+ ListMixin via MRO) \u2192 GEOSBase \u2192 CPointerBase`) cannot be verified from the documentation but is plausible code-level detail.",
      "The documentation shows `poly = Polygon(shell, hole1, hole2)` and `poly = Polygon(shell, (hole1, hole2))` as both valid forms of initialization, which is consistent with the answer's point 10.",
      "The `_get_single_internal` description in the answer (returns raw GEOM_PTR without cloning, uses get_extring/get_intring with index-1 offset) is consistent with the doc's warning about Internal/External vs Interior/Exterior and the note about returning a pointer from existing geometries."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments to remove 'CPointerBase' (not shown) and 'backward-compatibility' rationale characterization.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_04",
    "repo": "django",
    "question": "How does Django's SpatialReference class wrap the OGR C library, from pointer management and initialization through the ctypes prototype layer?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `SpatialReference` class in Django's GDAL wrapper is designed as a ctypes-based wrapper around the OGR/GDAL C library's `OGRSpatialReference` object. Here's how the system is designed:\n\n## Inheritance and Memory Management\n\n`SpatialReference` inherits from `GDALBase`, which inherits from `CPointerBase` (in `django/contrib/gis/ptr.py`). `CPointerBase` provides a `ptr` property that validates the underlying C pointer is not NULL before returning it (raising the class's `null_ptr_exception_class` if it is). `GDALBase` sets `null_ptr_exception_class = GDALException`. The `CPointerBase.__del__` method automatically calls the destructor when the object is garbage collected, with a try/except catching `(AttributeError, ImportError, TypeError)` in case parts are already garbage collected.\n\nThe `SpatialReference` destructor is `capi.release_srs`, which calls `OSRRelease` (not `OSRDestroySpatialReference`), meaning it uses reference counting.\n\n## Initialization Logic\n\nThe `__init__` method accepts an `axis_order` parameter (defaulting to `AxisOrder.TRADITIONAL = 0`). It validates this is either `None` or an `AxisOrder` instance. The method handles multiple input types:\n1. If `srs_type='wkt'`, it creates a new blank SRS pointer via `OSRNewSpatialReference(b\"\")`, imports the WKT, then sets the axis strategy.\n2. If the input is a string that can be parsed as an integer (e.g., '4326'), it prepends 'EPSG:' to make it valid user input.\n3. If the input is an integer, the srs_type is set to 'epsg'.\n4. If the input is already a `c_void_p` (the ptr_type), it's treated as a raw OGR pointer.\n\nAfter pointer creation, it calls `capi.set_axis_strategy` (wrapping `OSRSetAxisMappingStrategy`) with `AxisOrder.TRADITIONAL` to ensure traditional GIS longitude/latitude ordering.\n\n## ctypes Prototype Generation Layer\n\nThe prototypes in `django/contrib/gis/gdal/prototypes/srs.py` use generator functions from `prototypes/generation.py`:\n- `srs_output()` sets `restype=c_void_p` and errcheck to `check_srs` (which raises `SRSException` on NULL pointer)\n- `void_output()` sets `restype=c_int` and errcheck to `check_errcode` (which calls `check_err` to map OGR error codes to exceptions)\n- `string_output()` frees memory via `lgdal.VSIFree` after extracting the string value\n- `const_string_output()` does NOT free the pointer (used for `get_attr_value`, `get_auth_name`, `get_auth_code`)\n- `units_func()` uses `double_output` with `strarg=True`, triggering the `check_str_arg` errcheck that returns both the double value and the decoded string pointer value as a tuple\n\n## Export Format Specifics\n\nThe WKT export (`to_wkt`) decodes with 'utf-8', the PROJ export (`to_proj`) decodes with 'ascii', and the XML export (`to_xml`) decodes with 'utf-8'. The `to_pretty_wkt` and `to_xml` functions use `offset=-2` meaning the string pointer is the second-to-last argument.\n\n## Windows-Specific Design\n\nOn Windows (`os.name == 'nt'`), certain OSR functions use STDCALL convention, so the library is loaded both as `CDLL` (for `lgdal`) and `WinDLL` (for `lwingdal`). The `std_call()` function returns the appropriate version based on OS. Functions like `OSRClone`, `OSRNewSpatialReference`, `OSRExportToWkt`, `OSRImportFromEPSG`, `OSRSetFromUserInput`, and `OCTNewCoordinateTransformation` use `std_call`, while functions like `OSRRelease`, `OSRValidate`, `OSRGetSemiMajor`, and `OSRIsGeographic` use `lgdal` directly.\n\n## Error Handling\n\nOGR error code 7 specifically maps to `(SRSException, \"Unsupported SRS.\")` in the `OGRERR_DICT`. The `check_err` function raises the appropriate exception class based on the returned integer code.\n\n## CoordTransform\n\n`CoordTransform` also inherits from `GDALBase`, uses `capi.destroy_ct` (calling `OCTDestroyCoordinateTransformation`) as its destructor, stores source/target names as `_srs1_name` and `_srs2_name`, and passes `source._ptr`/`target._ptr` (the raw underlying pointer, not the property) to `capi.new_ct`. It's used in geometry transforms, the GeoJSON serializer (cached per SRID), and `LayerMapping`.\n\n## The `clone()` method preserves axis_order\n\nWhen cloning, `SpatialReference.clone()` passes `axis_order=self.axis_order` to the constructor, ensuring the clone maintains the same axis ordering strategy.",
    "rubric": [
      "Explains the inheritance chain: SpatialReference \u2192 GDALBase \u2192 CPointerBase, where CPointerBase provides the ptr property that validates non-NULL pointers and GDALBase sets null_ptr_exception_class to GDALException",
      "Describes CPointerBase.__del__ calling the destructor with try/except catching (AttributeError, ImportError, TypeError) for garbage collection safety",
      "Notes that SpatialReference uses capi.release_srs (OSRRelease, reference counting) as its destructor, not OSRDestroySpatialReference",
      "Explains __init__ handles multiple input types: WKT (creates blank pointer then imports), string integers (prepend 'EPSG:'), plain integers (set srs_type='epsg'), and raw c_void_p pointers",
      "Mentions the AxisOrder enum (TRADITIONAL=0, AUTHORITY=1), the axis_order parameter validation, and capi.set_axis_strategy call to OSRSetAxisMappingStrategy",
      "Describes the ctypes prototype generation layer: srs_output sets restype=c_void_p with check_srs errcheck, void_output sets restype=c_int with check_errcode, string_output frees memory via VSIFree, const_string_output does NOT free the pointer",
      "Explains units_func uses double_output with strarg=True, triggering check_str_arg which returns both the double value and decoded string as a tuple",
      "Notes the Windows-specific design: CDLL (lgdal) vs WinDLL (lwingdal) for STDCALL convention, with std_call() selecting the appropriate one",
      "Explains CoordTransform: also inherits GDALBase, uses destroy_ct (OCTDestroyCoordinateTransformation), passes source._ptr/target._ptr (raw pointer, not property), stores _srs1_name/_srs2_name",
      "Notes that clone() preserves axis_order by passing axis_order=self.axis_order to the constructor"
    ],
    "key_files": [
      "django/contrib/gis/gdal/srs.py",
      "django/contrib/gis/gdal/prototypes/srs.py",
      "django/contrib/gis/gdal/prototypes/generation.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/prototypes/errcheck.py",
      "django/contrib/gis/gdal/libgdal.py",
      "django/contrib/gis/gdal/error.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/srs.py] django.contrib.gis.gdal.srs.SpatialReference\ndjango.contrib.gis.gdal.srs.SpatialReference:\n  A wrapper for the OGRSpatialReference object. According to the GDAL web\n    site, the SpatialReference object \"provide[s] services to represent\n    coordinate systems (projections and datums) and to transform between them.\"\n\ndjango.contrib.gis.gdal.srs.SpatialReference.__init__:\n  Create a GDAL OSR Spatial Reference object from the given input.\n        The input may be string of OGC Well Known Text (WKT), an integer\n        EPSG code, a PROJ string, and/or a projection \"well known\" shorthand\n        string (one of 'WGS84', 'WGS72', 'NAD27', 'NAD83').\n\ndjango.contrib.gis.gdal.srs.SpatialReference.__getitem__:\n  Return the value of the given string attribute node, None if the node\n        doesn't exist. Can also take a tuple as a parameter, (target, child),\n        where child is the index of the attribute in the WKT. For example:\n\n        >>> wkt = (\n        ...     'GEOGCS[\"WGS 84\",'\n        ...     '  DATUM[\"WGS_1984, ... AUTHORITY[\"EPSG\",\"4326\"]'\n        ...     ']'\n        ... )\n        >>> srs = SpatialReference(wkt) # could also use 'WGS84', or 4326\n        >>> print(srs['GEOGCS'])\n        WGS 84\n        >>> print(srs['DATUM'])\n        WGS_1984\n        >>> print(srs['AUTHORITY'])\n        EPSG\n        >>> print(srs['AUTHORITY', 1]) # The authority value\n        4326\n        >>> print(srs['TOWGS84', 4]) # the fourth value in this wkt\n        0\n        >>> # For the units authority, have to use the pipe symbole.\n        >>> print(srs['UNIT|AUTHORITY'])\n        EPSG\n        >>> print(srs['UNIT|AUTHORITY', 1]) # The authority value for the units\n        9122\n\ndjango.contrib.gis.gdal.srs.SpatialReference.__str__:\n  Use 'pretty' WKT.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.attr_value:\n  The attribute value for the given target node (e.g. 'PROJCS'). The\n        index keyword specifies an index of the child node to return.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.auth_name:\n  Return the authority name for the given string target node.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.auth_code:\n  Return the authority code for the given string target node.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.clone:\n  Return a clone of this SpatialReference object.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.from_esri:\n  Morph this SpatialReference from ESRI's format to EPSG.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.identify_epsg:\n  This method inspects the WKT of this SpatialReference, and will\n        add EPSG authority nodes where an EPSG identifier is applicable.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.to_esri:\n  Morph this SpatialReference to ESRI's format.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.validate:\n  Check to see if the given spatial reference is valid.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.name:\n  Return the name of this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.srid:\n  Return the SRID of top-level authority, or None if undefined.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.linear_name:\n  Return the name of the linear units.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.linear_units:\n  Return the value of the linear units.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.angular_name:\n  Return the name of the angular units.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.angular_units:\n  Return the value of the angular units.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.units:\n  Return a 2-tuple of the units value and the units name. Automatically\n        determine whether to return the linear or angular units.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.ellipsoid:\n  Return a tuple of the ellipsoid parameters:\n         (semimajor axis, semiminor axis, and inverse flattening)\n\ndjango.contrib.gis.gdal.srs.SpatialReference.semi_major:\n  Return the Semi Major Axis for this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.semi_minor:\n  Return the Semi Minor Axis for this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.inverse_flattening:\n  Return the Inverse Flattening for this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.geographic:\n  Return True if this SpatialReference is geographic\n         (root node is GEOGCS).\n\ndjango.contrib.gis.gdal.srs.SpatialReference.local:\n  Return True if this SpatialReference is local (root node is LOCAL_CS).\n\ndjango.contrib.gis.gdal.srs.SpatialReference.projected:\n  Return True if this SpatialReference is a projected coordinate system\n         (root node is PROJCS).\n\ndjango.contrib.gis.gdal.srs.SpatialReference.import_epsg:\n  Import the Spatial Reference from the EPSG code (an integer).\n\ndjango.contrib.gis.gdal.srs.SpatialReference.import_proj:\n  Import the Spatial Reference from a PROJ string.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.import_user_input:\n  Import the Spatial Reference from the given user input string.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.import_wkt:\n  Import the Spatial Reference from OGC WKT (string)\n\ndjango.contrib.gis.gdal.srs.SpatialReference.import_xml:\n  Import the Spatial Reference from an XML string.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.wkt:\n  Return the WKT representation of this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.pretty_wkt:\n  Return the 'pretty' representation of the WKT.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.proj:\n  Return the PROJ representation for this Spatial Reference.\n\ndjango.contrib.gis.gdal.srs.SpatialReference.proj4:\n  Alias for proj().\n\ndjango.contrib.gis.gdal.srs.SpatialReference.xml:\n  Return the XML representation of this Spatial Reference.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (ctypes prototypes, Windows-specific design, memory management internals, CoordTransform details, error handling specifics) that are not present in the provided documentation. These appear to be extrapolations from actual source code knowledge rather than from the provided doc chunk.",
      "The documentation states that __init__ accepts 'string of OGC Well Known Text (WKT), an integer EPSG code, a PROJ string, and/or a projection well known shorthand string', while the answer adds details about axis_order parameter, c_void_p handling, and string-to-integer parsing that aren't in the doc.",
      "The answer's claim about clone() preserving axis_order is not mentioned in the documentation, which only says 'Return a clone of this SpatialReference object.'",
      "The documentation does not mention AxisOrder, CPointerBase, GDALBase, or any of the inheritance hierarchy described in the answer.",
      "The answer discusses CoordTransform extensively, but the provided documentation chunk only covers SpatialReference."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_03",
    "repo": "django",
    "question": "Where does OGRGeometry data flow when converting between GDAL and GEOS geometry systems, and how is memory managed?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The OGRGeometry data flows through a multi-layered architecture connecting GDAL C library calls to Python objects and bridging to the GEOS geometry system:\n\n**Data flow during construction (OGRGeometry.__init__):**\n1. Input is parsed using regex patterns imported from `django/contrib/gis/geometry.py`: `hex_regex`, `wkt_regex`, and `json_regex`.\n2. The geometry pointer is obtained via ctypes calls to GDAL C functions (`OGR_G_CreateFromWkt`, `OGR_G_CreateFromWkbEx`, `OGR_G_CreateGeometryFromJson`, `OGR_G_CreateGeometry`).\n3. The pointer is stored via `CPointerBase.ptr` setter (in `django/contrib/gis/ptr.py`), which validates it is `c_void_p` or None.\n4. The class is dynamically reassigned using the `GEO_CLASSES` dictionary lookup on `self.geom_type.num`, switching `self.__class__` to Point, LineString, Polygon, etc.\n\n**Data flow between OGR and GEOS (bidirectional WKB bridge):**\n- `OGRGeometry.geos` property: calls `self._geos_ptr()` which calls `GEOSGeometry._from_wkb(self.wkb)` \u2014 the WKB binary format is the interchange format.\n- `GEOSGeometry.ogr` property (in `django/contrib/gis/geos/geometry.py`): calls `gdal.OGRGeometry(self._ogr_ptr(), self.srs)` where `_ogr_ptr()` calls `gdal.OGRGeometry._from_wkb(self.wkb)` \u2014 again using WKB.\n\n**Data flow during transform:**\n- `OGRGeometry.transform()` accepts CoordTransform, SpatialReference, int, or str. For CoordTransform it calls `OGR_G_Transform`, for others it calls `OGR_G_TransformTo`.\n- `CoordTransform.__init__` (in srs.py) requires two SpatialReference objects and calls `capi.new_ct(source._ptr, target._ptr)`.\n\n**Memory management:**\n- `CPointerBase.__del__` calls `self.destructor(self.ptr)` \u2014 for OGRGeometry the destructor is `capi.destroy_geom` which wraps `OGR_G_DestroyGeometry`.\n- `GDALBase` (in base.py) inherits from `CPointerBase`.\n\n**The OGRGeomType._types mapping** covers types 0-12 (basic), 15-17, 100-102, 1008-1017 (Z variants), 2001-2017 (M variants), 3001-3017 (ZM variants), and types with wkb25bit offset (-2147483648 + type for 25D variants). The `GEO_CLASSES` dictionary maps these type numbers to corresponding Python geometry classes.",
    "rubric": [
      "Explains that WKB (Well-Known Binary) is the interchange format between OGR and GEOS in both directions",
      "Describes OGRGeometry._geos_ptr() calling GEOSGeometry._from_wkb(self.wkb) for OGR-to-GEOS conversion",
      "Describes GEOSGeometry.ogr or transform using OGRGeometry._from_wkb for GEOS-to-OGR conversion",
      "Mentions that OGRGeometry.__init__ uses regex patterns (hex_regex, wkt_regex, json_regex) to parse input and calls appropriate GDAL C functions",
      "Explains the dynamic class reassignment via GEO_CLASSES dictionary lookup on geom_type.num, switching __class__ to Point, LineString, etc.",
      "Describes CPointerBase (in ptr.py) managing pointer storage with validation and __del__ calling self.destructor(self.ptr) for cleanup",
      "Mentions that GDALBase inherits from CPointerBase and OGRGeometry's destructor is capi.destroy_geom"
    ],
    "key_files": [
      "django/contrib/gis/gdal/geometries.py",
      "django/contrib/gis/gdal/geomtype.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/prototypes/geom.py",
      "django/contrib/gis/gdal/srs.py",
      "django/contrib/gis/geos/geometry.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/geometries.py] django.contrib.gis.gdal.geometries\nThe OGRGeometry is a wrapper for using the OGR Geometry class\n(see https://gdal.org/api/ogrgeometry_cpp.html#_CPPv411OGRGeometry).\nOGRGeometry may be instantiated when reading geometries from OGR Data Sources\n(e.g. SHP files), or when given OGC WKT (a string).\n\nWhile the 'full' API is not present yet, the API is \"pythonic\" unlike\nthe traditional and \"next-generation\" OGR Python bindings. One major\nadvantage OGR Geometries have over their GEOS counterparts is support\nfor spatial reference systems and their transformation.\n\nExample:\n >>> from django.contrib.gis.gdal import (\n ...     OGRGeometry, OGRGeomType, SpatialReference\n ... )\n >>> wkt1, wkt2 = 'POINT(-90 30)', 'POLYGON((0 0, 5 0, 5 5, 0 5)'\n >>> pnt = OGRGeometry(wkt1)\n >>> print(pnt)\n POINT (-90 30)\n >>> mpnt = OGRGeometry(OGRGeomType('MultiPoint'), SpatialReference('WGS84'))\n >>> mpnt.add(wkt1)\n >>> mpnt.add(wkt1)\n >>> print(mpnt)\n MULTIPOINT (-90 30,-90 30)\n >>> print(mpnt.srs.name)\n WGS 84\n >>> print(mpnt.srs.proj)\n +proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs\n >>> mpnt.transform(SpatialReference('NAD27'))\n >>> print(mpnt.proj)\n +proj=longlat +ellps=clrk66 +datum=NAD27 +no_defs\n >>> print(mpnt)\n MULTIPOINT (-89.99993037860248 29.99979788655764,-89.99993037860248\n 29.99979788655764)\n\n The OGRGeomType class is to make it easy to specify an OGR geometry type:\n >>> from django.contrib.gis.gdal import OGRGeomType\n >>> gt1 = OGRGeomType(3) # Using an integer for the type\n >>> gt2 = OGRGeomType('Polygon') # Using a string\n >>> gt3 = OGRGeomType('POLYGON') # It's case-insensitive\n >>> # Equivalence works w/non-OGRGeomType objects:\n >>> print(gt1 == 3, gt1 == 'Polygon')\n True True",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of the answer's claims (about internal implementation details like regex patterns, ctypes calls, CPointerBase, GEO_CLASSES mapping, memory management, WKB bridge, etc.) go far beyond what the provided documentation contains. The documentation is a module-level docstring showing usage examples, not implementation details.",
      "These implementation claims cannot be verified against the provided documentation chunk, but they are plausible code-level details that could be derived from reading the actual source code.",
      "The answer does not contradict anything explicitly stated in the documentation.",
      "The documentation mentions transform() with SpatialReference('NAD27'), and the answer's description of transform() accepting SpatialReference/CoordTransform/int/str is consistent with the documented usage but extends beyond what is shown.",
      "The wkb25bit offset described as '-2147483648 + type' is a plausible implementation detail (0x80000000 is the wkb25DBit flag in OGR) but cannot be verified from the provided doc."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for line numbers, unshown file contents, and regex pattern descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_05",
    "repo": "django",
    "question": "What is the full class hierarchy and architectural design of Django's GEOS Point class, including how it manages coordinates through the C API?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `Point` class in Django's GIS module (`django/contrib/gis/geos/point.py`) is a geometry class that represents a 2D or 3D point. It inherits from `GEOSGeometry`, which itself inherits from both `GEOSGeometryBase` and `ListMixin`. `GEOSGeometryBase` extends `GEOSBase`, which extends `CPointerBase` from `django/contrib/gis/ptr.py` \u2014 the lowest-level base that manages a C pointer (`_ptr`) with null-pointer checking and automatic destructor invocation via `__del__`.\n\n**Key architectural details:**\n\n1. **Class attributes**: `Point` sets `_minlength = 2`, `_maxlength = 3`, and `has_cs = True`. The `_minlength` and `_maxlength` come from the `ListMixin` protocol, constraining valid coordinate counts. `has_cs = True` overrides the default `False` in `GEOSGeometryBase`, signaling that this geometry type has a coordinate sequence.\n\n2. **Coordinate sequence (`_cs`)**: During `_post_init()` (defined in `GEOSGeometryBase`), when `has_cs` is `True`, a `GEOSCoordSeq` object is created by calling `capi.get_cs(self.ptr)` and wrapping the result with `GEOSCoordSeq(ptr, self.hasz)`. All coordinate access (x, y, z properties) delegates to this `GEOSCoordSeq` instance via `getOrdinate` and `setOrdinate` methods, which in turn call the GEOS C library functions `GEOSCoordSeq_getOrdinate` and `GEOSCoordSeq_setOrdinate`.\n\n3. **Point creation via C API**: `_create_point` is a `@classmethod` that creates a coordinate sequence using `capi.create_cs(c_uint(1), c_uint(ndim))` (calling GEOS's `GEOSCoordSeq_create` with size=1), sets coordinates using `capi.cs_setx`, `capi.cs_sety`, and optionally `capi.cs_setz`, then calls `capi.create_point(cs)` which wraps the GEOS C function `GEOSGeom_createPoint`. For empty points, `_create_point` passes `None` to `capi.create_point`.\n\n4. **Empty point handling**: The `__init__` supports creating empty points when `x=None`. The `__len__` method returns 0 for empty points (checked via `self.empty`). Pickling of empty points is special-cased: `_to_pickle_wkb` returns `None` instead of WKB bytes, and `_from_pickle_wkb` recreates via `_create_empty()` when it receives `None`.\n\n5. **ListMixin integration**: `Point` implements the `ListMixin` protocol by providing `_get_single_external` (dispatching to x/y/z by index 0/1/2), `_set_single` (using `self._cs.setOrdinate(index, 0, value)`), and `_set_list` (which creates a new point via `_create_point`, destroys the old geometry with `capi.destroy_geom`, sets the new pointer, restores the SRID, and calls `_post_init`). `_get_single_internal` is explicitly set equal to `_get_single_external`.\n\n6. **Z setter guard**: The `z` setter raises `GEOSException('Cannot set Z on 2D Point.')` if the point doesn't have a Z dimension (`self.hasz` is False).\n\n7. **`coords` is an alias**: At class level, `coords = tuple` makes `coords` an alias for the `tuple` property.\n\n8. **GEOS type ID mapping**: In `GEOSGeometryBase.__init__`, Point is mapped to GEOS type ID `0` in the `_GEOS_CLASSES` dictionary.\n\n9. **`tuple` property**: The `tuple` getter delegates to `self._cs.tuple`, which in `GEOSCoordSeq` returns a single tuple (not a tuple of tuples) when `size == 1` (which is always the case for a Point). The `tuple` setter assigns to `self._cs[0]`, which invokes `GEOSCoordSeq.__setitem__` with dimension-aware point setters (_set_point_2d, _set_point_3d, etc.).",
    "rubric": [
      "Identifies the inheritance chain: Point \u2192 GEOSGeometry \u2192 GEOSGeometryBase (+ ListMixin) \u2192 GEOSBase \u2192 CPointerBase",
      "Explains CPointerBase manages a C pointer (_ptr) with null-pointer checking and destructor invocation in __del__",
      "Mentions Point sets _minlength=2, _maxlength=3, and has_cs=True as class attributes",
      "Explains that _post_init creates a GEOSCoordSeq from capi.get_cs(self.ptr) when has_cs is True",
      "Describes _create_point classmethod: creates coordinate sequence with capi.create_cs, sets x/y/z, and calls capi.create_point",
      "Explains empty point handling: x=None creates empty point, __len__ returns 0, pickling special-cases with _to_pickle_wkb returning None",
      "Describes ListMixin integration: _get_single_external dispatches by index to x/y/z, _set_single uses _cs.setOrdinate, _set_list recreates the point geometry",
      "Mentions z setter raises GEOSException if point is 2D",
      "Explains coordinate access delegates through GEOSCoordSeq's getOrdinate/setOrdinate which call capi.cs_getordinate/cs_setordinate",
      "Notes that the tuple property in GEOSCoordSeq returns a single tuple (not nested) when size==1, and coords is aliased to tuple at class level"
    ],
    "key_files": [
      "django/contrib/gis/geos/point.py",
      "django/contrib/gis/geos/geometry.py",
      "django/contrib/gis/geos/coordseq.py",
      "django/contrib/gis/geos/prototypes/geom.py",
      "django/contrib/gis/geos/prototypes/coordseq.py",
      "django/contrib/gis/geos/mutable_list.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/geos/base.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/geos/point.py] django.contrib.gis.geos.point.Point\ndjango.contrib.gis.geos.point.Point.__init__:\n  The Point object may be initialized with either a tuple, or individual\n        parameters.\n\n        For example:\n        >>> p = Point((5, 23))  # 2D point, passed in as a tuple\n        >>> p = Point(5, 23, 8)  # 3D point, passed as individual parameters\n\ndjango.contrib.gis.geos.point.Point._create_point:\n  Create a coordinate sequence, set X, Y, [Z], and create point\n\ndjango.contrib.gis.geos.point.Point.__iter__:\n  Iterate over coordinates of this Point.\n\ndjango.contrib.gis.geos.point.Point.__len__:\n  Return the number of dimensions for this Point (either 0, 2 or 3).\n\ndjango.contrib.gis.geos.point.Point.x:\n  Return the X component of the Point.\n\ndjango.contrib.gis.geos.point.Point.x:\n  Set the X component of the Point.\n\ndjango.contrib.gis.geos.point.Point.y:\n  Return the Y component of the Point.\n\ndjango.contrib.gis.geos.point.Point.y:\n  Set the Y component of the Point.\n\ndjango.contrib.gis.geos.point.Point.z:\n  Return the Z component of the Point.\n\ndjango.contrib.gis.geos.point.Point.z:\n  Set the Z component of the Point.\n\ndjango.contrib.gis.geos.point.Point.tuple:\n  Return a tuple of the point.\n\ndjango.contrib.gis.geos.point.Point.tuple:\n  Set the coordinates of the point with the given tuple.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code; no documentation-only claims found.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_06",
    "repo": "django",
    "question": "Why is `ogrinspect` split into a public function and a private `_ogrinspect` generator, and how does the management command exploit that design?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `ogrinspect` function in `django/contrib/gis/utils/ogrinspect.py` exists as a convenience wrapper that joins the output of the internal `_ogrinspect()` generator into a single string with newlines. The separation of `ogrinspect` from `_ogrinspect` (prefixed with underscore) serves two distinct purposes:\n\n1. **Streaming vs. String output**: `_ogrinspect` is a generator (uses `yield` statements) that produces model definition lines one at a time. This allows the management command (`django/contrib/gis/management/commands/ogrinspect.py`) to collect lines into a list and append additional output (like the `--mapping` dictionary) before joining everything together. The public `ogrinspect()` function simply does `\"\\n\".join(_ogrinspect(*args, **kwargs))` for users who just want the complete string.\n\n2. **Management command integration**: The management command (`Command.handle()`) calls `_ogrinspect` directly rather than `ogrinspect` because it needs to extend the output with optional mapping dictionary generation. It uses `get_func_args(_ogrinspect)` to dynamically filter command-line options to only those parameters accepted by `_ogrinspect`, providing a flexible interface without hard-coding parameter names.\n\nThe `_ogrinspect` function appends `'field'` to OGR field names that end with an underscore (e.g., `mfield += \"field\"`). It uses a nested `process_kwarg()` helper to normalize the `null`, `blank`, and `decimal` parameters \u2014 when these are boolean `True`, they expand to all field names in the layer; when they are lists/tuples, they are lowercased for case-insensitive matching.\n\nFor geometry field generation, `_ogrinspect` delegates to `OGRGeomType.django` property which converts the OGR geometry type name to a Django field name by stripping '25D' suffix, mapping 'Unknown' to 'Geometry', mapping 'PointZ' to 'Point', and appending 'Field'. The `to_multi()` method on `OGRGeomType` simply adds 3 to the type number (since Multi variants are offset by 3 in OGR's type enumeration: Point=1\u2192MultiPoint=4, LineString=2\u2192MultiLineString=5, Polygon=3\u2192MultiPolygon=6).\n\nThe SRID handling logic shows that when SRID is 4326, the `srid_str` is set to an empty string (omitting the srid parameter from the generated field). When the layer's SRS is None or cannot determine an SRID, it defaults to `srid=-1`.",
    "rubric": [
      "_ogrinspect is a generator (uses yield) that produces model definition lines one at a time, while ogrinspect joins them into a single string with newlines",
      "The management command calls _ogrinspect directly (not ogrinspect) so it can collect lines into a list and append additional output like the mapping dictionary before joining",
      "The command uses get_func_args(_ogrinspect) to dynamically filter command-line options to only those parameters accepted by _ogrinspect",
      "The process_kwarg nested helper normalizes null/blank/decimal: boolean True expands to all field names, lists/tuples are lowercased for case-insensitive matching",
      "OGR field names ending with underscore get 'field' appended to avoid trailing underscores in model field names",
      "SRID 4326 results in an empty srid_str since WGS84 is the default; None/unknown SRS defaults to srid=-1"
    ],
    "key_files": [
      "django/contrib/gis/utils/ogrinspect.py",
      "django/contrib/gis/management/commands/ogrinspect.py",
      "django/contrib/gis/gdal/geomtype.py",
      "django/contrib/gis/gdal/field.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/utils/ogrinspect.py] django.contrib.gis.utils.ogrinspect.ogrinspect\nGiven a data source (either a string or a DataSource object) and a string\n    model name this function will generate a GeoDjango model.\n\n    Usage:\n\n    >>> from django.contrib.gis.utils import ogrinspect\n    >>> ogrinspect('/path/to/shapefile.shp','NewModel')\n\n    ...will print model definition to stout\n\n    or put this in a Python script and use to redirect the output to a new\n    model like:\n\n    $ python generate_model.py > myapp/models.py\n\n    # generate_model.py\n    from django.contrib.gis.utils import ogrinspect\n    shp_file = 'data/mapping_hacks/world_borders.shp'\n    model_name = 'WorldBorders'\n\n    print(ogrinspect(shp_file, model_name, multi_geom=True, srid=4326,\n                     geom_name='shapes', blank=True))\n\n    Required Arguments\n     `datasource` => string or DataSource object to file pointer\n\n     `model name` => string of name of new model class to create\n\n    Optional Keyword Arguments\n     `geom_name` => For specifying the model name for the Geometry Field.\n       Otherwise will default to `geom`\n\n     `layer_key` => The key specifying which layer in the DataSource to use;\n       defaults to 0 (the first layer). May be an integer index or a string\n       identifier for the layer.\n\n     `srid` => The SRID to use for the Geometry Field. If it can be determined,\n       the SRID of the datasource is used.\n\n     `multi_geom` => Boolean (default: False) - specify as multigeometry.\n\n     `name_field` => String - specifies a field name to return for the\n       __str__() method (which will be generated if specified).\n\n     `imports` => Boolean (default: True) - set to False to omit the\n       `from django.contrib.gis.db import models` code from the\n       autogenerated models thus avoiding duplicated imports when building\n       more than one model by batching ogrinspect()\n\n     `decimal` => Boolean or sequence (default: False). When set to True\n       all generated model fields corresponding to the `OFTReal` type will\n       be `DecimalField` instead of `FloatField`. A sequence of specific\n       field names to generate as `DecimalField` may also be used.\n\n     `blank` => Boolean or sequence (default: False). When set to True all\n       generated model fields will have `blank=True`. If the user wants to\n       give specific fields to have blank, then a list/tuple of OGR field\n       names may be used.\n\n     `null` => Boolean (default: False) - When set to True all generated\n       model fields will have `null=True`. If the user wants to specify\n       give specific fields to have null, then a list/tuple of OGR field\n       names may be used.\n\n    Note: Call the _ogrinspect() helper to do the heavy lifting.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many implementation details (e.g., field name collision avoidance by appending 'field', process_kwarg() helper, OGRGeomType.django property behavior, to_multi() method adding 3 to type number, SRID 4326 resulting in empty string) that are not present in the documentation. These appear to be plausible code-level extrapolations but cannot be verified from the provided documentation alone.",
      "The claim that ogrinspect does '\\n'.join(_ogrinspect(*args, **kwargs)) is plausible but not explicitly stated in the documentation - the doc only says 'Call the _ogrinspect() helper to do the heavy lifting'.",
      "The claim about get_func_args(_ogrinspect) being used in the management command is not in the documentation and cannot be verified.",
      "The claim about srid=-1 as default when SRS is None is not in the documentation - the doc says 'If it can be determined, the SRID of the datasource is used' but doesn't specify the fallback value."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in design rationale language and the explanation of why 4326 gets empty string; nearly all claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_02",
    "repo": "django",
    "question": "Why does the ModelAdmin class use so many specific design patterns \u2014 like transaction wrapping only for non-safe methods, MRO-based formfield resolution, and unbound method fetching for actions?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `ModelAdmin` class in `django/contrib/admin/options.py` is designed with several key architectural and design rationale decisions:\n\n## Inheritance & Metaclass Architecture\n`ModelAdmin` inherits from `BaseModelAdmin`, which itself uses `forms.MediaDefiningClass` as its metaclass (line 130). This metaclass (defined in `django/forms/widgets.py:268`) automatically adds a `media` property to classes that don't define one explicitly. However, `ModelAdmin` explicitly defines its own `media` property (line 753) that includes jQuery, core.js, RelatedObjectLookups.js, actions.js, urlify.js, prepopulate.js, and xregexp, selecting `.min.js` versions when `settings.DEBUG` is False.\n\n## Transaction Wrapping Strategy\nBoth `changeform_view` and `delete_view` use a split-method pattern: the public method (e.g., `changeform_view` at line 1847) conditionally wraps calls in `transaction.atomic()` only for non-safe HTTP methods. GET/HEAD/OPTIONS/TRACE requests bypass the transaction wrapper and directly call the private `_changeform_view`/`_delete_view`. This is a deliberate performance optimization \u2014 read-only requests don't need transaction guarantees, while write operations (POST) are wrapped in `transaction.atomic(using=router.db_for_write(self.model))` to ensure database consistency across the correct database in multi-db setups.\n\n## Formfield Override Merging (Not Overwriting)\n`BaseModelAdmin.__init__` (line 154) uses `copy.deepcopy(FORMFIELD_FOR_DBFIELD_DEFAULTS)` and then merges subclass `formfield_overrides` using `setdefault().update()` rather than replacing. This design ensures that subclasses only need to specify the keys they want to change, while preserving defaults for other field types. The comment explicitly states: \"Merge FORMFIELD_FOR_DBFIELD_DEFAULTS with the formfield_overrides rather than simply overwriting.\"\n\n## MRO-Based Formfield Resolution\nIn `formfield_for_dbfield` (line 226), the method walks the field class's MRO (`db_field.__class__.mro()`) to find formfield overrides. This means overrides for parent field types (e.g., `CharField`) will also apply to subclass fields (e.g., `EmailField`) if no more specific override exists.\n\n## Permission System Design\n`has_view_permission` (line 601) deliberately checks BOTH the `view` AND `change` permission codenames \u2014 the rationale being that users with change permission should implicitly have view permission. The separate `has_view_or_change_permission` (line 619) exists for cases where subclasses override `has_view_permission` to no longer include the change permission check.\n\n## Action Resolution: Unbound Methods for Consistent Calling Conventions\n`get_action` (line 1073-1077) deliberately fetches methods from `self.__class__` rather than `self` to get an unbound method. The comment explains: \"this ensures that the calling conventions are the same for functions and methods.\" In `response_action` (line 1695), actions are always called as `func(self, request, queryset)`, so both standalone functions and methods receive the ModelAdmin as the first argument consistently.\n\n## Action Priority and Filtering\n`_get_base_actions` first collects site-wide actions, then extends with ModelAdmin-specific actions. Site-wide actions are skipped if their name conflicts with a ModelAdmin action (line 1015: `if name in base_action_names: continue`), giving ModelAdmin actions priority. Actions are further filtered by `_filter_actions_by_permissions`, which checks `callable.allowed_permissions` attribute (set by the `@admin.action(permissions=...)` decorator) and dynamically resolves `has_{permission}_permission` methods on the ModelAdmin.\n\n## Actions Disabled in Popups\n`get_actions` (line 1046) returns an empty dict if `IS_POPUP_VAR` is in `request.GET`, preventing actions from appearing in popup windows.\n\n## Logging: Single Object vs Bulk Create\n`log_addition` and `log_change` pass `single_object=True` to `LogEntry.objects.log_actions()`, which returns a single `LogEntry` instance. `log_deletions` does NOT pass `single_object=True` because deletions can be bulk operations. The `LogEntryManager.log_actions` method (in `models.py`) uses `save()` for single entries and `bulk_create()` for multiple, and `object_repr` is truncated to 200 characters (`str(obj)[:200]`).\n\n## View-Only Inline Form Validation Bypass\n`_create_formsets` (line 2380-2387) deliberately bypasses validation for view-only inline forms by setting `form._errors = {}` and `form.cleaned_data = form.initial`. This is because view-only form data won't be present in `request.POST`, so validation would fail. However, this bypass is skipped if the user has explicitly marked the form for deletion.\n\n## List Editable Security: Tampered PK Validation\n`_get_list_editable_queryset` (line 2015-2022) validates each PK from POST data using the model's pk field's `to_python()`. If any PK fails validation (indicating tampered POST data), it falls back to the full queryset rather than the optimized filtered queryset, disabling the optimization as a defensive measure.\n\n## Backward Compatibility URL Redirect\n`get_urls` (line 738-746) includes a catch-all `<path:object_id>/` URL pattern that redirects to the change view. The comment says this \"was the change url before 1.9,\" using `RedirectView.as_view()` for backward compatibility.\n\n## Save As New: Hiding Save Buttons on Failure\nWhen `_saveasnew` is in POST and form validation fails (line 1979-1987), the context sets `show_save` and `show_save_and_continue` to False and forces `add = False` to use the change template, preventing confusing UI states.",
    "rubric": [
      "Explain that changeform_view/delete_view only wrap non-safe HTTP methods (POST) in transaction.atomic() as a performance optimization, while GET/HEAD/OPTIONS/TRACE bypass the transaction wrapper",
      "Explain that formfield_for_dbfield walks the field class's MRO to find formfield overrides, allowing parent field type overrides to apply to subclass fields when no specific override exists",
      "Explain that get_action fetches methods from self.__class__ (not self) to get unbound methods, ensuring consistent calling conventions between standalone functions and methods \u2014 both receive ModelAdmin as first argument when called as func(self, request, queryset)",
      "Explain that BaseModelAdmin.__init__ merges formfield_overrides with FORMFIELD_FOR_DBFIELD_DEFAULTS using setdefault().update() rather than overwriting, so subclasses only need to specify keys they want to change",
      "Explain that has_view_permission checks both view AND change permission codenames because users with change permission should implicitly have view access",
      "Explain that _get_base_actions skips site-wide actions whose names conflict with ModelAdmin actions, giving ModelAdmin actions priority",
      "Explain that _create_formsets bypasses validation for view-only inline forms (setting form._errors={} and form.cleaned_data=form.initial) because their data won't be in request.POST, but this is skipped for forms marked for deletion"
    ],
    "key_files": [
      "django/contrib/admin/options.py",
      "django/contrib/admin/models.py",
      "django/contrib/admin/decorators.py",
      "django/forms/widgets.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/options.py] django.contrib.admin.options.ModelAdmin\ndjango.contrib.admin.options.ModelAdmin:\n  Encapsulate all admin options and functionality for a given model.\n\ndjango.contrib.admin.options.ModelAdmin.get_model_perms:\n  Return a dict of all perms for this model. This dict has the keys\n        ``add``, ``change``, ``delete``, and ``view`` mapping to the True/False\n        for each of those actions.\n\ndjango.contrib.admin.options.ModelAdmin.get_form:\n  Return a Form class for use in the admin add view. This is used by\n        add_view and change_view.\n\ndjango.contrib.admin.options.ModelAdmin.get_changelist:\n  Return the ChangeList class for use on the changelist page.\n\ndjango.contrib.admin.options.ModelAdmin.get_changelist_instance:\n  Return a `ChangeList` instance based on `request`. May raise\n        `IncorrectLookupParameters`.\n\ndjango.contrib.admin.options.ModelAdmin.get_object:\n  Return an instance matching the field and value provided, the primary\n        key is used if no field is provided. Return ``None`` if no match is\n        found or the object_id fails validation.\n\ndjango.contrib.admin.options.ModelAdmin.get_changelist_form:\n  Return a Form class for use in the Formset on the changelist page.\n\ndjango.contrib.admin.options.ModelAdmin.get_changelist_formset:\n  Return a FormSet class for use on the changelist page if list_editable\n        is used.\n\ndjango.contrib.admin.options.ModelAdmin.get_formsets_with_inlines:\n  Yield formsets and the corresponding inlines.\n\ndjango.contrib.admin.options.ModelAdmin.log_addition:\n  Log that an object has been successfully added.\n\n        The default implementation creates an admin LogEntry object.\n\ndjango.contrib.admin.options.ModelAdmin.log_change:\n  Log that an object has been successfully changed.\n\n        The default implementation creates an admin LogEntry object.\n\ndjango.contrib.admin.options.ModelAdmin.log_deletions:\n  Log that objects will be deleted. Note that this method must be called\n        before the deletion.\n\n        The default implementation creates admin LogEntry objects.\n\ndjango.contrib.admin.options.ModelAdmin.action_checkbox:\n  A list_display column containing a checkbox widget.\n\ndjango.contrib.admin.options.ModelAdmin._get_base_actions:\n  Return the list of actions, prior to any request-based filtering.\n\ndjango.contrib.admin.options.ModelAdmin._filter_actions_by_permissions:\n  Filter out any actions that the user doesn't have access to.\n\ndjango.contrib.admin.options.ModelAdmin.get_actions:\n  Return a dictionary mapping the names of all actions for this\n        ModelAdmin to a tuple of (callable, name, description) for each action.\n\ndjango.contrib.admin.options.ModelAdmin.get_action_choices:\n  Return a list of choices for use in a form object. Each choice is a\n        tuple (name, description).\n\ndjango.contrib.admin.options.ModelAdmin.get_action:\n  Return a given action from a parameter, which can either be a callable,\n        or the name of a method on the ModelAdmin. Return is a tuple of\n        (callable, name, description).\n\ndjango.contrib.admin.options.ModelAdmin.get_list_display:\n  Return a sequence containing the fields to be displayed on the\n        changelist.\n\ndjango.contrib.admin.options.ModelAdmin.get_list_display_links:\n  Return a sequence containing the fields to be displayed as links\n        on the changelist. The list_display parameter is the list of fields\n        returned by get_list_display().\n\ndjango.contrib.admin.options.ModelAdmin.get_list_filter:\n  Return a sequence containing the fields to be displayed as filters in\n        the right sidebar of the changelist page.\n\ndjango.contrib.admin.options.ModelAdmin.get_list_select_related:\n  Return a list of fields to add to the select_related() part of the\n        changelist items query.\n\ndjango.contrib.admin.options.ModelAdmin.get_search_fields:\n  Return a sequence containing the fields to be searched whenever\n        somebody submits a search query.\n\ndjango.contrib.admin.options.ModelAdmin.get_search_results:\n  Return a tuple containing a queryset to implement the search\n        and a boolean indicating if the results may contain duplicates.\n\ndjango.contrib.admin.options.ModelAdmin.get_preserved_filters:\n  Return the preserved filters querystring.\n\ndjango.contrib.admin.options.ModelAdmin.construct_change_message:\n  Construct a JSON structure describing changes from a changed object.\n\ndjango.contrib.admin.options.ModelAdmin.message_user:\n  Send a message to the user. The default implementation\n        posts a message using the django.contrib.messages backend.\n\n        Exposes almost the same API as messages.add_message(), but accepts the\n        positional arguments in a different order to maintain backwards\n        compatibility. For convenience, it accepts the `level` argument as\n        a string rather than the usual level number.\n\ndjango.contrib.admin.options.ModelAdmin.save_form:\n  Given a ModelForm return an unsaved instance. ``change`` is True if\n        the object is being changed, and False if it's being added.\n\ndjango.contrib.admin.options.ModelAdmin.save_model:\n  Given a model instance save it to the database.\n\ndjango.contrib.admin.options.ModelAdmin.delete_model:\n  Given a model instance delete it from the database.\n\ndjango.contrib.admin.options.ModelAdmin.delete_queryset:\n  Given a queryset, delete it from the database.\n\ndjango.contrib.admin.options.ModelAdmin.save_formset:\n  Given an inline formset save it to the database.\n\ndjango.contrib.admin.options.ModelAdmin.save_related:\n  Given the ``HttpRequest``, the parent ``ModelForm`` instance, the\n        list of inline formsets and a boolean value based on whether the\n        parent is being added or changed, save the related objects to the\n        database. Note that at this point save_form() and save_model() have\n        already been called.\n\ndjango.contrib.admin.options.ModelAdmin.response_add:\n  Determine the HttpResponse for the add_view stage.\n\ndjango.contrib.admin.options.ModelAdmin.response_change:\n  Determine the HttpResponse for the change_view stage.\n\ndjango.contrib.admin.options.ModelAdmin.response_post_save_add:\n  Figure out where to redirect after the 'Save' button has been pressed\n        when adding a new object.\n\ndjango.contrib.admin.options.ModelAdmin.response_post_save_change:\n  Figure out where to redirect after the 'Save' button has been pressed\n        when editing an existing object.\n\ndjango.contrib.admin.options.ModelAdmin.response_action:\n  Handle an admin action. This is called if a request is POSTed to the\n        changelist; it returns an HttpResponse if the action was handled, and\n        None otherwise.\n\ndjango.contrib.admin.options.ModelAdmin.response_delete:\n  Determine the HttpResponse for the delete_view stage.\n\ndjango.contrib.admin.options.ModelAdmin.get_changeform_initial_data:\n  Get the initial form data from the request's GET params.\n\ndjango.contrib.admin.options.ModelAdmin._get_obj_does_not_exist_redirect:\n  Create a message informing the user that the object doesn't exist\n        and return a redirect to the admin index page.\n\ndjango.contrib.admin.options.ModelAdmin._get_edited_object_pks:\n  Return POST data values of list_editable primary keys.\n\ndjango.contrib.admin.options.ModelAdmin._get_list_editable_queryset:\n  Based on POST data, return a queryset of the objects that were edited\n        via list_editable.\n\ndjango.contrib.admin.options.ModelAdmin.changelist_view:\n  The 'change list' admin view for this model.\n\ndjango.contrib.admin.options.ModelAdmin.get_deleted_objects:\n  Hook for customizing the delete process for the delete view and the\n        \"delete selected\" action.\n\ndjango.contrib.admin.options.ModelAdmin._delete_view:\n  The 'delete' admin view for this model.\n\ndjango.contrib.admin.options.ModelAdmin.history_view:\n  The 'history' admin view for this model.\n\ndjango.contrib.admin.options.ModelAdmin._create_formsets:\n  Helper function to generate formsets for add/change_view.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (line numbers, specific code patterns, variable names, internal method logic) that go far beyond what the provided documentation describes. The documentation only contains docstrings/method descriptions, not source code details.",
      "Claims about specific line numbers (e.g., 'line 130', 'line 753', 'line 1847') cannot be verified from the documentation provided.",
      "Claims about transaction wrapping strategy, formfield override merging, MRO-based resolution, action priority filtering logic, etc. are plausible but entirely unsupported by the provided documentation which only contains method docstrings.",
      "The claim about `has_view_permission` checking both view AND change permissions is not mentioned in the documentation, which only describes `get_model_perms` returning a dict with add/change/delete/view keys.",
      "The documentation for `log_deletions` says 'Log that objects will be deleted. Note that this method must be called before the deletion.' - the answer's claim about `log_deletions` not passing `single_object=True` and using `bulk_create()` is an implementation detail not in the docs.",
      "The answer's claims about `_create_formsets` bypassing validation are implementation details beyond the doc's simple description of 'Helper function to generate formsets for add/change_view.'"
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_07",
    "repo": "django",
    "question": "Where is `add_srs_entry` defined, how does it determine backend support, and what models does it interact with across different spatial backends?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `add_srs_entry` function is located in `django/contrib/gis/utils/srs.py` and is re-exported through `django/contrib/gis/utils/__init__.py` (included in the `__all__` list alongside `mapping`, `ogrinfo`, `ogrinspect`, `LayerMapError`, and `LayerMapping`).\n\n**Where it lives and connects:**\n\n1. **Definition**: `django/contrib/gis/utils/srs.py` \u2014 the function takes a `srs` argument (either a GDAL `SpatialReference` or something that can be used to construct one, like an integer SRID).\n\n2. **Backend support check**: The function first checks `connection.ops.spatial_version` via `hasattr` to verify it's a spatial backend, then checks `connection.features.supports_add_srs_entry`. This feature flag is:\n   - **`True` by default** in `BaseSpatialFeatures` (`base/features.py`)\n   - **Explicitly `False`** in `MySQL` (`mysql/features.py`) and `Oracle` (`oracle/features.py`)\n   - **Inherited as `True`** in PostGIS (`postgis/features.py`) and SpatiaLite (`spatialite/features.py`), since neither overrides it\n\n3. **SpatialRefSys model retrieval**: The function calls `connection.ops.spatial_ref_sys()` which is an abstract method on `BaseSpatialOperations` (raises `NotImplementedError`). Concrete implementations return:\n   - `PostGISSpatialRefSys` (from `postgis/models.py`) \u2014 has fields: `srid`, `auth_name`, `auth_srid`, `srtext`, `proj4text`\n   - `SpatialiteSpatialRefSys` (from `spatialite/models.py`) \u2014 has fields: `srid`, `auth_name`, `auth_srid`, `ref_sys_name`, `proj4text`, `srtext`\n   Both models inherit from `SpatialRefSysMixin` (in `base/models.py`) and both map to the `spatial_ref_sys` database table with `managed = False`.\n\n4. **Backend-specific field handling**: The function dynamically inspects the model's field names via `SpatialRefSys._meta.get_fields()`. If `\"srtext\"` is present (both PostGIS and SpatiaLite have it), it adds `srs.wkt` to kwargs. If `\"ref_sys_name\"` is present (only SpatiaLite has this field), it adds the `ref_sys_name` value.\n\n5. **Idempotency**: The function uses a try/except pattern \u2014 it first tries `SpatialRefSys.objects.using(database).get(srid=srs.srid)`. Only if `DoesNotExist` is raised does it call `create(**kwargs)`.\n\n6. **SpatialRefSysMixin**: Located in `django/contrib/gis/db/backends/base/models.py`, this mixin provides a `srs` cached_property that first tries to construct a GDAL `SpatialReference` from `self.wkt`, and if that fails, falls back to `self.proj4text`. It also provides properties like `ellipsoid`, `name`, `spheroid`, `datum`, `projected`, `local`, `geographic`, `linear_name`, `linear_units`, `angular_name`, `angular_units`, and `units`, plus class methods `get_units` and `get_spheroid`.\n\n7. **Testing**: In `tests/gis_tests/test_spatialrefsys.py`, the `test_add_entry` method is decorated with `@skipUnlessDBFeature(\"supports_add_srs_entry\")` and tests adding SRID 3857, then verifies the entry exists and that `get_spheroid(srs.wkt)` starts with `\"SPHEROID[\"`.",
    "rubric": [
      "Defined in django/contrib/gis/utils/srs.py and re-exported through django/contrib/gis/utils/__init__.py",
      "Checks connection.ops.spatial_version (via hasattr) and connection.features.supports_add_srs_entry to verify backend support",
      "supports_add_srs_entry is True by default in BaseSpatialFeatures but explicitly False in MySQL and Oracle backends",
      "PostGIS and SpatiaLite inherit the True default (neither overrides it)",
      "Calls connection.ops.spatial_ref_sys() to get the SpatialRefSys model (abstract/NotImplementedError in base, concrete in PostGIS and SpatiaLite)",
      "PostGISSpatialRefSys has fields srid, auth_name, auth_srid, srtext, proj4text; SpatialiteSpatialRefSys adds ref_sys_name",
      "Both models inherit from SpatialRefSysMixin in base/models.py and both use the spatial_ref_sys table with managed=False",
      "Dynamically inspects model fields: adds srs.wkt if 'srtext' is present, adds ref_sys_name if that field is present (SpatiaLite only)",
      "Uses try/except with get(srid=srs.srid) for idempotency \u2014 only creates if DoesNotExist is raised"
    ],
    "key_files": [
      "django/contrib/gis/utils/srs.py",
      "django/contrib/gis/db/backends/base/features.py",
      "django/contrib/gis/db/backends/base/models.py",
      "django/contrib/gis/db/backends/postgis/models.py",
      "django/contrib/gis/db/backends/spatialite/models.py",
      "django/contrib/gis/db/backends/base/operations.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/utils/srs.py] django.contrib.gis.utils.srs.add_srs_entry\nTake a GDAL SpatialReference system and add its information to the\n    `spatial_ref_sys` table of the spatial backend. Doing this enables\n    database-level spatial transformations for the backend. Thus, this utility\n    is useful for adding spatial reference systems not included by default with\n    the backend:\n\n    >>> from django.contrib.gis.utils import add_srs_entry\n    >>> add_srs_entry(3857)\n\n    Keyword Arguments:\n     auth_name:\n       This keyword may be customized with the value of the `auth_name` field.\n       Defaults to 'EPSG'.\n\n     auth_srid:\n       This keyword may be customized with the value of the `auth_srid` field.\n       Defaults to the SRID determined by GDAL.\n\n     ref_sys_name:\n       For SpatiaLite users only, sets the value of the `ref_sys_name` field.\n       Defaults to the name determined by GDAL.\n\n     database:\n      The name of the database connection to use; the default is the value\n      of `django.db.DEFAULT_DB_ALIAS` (at the time of this writing, its value\n      is 'default').",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Only one claim about a code comment was removed; nearly all claims are verifiable from stripped source code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_08",
    "repo": "django",
    "question": "How does Django implement portable file locking across different platforms, and what happens when no locking mechanism is available?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `django/core/files/locks.py` module implements portable file locking through platform-specific branching using `os.name == 'nt'`.\n\n**Platform detection and three code paths:**\n\n1. **Windows (`os.name == 'nt'`):** Uses `ctypes` to call Windows kernel32 `LockFileEx`/`UnlockFileEx` functions directly. The lock constants are: `LOCK_SH = 0`, `LOCK_NB = 0x1`, `LOCK_EX = 0x2`. The implementation defines a custom ctypes `OVERLAPPED` structure (containing `_OFFSET` and `_OFFSET_UNION` inner structures). The `ULONG_PTR` type is dynamically determined by comparing `sizeof(c_ulong)` with `sizeof(c_void_p)` \u2014 if they differ, `c_int64` is used instead of `c_ulong`. The `lock()` function obtains a Windows file handle via `msvcrt.get_osfhandle(_fd(f))`, creates a zeroed `OVERLAPPED` struct, and calls `LockFileEx(hfile, flags, 0, 0, 0xFFFF0000, byref(overlapped))`. The `0xFFFF0000` is passed as an argument to `LockFileEx`. Both `lock()` and `unlock()` return `bool(ret)` where ret is the Windows API return value.\n\n2. **Non-Windows with `fcntl` available:** Uses `fcntl.flock()`. Lock constants are taken directly from the `fcntl` module (`fcntl.LOCK_SH`, `fcntl.LOCK_NB`, `fcntl.LOCK_EX`). The `lock()` function calls `fcntl.flock(_fd(f), flags)` and returns `True` on success; it catches `BlockingIOError` and returns `False`. The `unlock()` function calls `fcntl.flock(_fd(f), fcntl.LOCK_UN)` and always returns `True`.\n\n3. **Fallback (no `fcntl` available):** If `fcntl` import raises `ImportError` or `AttributeError`, all lock constants are set to 0, `lock()` always returns `False`, and `unlock()` always returns `True` \u2014 effectively making locking a no-op.\n\n**The `_fd()` helper:** Accepts either a file object or a raw file descriptor integer. It checks `hasattr(f, 'fileno')` and calls `f.fileno()` if available, otherwise returns `f` directly. This allows the lock functions to work with both file objects and raw integer file descriptors.\n\n**Usage in Django:**\n- `django/core/files/move.py`: `file_move_safe()` acquires `LOCK_EX` on a raw fd (from `os.open()`) when `os.rename()` fails and it falls back to manual copying, unlocking in a `finally` block.\n- `django/core/files/storage/filesystem.py`: `FileSystemStorage._save()` acquires `LOCK_EX` on a raw fd, with handling to use `os.fdopen()` only after the first chunk determines the write mode ('wb' vs 'wt').\n- `django/core/cache/backends/filebased.py`: `FileBasedCache.touch()` acquires `LOCK_EX` on a file object to read and rewrite cache entries.\n- `django/test/testcases.py`: `SerializeMixin.setUpClass()` acquires `LOCK_EX` on a lockfile to serialize test case execution; the lockfile is closed via `addClassCleanup()`.\n\nThe module exports exactly five names via `__all__`: `('LOCK_EX', 'LOCK_SH', 'LOCK_NB', 'lock', 'unlock')`.",
    "rubric": [
      "Explains platform detection uses `os.name == 'nt'` to branch between Windows and non-Windows paths",
      "Describes the Windows path: uses ctypes to call kernel32 LockFileEx/UnlockFileEx with a custom OVERLAPPED structure, and msvcrt.get_osfhandle to get the Windows file handle",
      "Mentions Windows lock constants: LOCK_SH=0, LOCK_NB=0x1, LOCK_EX=0x2",
      "Describes ULONG_PTR dynamic sizing by comparing sizeof(c_ulong) vs sizeof(c_void_p), using c_int64 if they differ",
      "Describes the non-Windows path with fcntl: uses fcntl.flock(), catches BlockingIOError to return False, lock constants come from the fcntl module",
      "Describes the fallback no-op path when fcntl import fails (ImportError or AttributeError): all constants set to 0, lock() returns False, unlock() returns True",
      "Explains the _fd() helper: checks hasattr(f, 'fileno') to support both file objects and raw integer file descriptors",
      "Mentions at least two places in Django that use the locks module (e.g., file_move_safe, FileSystemStorage._save, FileBasedCache, SerializeMixin)",
      "Notes __all__ exports five names: LOCK_EX, LOCK_SH, LOCK_NB, lock, unlock"
    ],
    "key_files": [
      "django/core/files/locks.py",
      "django/core/files/move.py",
      "django/core/files/storage/filesystem.py",
      "django/core/cache/backends/filebased.py",
      "django/test/testcases.py"
    ],
    "source_doc": "[docstring: django/core/files/locks.py] django.core.files.locks\nPortable file locking utilities.\n\nBased partially on an example by Jonathan Feignberg in the Python\nCookbook [1] (licensed under the Python Software License) and a ctypes port by\nAnatoly Techtonik for Roundup [2] (license [3]).\n\n[1] https://code.activestate.com/recipes/65203/\n[2] https://sourceforge.net/p/roundup/code/ci/default/tree/roundup/backends/portalocker.py  # NOQA\n[3] https://sourceforge.net/p/roundup/code/ci/default/tree/COPYING.txt\n\nExample Usage::\n\n    >>> from django.core.files import locks\n    >>> with open('./file', 'wb') as f:\n    ...     locks.lock(f, locks.LOCK_EX)\n    ...     f.write('Django')",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in Windows API constant names and purpose descriptions; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_11",
    "repo": "django",
    "question": "Where is FileDescriptor defined and how does it integrate with DeferredAttribute to handle file field access on model instances?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "`FileDescriptor` is located in `django/db/models/fields/files.py`. It inherits from `DeferredAttribute` which is defined in `django/db/models/query_utils.py`. The `FileDescriptor` is installed on model classes via `FileField.contribute_to_class()`, which calls `setattr(cls, self.attname, self.descriptor_class(self))`. The `ImageFileDescriptor` subclass of `FileDescriptor` is defined in the same file and is used by `ImageField`.\n\nKey architectural details:\n\n1. `FileDescriptor` inherits from `DeferredAttribute` (defined in `django/db/models/query_utils.py`). The parent class `DeferredAttribute` does NOT define a `__set__` method; it only defines `__init__`, `__get__`, `_check_parent_chain`, `fetch_one`, and `fetch_many`. `FileDescriptor` adds its own `__set__` method which simply does `instance.__dict__[self.field.attname] = value`.\n\n2. In `FileDescriptor.__get__`, when the value retrieved from `super().__get__()` (which calls `DeferredAttribute.__get__`) is a `File` instance but NOT a `FieldFile`, the descriptor wraps it in `self.field.attr_class` (which is `FieldFile` for `FileField` or `ImageFieldFile` for `ImageField`), sets `file_copy._committed = False`, and stores the wrapped file in `instance.__dict__`.\n\n3. `FileDescriptor.__get__` handles five distinct cases: (a) string or None \u2192 wraps in attr_class, (b) `DatabaseDefault` instance \u2192 uses `self.field.db_default`, (c) `File` but not `FieldFile` \u2192 wraps and marks as uncommitted, (d) `FieldFile` without `field` attr \u2192 restores instance/field/storage, (e) `FieldFile` with mismatched instance \u2192 corrects the instance reference.\n\n4. The `DeferredAttribute.__get__` parent method handles deferred field loading: if the field name is not in `instance.__dict__`, it either checks the parent chain or calls `instance._state.fetch_mode.fetch(self, instance)` to load the value.\n\n5. `ImageFileDescriptor.__set__` extends `FileDescriptor.__set__` by additionally calling `self.field.update_dimension_fields(instance, force=True)` when `previous_file is not None`.",
    "rubric": [
      "FileDescriptor is located in django/db/models/fields/files.py and inherits from DeferredAttribute (defined in django/db/models/query_utils.py)",
      "FileDescriptor is installed on model classes via FileField.contribute_to_class() using setattr(cls, self.attname, self.descriptor_class(self))",
      "DeferredAttribute does NOT define __set__; FileDescriptor adds its own __set__ that stores the value in instance.__dict__[self.field.attname]",
      "FileDescriptor.__get__ calls super().__get__() (DeferredAttribute.__get__) and then handles multiple cases: string/None wrapping, File-but-not-FieldFile wrapping with _committed=False, and FieldFile instance/field restoration",
      "DeferredAttribute.__get__ handles deferred field loading by checking instance.__dict__ and fetching values if not present",
      "ImageFileDescriptor subclasses FileDescriptor and extends __set__ to call update_dimension_fields when previous_file is not None"
    ],
    "key_files": [
      "django/db/models/fields/files.py",
      "django/db/models/query_utils.py",
      "django/core/files/base.py"
    ],
    "source_doc": "[docstring: django/db/models/fields/files.py] django.db.models.fields.files.FileDescriptor\ndjango.db.models.fields.files.FileDescriptor:\n  The descriptor for the file attribute on the model instance. Return a\n    FieldFile when accessed so you can write code like::\n\n        >>> from myapp.models import MyModel\n        >>> instance = MyModel.objects.get(pk=1)\n        >>> instance.file.size\n\n    Assign a file object on assignment so you can do::\n\n        >>> with open('/path/to/hello.world') as f:\n        ...     instance.file = File(f)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: line numbers, 'pickle recovery' annotation, and 'not during Model.__init__()' rationale are not verifiable from stripped code; all structural and logic claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_10",
    "repo": "django",
    "question": "Why does Django's model Field class use so many cached_property decorators and custom copy/pickle methods instead of simpler implementations?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The Django `Field` class (in `django/db/models/fields/__init__.py`) employs several deliberate performance optimizations:\n\n**1. `cached_col` and `get_col()` \u2014 Avoiding repeated Col object creation during query compilation**\nThe `get_col()` method checks if the requested alias matches `self.model._meta.db_table` and the output_field is self or None; if so, it returns `self.cached_col`, a `@cached_property` that creates a single `Col(self.model._meta.db_table, self)` expression object and reuses it. Since `get_col()` is called during SQL compilation, the caching avoids repeatedly constructing identical `Col` objects for the most common case.\n\n**2. `_get_default` as `@cached_property` \u2014 Caching callable resolution once**\nThe `get_default()` method delegates to `self._get_default()`, but `_get_default` is a `@cached_property` that returns a *callable* (not the default value itself). This means the decision logic about what kind of default to use (callable default, static default wrapped in a lambda, `DatabaseDefault`, `return_None`, or `str`) is computed only once. The module-level `return_None` function exists so that fields returning `None` as their default can reference a shared function.\n\n**3. `__deepcopy__` uses shallow copy**\nThe `__deepcopy__` method uses `copy.copy(self)` (shallow copy) rather than true deep copy. Only `self.remote_field` gets shallow-copied if present.\n\n**4. `__copy__` uses `Empty()` to bypass `__reduce__`**\nThe `__copy__` method creates an `Empty()` instance and then reassigns its `__class__`, rather than using normal copy mechanics, which avoids triggering `__reduce__`. The `_empty()` function uses the same pattern for pickling unattached fields.\n\n**5. `__reduce__` \u2014 Pickling returns the canonical instance, not a copy**\nFor model-attached fields, `__reduce__` returns `_load_field` with `(app_label, model_name, field_name)`, which on unpickling fetches the field from `model._meta.get_field()`. It also pops `_get_default` from the pickle state.\n\n**6. `validators` and `error_messages` as `@cached_property` \u2014 Lazy computation**\nBoth `validators` and `error_messages` are `@cached_property`, meaning their values are computed once on first access and cached. The raw user-provided values are stored in `_validators` and `_error_messages` for deconstruction.\n\n**7. `unique` as `@cached_property` \u2014 Avoiding repeated boolean computation**\nThe `unique` property is a `@cached_property` that returns `self._unique or self.primary_key`, so the short-circuit boolean is evaluated once.\n\n**8. `creation_counter` and `__hash__` \u2014 O(1) hashing and O(log n) field insertion via bisect**\nThe `__hash__` returns `hash(self.creation_counter)`, an integer, which is O(1). The `__lt__` method (from `@total_ordering`) enables comparison. In `Options.add_field()` (options.py), `bisect.insort()` is used to insert fields into `local_fields` and `local_many_to_many` in sorted order, leveraging the `creation_counter` for O(log n) insertion.\n\n**9. `get_db_converters` uses `hasattr` check for `from_db_value`**\nThe `get_db_converters` method uses `hasattr(self, 'from_db_value')` rather than a default no-op method, so that the base Field class (which has no `from_db_value`) returns an empty list.\n\n**10. `get_class_lookups` uses `functools.cache` in RegisterLookupMixin**\nThe parent class `RegisterLookupMixin` (query_utils.py) uses `@functools.cache` on `get_class_lookups`, caching the merged MRO lookup dictionary per class. Cache is explicitly invalidated via `_clear_cached_class_lookups` only when new lookups are registered.",
    "rubric": [
      "Mentions cached_col/get_col() caching Col objects to avoid repeated creation during SQL compilation",
      "Explains _get_default as a cached_property that caches the callable resolution logic (not the default value itself) so decision logic runs only once",
      "Describes __deepcopy__ using shallow copy instead of true deep copy for performance",
      "Explains __copy__ using Empty() class trick to bypass __reduce__ overhead",
      "Mentions __reduce__ returning _load_field for model-attached fields to fetch canonical instance rather than copying, and popping _get_default from state",
      "Notes validators and/or error_messages as @cached_property for lazy one-time computation",
      "Mentions unique as @cached_property to avoid repeated boolean evaluation",
      "Explains creation_counter enabling O(1) hashing and O(log n) sorted insertion via bisect",
      "Notes get_db_converters using hasattr check for from_db_value rather than a no-op method to avoid unnecessary converter overhead",
      "Mentions get_class_lookups using functools.cache in RegisterLookupMixin to cache merged MRO lookup dictionaries"
    ],
    "key_files": [
      "django/db/models/fields/__init__.py",
      "django/db/models/query_utils.py",
      "django/db/models/options.py",
      "django/db/models/expressions.py"
    ],
    "source_doc": "[docstring: django/db/models/fields/__init__.py] django.db.models.fields.Field\ndjango.db.models.fields.Field:\n  Base class for all field types\n\ndjango.db.models.fields.Field.__str__:\n  Return \"app_label.model_label.field_name\" for fields attached to\n        models.\n\ndjango.db.models.fields.Field.__repr__:\n  Display the module, class, and name of the field.\n\ndjango.db.models.fields.Field._check_field_name:\n  Check if field name is valid, i.e. 1) does not end with an\n        underscore, 2) does not contain \"__\" and 3) is not \"pk\".\n\ndjango.db.models.fields.Field.select_format:\n  Custom format for select clauses. For example, GIS columns need to be\n        selected as AsText(table.col) on MySQL as the table.col data can't be\n        used by Django.\n\ndjango.db.models.fields.Field.deconstruct:\n  Return enough information to recreate the field as a 4-tuple:\n\n         * The name of the field on the model, if contribute_to_class() has\n           been run.\n         * The import path of the field, including the class, e.g.\n           django.db.models.IntegerField. This should be the most portable\n           version, so less specific may be better.\n         * A list of positional arguments.\n         * A dict of keyword arguments.\n\n        Note that the positional or keyword arguments must contain values of\n        the following types (including inner values of collection types):\n\n         * None, bool, str, int, float, complex, set, frozenset, list, tuple,\n           dict\n         * UUID\n         * datetime.datetime (naive), datetime.date\n         * top-level classes, top-level functions - will be referenced by their\n           full import path\n         * Storage instances - these have their own deconstruct() method\n\n        This is because the values here must be serialized into a text format\n        (possibly new Python code, possibly JSON) and these are the only types\n        with encoding handlers defined.\n\n        There's no need to return the exact way the field was instantiated this\n        time, just ensure that the resulting field is the same - prefer keyword\n        arguments over positional ones, and omit parameters with their default\n        values.\n\ndjango.db.models.fields.Field.clone:\n  Uses deconstruct() to clone a new copy of this Field.\n        Will not preserve any class attachments/attribute names.\n\ndjango.db.models.fields.Field.__reduce__:\n  Pickling should return the model._meta.fields instance of the field,\n        not a new copy of that field. So, use the app registry to load the\n        model and then the field back.\n\ndjango.db.models.fields.Field.get_pk_value_on_save:\n  Hook to generate new PK values on save. This method is called when\n        saving instances with no primary key value set. If this method returns\n        something else than None, then the returned value is used when saving\n        the new instance.\n\ndjango.db.models.fields.Field.to_python:\n  Convert the input value into the expected Python data type, raising\n        django.core.exceptions.ValidationError if the data can't be converted.\n        Return the converted value. Subclasses should override this.\n\ndjango.db.models.fields.Field.validators:\n  Some validators can't be created at field initialization time.\n        This method provides a way to delay their creation until required.\n\ndjango.db.models.fields.Field.validate:\n  Validate value and raise ValidationError if necessary. Subclasses\n        should override this to provide validation logic.\n\ndjango.db.models.fields.Field.clean:\n  Convert the value's type and run validation. Validation errors\n        from to_python() and validate() are propagated. Return the correct\n        value if no error is raised.\n\ndjango.db.models.fields.Field.db_check:\n  Return the database column check constraint for this field, for the\n        provided connection. Works the same way as db_type() for the case that\n        get_internal_type() does not map to a preexisting model field.\n\ndjango.db.models.fields.Field.db_type:\n  Return the database column data type for this field, for the provided\n        connection.\n\ndjango.db.models.fields.Field.rel_db_type:\n  Return the data type that a related field pointing to this field should\n        use. For example, this method is called by ForeignKey and OneToOneField\n        to determine its data type.\n\ndjango.db.models.fields.Field.cast_db_type:\n  Return the data type to use in the Cast() function.\n\ndjango.db.models.fields.Field.db_parameters:\n  Extension of db_type(), providing a range of different return values\n        (type, checks). This will look at db_type(), allowing custom model\n        fields to override it.\n\ndjango.db.models.fields.Field.db_returning:\n  Private API intended only to be used by Django itself.\n\ndjango.db.models.fields.Field.contribute_to_class:\n  Register the field with the model class it belongs to.\n\n        If private_only is True, create a separate instance of this field\n        for every subclass of cls, even if cls is not an abstract model.\n\ndjango.db.models.fields.Field.get_filter_kwargs_for_object:\n  Return a dict that when passed as kwargs to self.model.filter(), would\n        yield all instances having the same value for this field as obj has.\n\ndjango.db.models.fields.Field.pre_save:\n  Return field's value just before saving.\n\ndjango.db.models.fields.Field.get_prep_value:\n  Perform preliminary non-db specific value checks and conversions.\n\ndjango.db.models.fields.Field.get_db_prep_value:\n  Return field's value prepared for interacting with the database\n        backend.\n\n        Used by the default implementations of get_db_prep_save().\n\ndjango.db.models.fields.Field.get_db_prep_save:\n  Return field's value prepared for saving into a database.\n\ndjango.db.models.fields.Field.has_default:\n  Return a boolean of whether this field has a default value.\n\ndjango.db.models.fields.Field.has_db_default:\n  Return a boolean of whether this field has a db_default value.\n\ndjango.db.models.fields.Field.get_default:\n  Return the default value for this field.\n\ndjango.db.models.fields.Field.get_choices:\n  Return choices with a default blank choices included, for use\n        as <select> choices for this field.\n\ndjango.db.models.fields.Field.value_to_string:\n  Return a string value of this field from the passed obj.\n        This is used by the serialization framework.\n\ndjango.db.models.fields.Field.flatchoices:\n  Flattened version of choices tuple.\n\ndjango.db.models.fields.Field.formfield:\n  Return a django.forms.Field instance for this field.\n\ndjango.db.models.fields.Field.value_from_object:\n  Return the value of this field in the given model instance.\n\ndjango.db.models.fields.Field.slice_expression:\n  Return a slice of this field.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code structure; some design rationale quotes and performance justifications come from stripped comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_09",
    "repo": "django",
    "question": "What is the role of `BaseExpression`, what are its class-level attributes, and how do `Expression` and `Query` build on it?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `BaseExpression` class in `django/db/models/expressions.py` is a base class that other expression classes inherit from. It defines 8 class-level attributes: `empty_result_set_value = NotImplemented`, `is_summary = False`, `filterable = True`, `window_compatible = False`, `allowed_default = False`, `constraint_validation_compatible = True`, `set_returning = False`, and `allows_composite_expressions = False`.\n\n**Key inheritance structure:**\n- `Expression(BaseExpression, Combinable)` - the `@deconstructible` decorated class that adds arithmetic combining and identity/equality based on constructor signatures\n- `Query(BaseExpression)` in `django/db/models/sql/query.py` - the SQL query object itself inherits from `BaseExpression`\n\n**The `convert_value` cached_property:**\nThe `convert_value` property returns type-specific converter lambdas based on `output_field.get_internal_type()`: `float()` for FloatField, `int()` for any field ending in 'IntegerField', `Decimal()` for DecimalField, or the `_convert_value_noop` static method which simply returns the value unchanged. The `__getstate__` method explicitly pops `convert_value` from state so it is not included when the object's state is captured.\n\n**The `get_db_converters` method** checks if `self.convert_value is self._convert_value_noop` \u2014 if it is, it returns an empty list plus the output field's converters; otherwise it prepends its own `convert_value` to the output field's converters.\n\n**The `@deconstructible` decorator** (from `django/utils/deconstruct.py`) adds a custom `__new__` that stores `_constructor_args = (args, kwargs)` and a `deconstruct()` method returning a 3-tuple of (import path, args, kwargs).\n\n**Expression identity:** The `Expression` class uses a `_constructor_signature` classproperty (cached with `@functools.lru_cache(maxsize=128)`) that calls `signature(cls.__init__)` from `django.utils.inspect`. The `identity` cached_property binds partial constructor arguments, skips 'self', and builds a tuple of `[class, (arg_name, value), ...]` for use in `__eq__` and `__hash__`.",
    "rubric": [
      "Lists the 8 class-level attributes of BaseExpression: empty_result_set_value=NotImplemented, is_summary=False, filterable=True, window_compatible=False, allowed_default=False, constraint_validation_compatible=True, set_returning=False, allows_composite_expressions=False",
      "Identifies that Expression inherits from both BaseExpression and Combinable, and is decorated with @deconstructible",
      "Identifies that Query (in the SQL module) also inherits from BaseExpression",
      "Explains the convert_value cached_property: returns type-specific converter lambdas for FloatField (float), IntegerField (int), DecimalField (Decimal), or the _convert_value_noop static method for other types",
      "Explains that __getstate__ pops 'convert_value' from the instance state to avoid serializing the cached_property",
      "Explains get_db_converters checks if convert_value is _convert_value_noop to decide whether to include it in the converter list alongside output_field converters",
      "Describes the @deconstructible decorator: adds __new__ that stores _constructor_args=(args, kwargs) and a deconstruct() method returning a 3-tuple of (import path, args, kwargs)",
      "Describes Expression's identity mechanism: _constructor_signature uses lru_cache and signature(cls.__init__); the identity cached_property binds constructor args, skips 'self', and builds a tuple used for __eq__ and __hash__"
    ],
    "key_files": [
      "django/db/models/expressions.py",
      "django/db/models/sql/compiler.py",
      "django/db/models/sql/query.py",
      "django/utils/deconstruct.py"
    ],
    "source_doc": "[docstring: django/db/models/expressions.py] django.db.models.expressions.BaseExpression\ndjango.db.models.expressions.BaseExpression:\n  Base class for all query expressions.\n\ndjango.db.models.expressions.BaseExpression.as_sql:\n  Responsible for returning a (sql, [params]) tuple to be included\n        in the current query.\n\n        Different backends can provide their own implementation, by\n        providing an `as_{vendor}` method and patching the Expression:\n\n        ```\n        def override_as_sql(self, compiler, connection):\n            # custom logic\n            return super().as_sql(compiler, connection)\n        setattr(Expression, 'as_' + connection.vendor, override_as_sql)\n        ```\n\n        Arguments:\n         * compiler: the query compiler responsible for generating the query.\n           Must have a compile method, returning a (sql, [params]) tuple.\n           Calling compiler(value) will return a quoted `value`.\n\n         * connection: the database connection used for the current query.\n\n        Return: (sql, params)\n          Where `sql` is a string containing ordered sql parameters to be\n          replaced with the elements of the list `params`.\n\ndjango.db.models.expressions.BaseExpression.resolve_expression:\n  Provide the chance to do any preprocessing or validation before being\n        added to the query.\n\n        Arguments:\n         * query: the backend query implementation\n         * allow_joins: boolean allowing or denying use of joins\n           in this query\n         * reuse: a set of reusable joins for multijoins\n         * summarize: a terminal aggregate clause\n         * for_save: whether this expression about to be used in a save or\n           update\n\n        Return: an Expression to be added to the query.\n\ndjango.db.models.expressions.BaseExpression.output_field:\n  Return the output type of this expressions.\n\ndjango.db.models.expressions.BaseExpression._output_field_or_none:\n  Return the output field of this expression, or None if\n        _resolve_output_field() didn't return an output type.\n\ndjango.db.models.expressions.BaseExpression._resolve_output_field:\n  Attempt to infer the output type of the expression.\n\n        As a guess, if the output fields of all source fields match then simply\n        infer the same type here.\n\n        If a source's output field resolves to None, exclude it from this\n        check. If all sources are None, then an error is raised higher up the\n        stack in the output_field property.\n\ndjango.db.models.expressions.BaseExpression.convert_value:\n  Expressions provide their own converters because users have the option\n        of manually specifying the output_field which may be a different type\n        from the one the database returns.\n\ndjango.db.models.expressions.BaseExpression.get_source_fields:\n  Return the underlying field types used by this aggregate.\n\ndjango.db.models.expressions.BaseExpression.flatten:\n  Recursively yield this expression and all subexpressions, in\n        depth-first order.\n\ndjango.db.models.expressions.BaseExpression.select_format:\n  Custom format for select clauses. For example, EXISTS expressions need\n        to be wrapped in CASE WHEN on Oracle.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 10 class-level attributes but only lists 8 explicitly (empty_result_set_value, is_summary, filterable, window_compatible, allowed_default, constraint_validation_compatible, set_returning, allows_composite_expressions).",
      "Many specific implementation details (line numbers, exact attribute values, decorator details, lru_cache maxsize, _connector_combinators, etc.) go far beyond what the documentation provides and cannot be verified from it. These appear to be sourced from actual code inspection rather than the provided documentation.",
      "The description of convert_value as a 'cached_property' returning 'type-specific converter lambdas' goes beyond the documentation, which only says 'Expressions provide their own converters because users have the option of manually specifying the output_field which may be a different type from the one the database returns.' No contradiction, but unverifiable from the doc.",
      "The claim about compiler.compile() dispatching at 'line 573' is unverifiable from the documentation, though the vendor-specific dispatch mechanism is described in the as_sql docstring.",
      "No claims directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; minor doc-leakage in design rationale phrases and claims about truncated code sections.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_14",
    "repo": "django",
    "question": "Why does the `@register` decorator use local imports and its own validation layer instead of relying solely on `AdminSite.register()`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `register` decorator in `django/contrib/admin/decorators.py` has several design decisions visible in the code:\n\n**1. Local imports inside the function body:** The decorator uses local imports inside the function body for `ModelAdmin`, `AdminSite`, and the default `site` object. These imports are placed inside `register()` rather than at module level, which avoids circular import issues given the cross-dependencies between `decorators.py`, `options.py`, and `sites.py`.\n\n**2. Three-layered validation before registration:** The inner `_model_admin_wrapper` performs three defensive checks before delegating to `admin_site.register()`:\n  - Verifies at least one model was passed (`if not models`), raising `ValueError`\n  - Verifies the site is an `AdminSite` instance via `isinstance(admin_site, AdminSite)`, raising `ValueError`\n  - Verifies the decorated class is a `ModelAdmin` subclass via `issubclass(admin_class, ModelAdmin)`, raising `ValueError`\n\nThe decorator has a different call signature than `AdminSite.register()`. The site's `register()` method has its own validation (checking for abstract models, composite PKs, already-registered models, and swapped models), while the decorator adds checks specific to its usage context.\n\n**3. The default site is a `DefaultAdminSite` (a `LazyObject`):** The `site` imported as `default_site` is actually a `DefaultAdminSite` instance at module level in `sites.py`, which is a `LazyObject`. It only instantiates the actual `AdminSite` when first accessed (via `_setup()`), reading the `default_site` attribute from the app config (which defaults to `\"django.contrib.admin.sites.AdminSite\"` in `SimpleAdminConfig`).\n\n**4. The decorator returns the admin class unchanged:** `_model_admin_wrapper` returns `admin_class` after registration.\n\n**5. Delegation to `AdminSite.register()` passes models as a tuple:** The decorator calls `admin_site.register(models, admin_class=admin_class)` where `models` is the `*models` tuple from the decorator arguments. The `AdminSite.register()` method handles this by checking if the input is a `ModelBase` instance \u2014 if so it wraps it in a list; otherwise it iterates over the iterable. This means the decorator can accept multiple models in one call (e.g., `@register(Author, Book)`).\n\n**6. The `site or default_site` fallback** uses Python's truthiness, which works because `DefaultAdminSite` is a `LazyObject` that proxies all attribute access to the wrapped `AdminSite` instance. If a user passes `site=None` explicitly or omits it, the global default site is used. The `isinstance` check still works correctly with the `LazyObject`-based `DefaultAdminSite` because `LazyObject` is designed to proxy operations to the wrapped instance.",
    "rubric": [
      "Explains that local/deferred imports of ModelAdmin, AdminSite, and the default site inside the function body avoid circular import issues between decorators.py, options.py, and sites.py",
      "Identifies the three validation checks in the decorator: at least one model must be passed, the site must be an AdminSite instance, and the decorated class must be a ModelAdmin subclass \u2014 all raising ValueError",
      "Notes that AdminSite.register() has its own separate validations (abstract models, composite PKs, already-registered models, swapped models) which serve different purposes than the decorator's checks",
      "Explains that the default site is a DefaultAdminSite (a LazyObject) that defers actual AdminSite instantiation until first access via _setup(), and the 'site or default_site' fallback uses this lazy proxy",
      "Mentions that the decorator returns the admin class unchanged after registration and passes models as a tuple to admin_site.register()"
    ],
    "key_files": [
      "django/contrib/admin/decorators.py",
      "django/contrib/admin/sites.py",
      "django/contrib/admin/apps.py",
      "tests/admin_registration/tests.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/decorators.py] django.contrib.admin.decorators.register\nRegister the given model(s) classes and wrapped ModelAdmin class with\n    admin site:\n\n    @register(Author)\n    class AuthorAdmin(admin.ModelAdmin):\n        pass\n\n    The `site` kwarg is an admin site to use instead of the default admin site.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a brief docstring about the `register` decorator's purpose and usage. The generated answer makes extensive claims about implementation details (line numbers, lazy imports, three-layered validation, DefaultAdminSite as LazyObject, etc.) that cannot be verified from the provided documentation alone.",
      "While the claims about implementation details are plausible extrapolations based on Django's actual source code, they go far beyond what the documentation snippet supports and include specific line numbers that could be inaccurate.",
      "The core claim that `register` registers model(s) with an admin site and accepts a `site` kwarg is supported by the documentation.",
      "The claim about multiple models being supported (e.g., `@register(Author, Book)`) is consistent with the docstring mentioning 'model(s) classes'."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in design rationale claims about why patterns were chosen; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_13",
    "repo": "django",
    "question": "What is the class hierarchy and integration mechanism for `BaseModelAdminChecks` in Django's admin system checks?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `BaseModelAdminChecks` class in `django/contrib/admin/checks.py` is a plain class (no parent class - it doesn't inherit from anything) that serves as the base class for Django's admin system validation checks. Here's how it fits into the broader architecture:\n\n**Class Hierarchy:**\n- `BaseModelAdminChecks` (base, no parent)\n  - `ModelAdminChecks(BaseModelAdminChecks)` \u2014 adds checks for `save_as`, `save_on_top`, `inlines`, `list_display`, `list_display_links`, `list_filter`, `list_select_related`, `list_per_page`, `list_max_show_all`, `list_editable`, `search_fields`, `date_hierarchy`, and `actions`\n  - `InlineModelAdminChecks(BaseModelAdminChecks)` \u2014 adds checks for `_check_relation`, `_check_exclude_of_parent_model`, `_check_extra`, `_check_max_num`, `_check_min_num`, and `_check_formset`\n\n**Integration with ModelAdmin classes:**\nThe checks classes are instantiated and invoked through a `checks_class` attribute on admin option classes:\n- `BaseModelAdmin` (line 149 of options.py) sets `checks_class = BaseModelAdminChecks`\n- `ModelAdmin` (line 674) overrides it to `checks_class = ModelAdminChecks`\n- `InlineModelAdmin` (line 2413) overrides it to `checks_class = InlineModelAdminChecks`\n- The `check()` method on `BaseModelAdmin` simply calls `self.checks_class().check(self, **kwargs)` \u2014 instantiating the checks class and passing the admin object (`self`) as the first argument.\n\n**System check registration:**\nTwo functions \u2014 `check_admin_app` and `check_dependencies` \u2014 are registered as system checks in `django/contrib/admin/apps.py`'s `SimpleAdminConfig.ready()` with tag `checks.Tags.admin`. `check_admin_app` iterates over `all_sites` (a `WeakSet` from `django/contrib/admin/sites.py`) and calls `site.check(app_configs)`. The `AdminSite.check()` method (sites.py line 77) skips `ModelAdmin` instances that are not subclassed (`o.__class__ is not ModelAdmin`) and calls `modeladmin.check()` for each customized admin.\n\n**Key implementation details:**\n1. `BaseModelAdminChecks.check()` also calls `_check_view_on_site_url(admin_obj)` (error id `admin.E025`), which is included in the check method's aggregated list.\n2. The `_check_ordering_item` method handles `Combinable` and `OrderBy` expressions \u2014 if the field is a `Combinable` (but not `OrderBy`), it calls `.asc()` to convert it, then extracts the name from `F` expressions. It also uses `str.removeprefix(\"-\")` to strip descending order prefixes, and specially handles `\"pk\"` by returning no errors.\n3. The `_check_prepopulated_fields_key` method disallows `DateTimeField`, `ForeignKey`, and `ManyToManyField` via `isinstance()` check \u2014 notably, `OneToOneField` is implicitly covered because it's a subclass of `ForeignKey` in Django's ORM.\n4. The `_check_raw_id_fields_item` method explicitly checks `field.name != field_name` to reject the use of `attname` (e.g., `\"field_id\"` instead of `\"field\"`).\n5. The `_check_filter_item` method (for `filter_vertical`/`filter_horizontal`) also checks for `ManyToManyRel` (reverse side) specifically rejecting it, and additionally verifies that the M2M field doesn't have a manually specified through model (`field.remote_field.through._meta.auto_created`).\n6. The `_check_radio_fields_value` method imports `HORIZONTAL` and `VERTICAL` from `django.contrib.admin.options`, which are defined as `HORIZONTAL, VERTICAL = 1, 2`.\n7. The three helper functions at module level \u2014 `must_be()`, `must_inherit_from()`, and `refer_to_missing_field()` \u2014 all return a single-element list containing a `checks.Error` and are used by the checks methods to produce consistent error messages. The `refer_to_missing_field` function uses `obj.model._meta.label` for the model name in the error message.\n8. The `_check_field_spec_item` method allows non-model fields to pass validation silently (returns `[]`) if they can't be found via `_meta.get_field()`.",
    "rubric": [
      "Identifies that BaseModelAdminChecks is a plain class with no parent/inheritance",
      "Names ModelAdminChecks and InlineModelAdminChecks as its two subclasses",
      "Explains the checks_class attribute pattern: BaseModelAdmin sets checks_class = BaseModelAdminChecks, ModelAdmin overrides to ModelAdminChecks, InlineModelAdmin overrides to InlineModelAdminChecks",
      "Explains that BaseModelAdmin.check() instantiates checks_class and calls .check(self, **kwargs) on it",
      "Describes system check registration: check_admin_app and check_dependencies are registered in SimpleAdminConfig.ready() with Tags.admin",
      "Mentions that AdminSite.check() skips non-customized ModelAdmin instances (o.__class__ is not ModelAdmin) when running checks",
      "Lists key checks in BaseModelAdminChecks.check(): autocomplete_fields, raw_id_fields, fields, fieldsets, exclude, form, filter_vertical, filter_horizontal, radio_fields, prepopulated_fields, view_on_site_url, ordering, readonly_fields",
      "Lists additional checks added by ModelAdminChecks: save_as, save_on_top, inlines, list_display, list_display_links, list_filter, list_select_related, list_per_page, list_max_show_all, list_editable, search_fields, date_hierarchy, actions",
      "Lists additional checks added by InlineModelAdminChecks: _check_relation, _check_exclude_of_parent_model, _check_extra, _check_max_num, _check_min_num, _check_formset",
      "Mentions the module-level helper functions must_be(), must_inherit_from(), and refer_to_missing_field() that return single-element lists of checks.Error"
    ],
    "key_files": [
      "django/contrib/admin/checks.py",
      "django/contrib/admin/options.py",
      "django/contrib/admin/sites.py",
      "django/contrib/admin/apps.py",
      "django/contrib/admin/utils.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/checks.py] django.contrib.admin.checks.BaseModelAdminChecks\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_autocomplete_fields:\n  Check that `autocomplete_fields` is a list or tuple of model fields.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_autocomplete_fields_item:\n  Check that an item in `autocomplete_fields` is a ForeignKey or a\n        ManyToManyField and that the item has a related ModelAdmin with\n        search_fields defined.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_raw_id_fields:\n  Check that `raw_id_fields` only contains field names that are listed\n        on the model.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_raw_id_fields_item:\n  Check an item of `raw_id_fields`, i.e. check that field named\n        `field_name` exists in model `model` and is a ForeignKey or a\n        ManyToManyField.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_fields:\n  Check that `fields` only refer to existing fields, doesn't contain\n        duplicates. Check if at most one of `fields` and `fieldsets` is\n        defined.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_fieldsets:\n  Check that fieldsets is properly formatted and doesn't contain\n        duplicates.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_fieldsets_item:\n  Check an item of `fieldsets`, i.e. check that this is a pair of a\n        set name and a dictionary containing \"fields\" key.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_field_spec:\n  `fields` should be an item of `fields` or an item of\n        fieldset[1]['fields'] for any `fieldset` in `fieldsets`. It should be a\n        field name or a tuple of field names.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_exclude:\n  Check that exclude is a sequence without duplicates.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_form:\n  Check that form subclasses BaseModelForm.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_filter_vertical:\n  Check that filter_vertical is a sequence of field names.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_filter_horizontal:\n  Check that filter_horizontal is a sequence of field names.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_filter_item:\n  Check one item of `filter_vertical` or `filter_horizontal`, i.e.\n        check that given field exists and is a ManyToManyField.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_radio_fields:\n  Check that `radio_fields` is a dictionary.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_radio_fields_key:\n  Check that a key of `radio_fields` dictionary is name of existing\n        field and that the field is a ForeignKey or has `choices` defined.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_radio_fields_value:\n  Check type of a value of `radio_fields` dictionary.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_prepopulated_fields:\n  Check that `prepopulated_fields` is a dictionary containing allowed\n        field types.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_prepopulated_fields_key:\n  Check a key of `prepopulated_fields` dictionary, i.e. check that it\n        is a name of existing field and the field is one of the allowed types.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_prepopulated_fields_value:\n  Check a value of `prepopulated_fields` dictionary, i.e. it's an\n        iterable of existing fields.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_prepopulated_fields_value_item:\n  For `prepopulated_fields` equal to {\"slug\": (\"title\",)},\n        `field_name` is \"title\".\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_ordering:\n  Check that ordering refers to existing fields or is random.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_ordering_item:\n  Check that `ordering` refers to existing fields.\n\ndjango.contrib.admin.checks.BaseModelAdminChecks._check_readonly_fields:\n  Check that readonly_fields refers to proper attribute or field.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims BaseModelAdminChecks has 'no parent class - it doesn't inherit from anything'. This is plausible but not stated in the documentation.",
      "The answer describes ModelAdminChecks and InlineModelAdminChecks as subclasses with specific additional checks - this is not in the documentation but is plausible knowledge about Django's codebase.",
      "The answer provides specific line numbers (e.g., 'line 149 of options.py', 'line 674', 'line 2413', 'sites.py line 77') which cannot be verified from the documentation provided.",
      "The answer mentions _check_view_on_site_url, check_admin_app, check_dependencies, SimpleAdminConfig.ready(), AdminSite.check(), and other implementation details not present in the documentation - these are plausible but unverifiable from the given doc.",
      "Claims about _check_prepopulated_fields_key disallowing DateTimeField, ForeignKey, and ManyToManyField are consistent with the doc saying it checks 'allowed field types' but the specific types are not documented here.",
      "The claim about _check_filter_item checking for ManyToManyRel and through model auto_created goes beyond but doesn't contradict the doc which says it checks 'that given field exists and is a ManyToManyField'.",
      "The claim about HORIZONTAL, VERTICAL = 1, 2 is plausible Django knowledge but not in the documentation.",
      "The claim about must_be(), must_inherit_from(), and refer_to_missing_field() helper functions is not in the documentation but is plausible.",
      "The claim about _check_field_spec_item allowing non-model fields - the documentation mentions _check_field_spec but not _check_field_spec_item specifically, though such a method could exist."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; minor rewrites needed for documentation-derived rationale about purpose and assumptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_12",
    "repo": "django",
    "question": "How does Django's SQLCompiler class generate SQL from ORM queries, including vendor dispatch, GROUP BY optimization, and result handling?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `SQLCompiler` class in `django/db/models/sql/compiler.py` is the core SQL generation engine in Django's ORM. Here's how it works in detail, based on code analysis:\n\n## Architecture & Initialization\n\n`SQLCompiler.__init__` takes `query`, `connection`, `using`, and `elide_empty=True`. It initializes `quote_cache` with `{\"*\": \"*\"}` as a memoization seed, and sets `select`, `annotation_col_map`, `klass_info`, and `_meta_ordering` all to `None`.\n\nCompilers are instantiated via `Query.get_compiler()` (query.py line 374), which calls `connection.ops.compiler(self.compiler)(self, connection, using, elide_empty)`. The `Query.compiler` class attribute is the string `\"SQLCompiler\"`, and subclasses like `SQLInsertCompiler`, `SQLDeleteCompiler`, `SQLUpdateCompiler`, and `SQLAggregateCompiler` each extend it.\n\n## The `compile()` Method \u2014 Vendor Dispatch\n\nThe `compile()` method (line 573) implements a vendor-specific dispatch pattern: it checks for `as_<vendor>` method on the node (e.g., `as_postgresql`, `as_mysql`) via `getattr(node, \"as_\" + self.connection.vendor, None)`. If found, it calls that; otherwise falls back to `node.as_sql(self, self.connection)`.\n\n## `pre_sql_setup()` \u2014 The Orchestration Method\n\n`pre_sql_setup()` (line 82) orchestrates the SQL build process:\n1. Calls `setup_query()` which calls `get_select()` and sets `self.select`, `self.klass_info`, `self.annotation_col_map`, and `self.col_count`\n2. Calls `get_order_by()` to compute ordering\n3. Splits `self.query.where` into `self.where`, `self.having`, and `self.qualify` via `split_having_qualify()` (from `where.py`), passing `must_group_by=self.query.group_by is not None`\n4. Computes `extra_select` via `get_extra_select()` (needed for DISTINCT + ORDER BY compatibility)\n5. Computes `group_by` from `self.select + extra_select` and `order_by`\n\n## `PositionRef` \u2014 Order By Position Optimization\n\nA helper class `PositionRef(Ref)` (line 33) is defined to support ordering by SELECT clause position. Its `as_sql()` simply returns `str(self.ordinal), ()`. In `_order_by_pairs()`, when ordering fields match selected expressions, they're replaced with `PositionRef` instances containing the 1-based ordinal position.\n\n## `get_group_by()` \u2014 Functional Dependency Optimization\n\nThe `collapse_group_by()` method (line 201) implements a significant optimization: when `connection.features.allows_group_by_selected_pks` is True, it reduces GROUP BY expressions by identifying table primary keys in the grouped columns. Non-PK columns whose alias matches a table with a present PK are removed. This optimization is gated by `allows_group_by_selected_pks_on_model()` which checks `model._meta.managed` \u2014 unmanaged models (potentially database views) are excluded.\n\nAdditionally, `get_group_by()` supports group-by-index: when `connection.features.allows_group_by_select_index` is True, expressions that match a selected aliased expression are replaced with their ordinal position number.\n\n## `as_sql()` \u2014 The Main SQL Generator\n\n`as_sql()` (line 756) saves `alias_refcount` before generating SQL and resets it in a `finally` block via `query.reset_refcounts()`. Key implementation details:\n\n- For combinator queries (UNION, INTERSECT, DIFFERENCE), it checks `features.supports_select_{combinator}` and delegates to `get_combinator_sql()`\n- For QUALIFY (window function filtering), it delegates to `get_qualify_sql()` and sets `order_by = None` since qualify handles ordering separately\n- The FOR UPDATE clause placement depends on `features.for_update_after_from` \u2014 some backends (MySQL) need it after FROM, others after LIMIT/OFFSET\n- When `from_` is empty (no tables), it appends `connection.features.bare_select_suffix` (Oracle < 23 uses `\" FROM DUAL\"`)\n- When `_meta_ordering` is set and grouping exists, `order_by` is set to `None` to suppress meta ordering in grouped queries\n- If `having` exists without `grouping`, it calls `connection.ops.force_group_by()` to inject a GROUP BY clause\n- Subqueries with extra_select columns get wrapped in another `SELECT ... FROM (...) subquery` to hide extraneous columns\n\n## `execute_sql()` \u2014 Result Type Handling\n\nThe `execute_sql()` method (line 1594) defaults `result_type` to `MULTI`. A falsy value is converted to `NO_RESULTS`. On `EmptyResultSet`, MULTI returns `iter([])` and all others return `None`. The five result types are string constants: `MULTI=\"multi\"`, `SINGLE=\"single\"`, `NO_RESULTS=\"no results\"`, `CURSOR=\"cursor\"`, `ROW_COUNT=\"row count\"`.\n\nFor MULTI results, `cursor_iter()` (line 2257) is a generator that uses `cursor.fetchmany(itersize)` with `connection.features.empty_fetchmany_value` as the sentinel. When `has_extra_select` is True, rows are truncated to `col_count` columns. Non-chunked fetches materialize results into a `list()`.\n\n## `results_iter()` \u2014 Post-Processing Pipeline\n\n`results_iter()` (line 1564) builds a processing pipeline:\n1. Execute SQL with MULTI result type\n2. Extract field expressions from `self.select[0:self.col_count]`\n3. Apply database converters via `get_converters()` (which handles `ColPairs` recursively)\n4. Handle composite fields via `composite_fields_to_tuples()` converting slices to tuples\n5. Optionally convert rows to tuples if `tuple_expected=True`\n\n## `get_qualify_sql()` \u2014 Window Function Filtering\n\nWhen filtering involves window functions, `get_qualify_sql()` wraps the query as `SELECT * FROM (inner_query) qualify WHERE qualify_condition`. If additional aliases were introduced for qualify filtering, a second wrapping occurs: `SELECT cols FROM (SELECT * FROM (...) qualify WHERE ...) qualify_mask`.\n\n## Backend-Specific Subclasses\n\nMySQL's `SQLDeleteCompiler` overrides `as_sql()` to use `DELETE table FROM table1, table2 WHERE ...` syntax instead of subqueries for multi-table deletes. MySQL's `SQLUpdateCompiler` adds `ORDER BY` support to UPDATE statements. PostgreSQL's `SQLInsertCompiler` overrides `assemble_as_sql()` to use `UNNEST` for bulk inserts when possible (must have >1 row, no field with `get_placeholder_sql`, all standard internal types, and no compilable values).\n\n## Deprecation\n\n`quote_name_unless_alias()` is deprecated with `RemovedInDjango70Warning` and simply delegates to `quote_name()`.",
    "rubric": [
      "Explains that compile() checks for vendor-specific as_<vendor> methods on nodes before falling back to as_sql()",
      "Describes pre_sql_setup() orchestrating setup_query/get_select, get_order_by, split_having_qualify, get_extra_select, and get_group_by",
      "Explains the collapse_group_by optimization that removes non-PK columns when a table's PK is already grouped (gated by allows_group_by_selected_pks and managed model check)",
      "Describes PositionRef class and how order-by or group-by expressions matching selected expressions get replaced with ordinal positions",
      "Explains as_sql() handling of combinator queries, qualify (window function filtering), FOR UPDATE placement, bare_select_suffix, and subquery wrapping for extra_select",
      "Describes execute_sql() result type handling (MULTI returns iter/list, SINGLE fetches one row truncated to col_count, CURSOR returns cursor, ROW_COUNT returns rowcount, NO_RESULTS closes cursor)",
      "Explains results_iter() pipeline: execute SQL, get converters, apply converters, handle composite fields, optionally convert to tuples",
      "Mentions cursor_iter uses fetchmany with itersize and truncates rows to col_count when has_extra_select is True"
    ],
    "key_files": [
      "django/db/models/sql/compiler.py",
      "django/db/models/sql/query.py",
      "django/db/models/sql/constants.py",
      "django/db/models/sql/where.py",
      "django/db/backends/mysql/compiler.py",
      "django/db/backends/postgresql/compiler.py",
      "django/db/models/expressions.py"
    ],
    "source_doc": "[docstring: django/db/models/sql/compiler.py] django.db.models.sql.compiler.SQLCompiler\ndjango.db.models.sql.compiler.SQLCompiler.pre_sql_setup:\n  Do any necessary class setup immediately prior to producing SQL. This\n        is for things that can't necessarily be done in __init__ because we\n        might not have all the pieces in place at that time.\n\ndjango.db.models.sql.compiler.SQLCompiler.get_group_by:\n  Return a list of 2-tuples of form (sql, params).\n\n        The logic of what exactly the GROUP BY clause contains is hard\n        to describe in other words than \"if it passes the test suite,\n        then it is correct\".\n\ndjango.db.models.sql.compiler.SQLCompiler.get_select:\n  Return three values:\n        - a list of 3-tuples of (expression, (sql, params), alias)\n        - a klass_info structure,\n        - a dictionary of annotations\n\n        The (sql, params) is what the expression will produce, and alias is the\n        \"AS alias\" for the column (possibly None).\n\n        The klass_info structure contains the following information:\n        - The base model of the query.\n        - Which columns for that model are present in the query (by\n          position of the select clause).\n        - related_klass_infos: [f, klass_info] to descent into\n\n        The annotations is a dictionary of {'attname': column position} values.\n\ndjango.db.models.sql.compiler.SQLCompiler.get_order_by:\n  Return a list of 2-tuples of the form (expr, (sql, params, is_ref)) for\n        the ORDER BY clause.\n\n        The order_by clause can alter the select clause (for example it can add\n        aliases to clauses that do not yet have one, or it can add totally new\n        select clauses).\n\ndjango.db.models.sql.compiler.SQLCompiler.quote_name:\n  A wrapper around connection.ops.quote_name that memoizes quoted\n        name values.\n\ndjango.db.models.sql.compiler.SQLCompiler.as_sql:\n  Create the SQL for this query. Return the SQL string and list of\n        parameters.\n\n        If 'with_limits' is False, any limit/offset information is not included\n        in the query.\n\ndjango.db.models.sql.compiler.SQLCompiler.get_default_columns:\n  Compute the default columns for selecting every field in the base\n        model. Will sometimes be called to pull in related models (e.g. via\n        select_related), in which case \"opts\" and \"start_alias\" will be given\n        to provide a starting point for the traversal.\n\n        Return a list of strings, quoted appropriately for use in SQL\n        directly, as well as a set of aliases used in the select statement (if\n        'as_pairs' is True, return a list of (alias, col_name) pairs instead\n        of strings as the first component and None as the second component).\n\ndjango.db.models.sql.compiler.SQLCompiler.get_distinct:\n  Return a quoted list of fields to use in DISTINCT ON part of the query.\n\n        This method can alter the tables in the query, and thus it must be\n        called before get_from_clause().\n\ndjango.db.models.sql.compiler.SQLCompiler.find_ordering_name:\n  Return the table alias (the name might be ambiguous, the alias will\n        not be) and column name for ordering by the given 'name' parameter.\n        The 'name' is of the form 'field1__field2__...__fieldN'.\n\ndjango.db.models.sql.compiler.SQLCompiler._setup_joins:\n  Helper method for get_order_by() and get_distinct().\n\n        get_ordering() and get_distinct() must produce same target columns on\n        same input, as the prefixes of get_ordering() and get_distinct() must\n        match. Executing SQL where this is not true is an error.\n\ndjango.db.models.sql.compiler.SQLCompiler.get_from_clause:\n  Return a list of strings that are joined together to go after the\n        \"FROM\" part of the query, as well as a list any extra parameters that\n        need to be included. Subclasses, can override this to create a\n        from-clause via a \"select\".\n\n        This should only be called after any SQL construction methods that\n        might change the tables that are needed. This means the select columns,\n        ordering, and distinct must be done first.\n\ndjango.db.models.sql.compiler.SQLCompiler.get_related_selections:\n  Fill in the information needed for a select_related query. The current\n        depth is measured as the number of connections away from the root model\n        (for example, cur_depth=1 means we are looking at models with direct\n        connections to the root model).\n\ndjango.db.models.sql.compiler.SQLCompiler.get_select_for_update_of_arguments:\n  Return a quoted list of arguments for the SELECT FOR UPDATE OF part of\n        the query.\n\ndjango.db.models.sql.compiler.SQLCompiler.results_iter:\n  Return an iterator over the results from executing this query.\n\ndjango.db.models.sql.compiler.SQLCompiler.has_results:\n  Backends (e.g. NoSQL) can override this in order to use optimized\n        versions of \"query has any results.\"\n\ndjango.db.models.sql.compiler.SQLCompiler.execute_sql:\n  Run the query against the database and return the result(s). The\n        return value depends on the value of result_type.\n\n        When result_type is:\n        - MULTI: Retrieves all rows using fetchmany(). Wraps in an iterator for\n           chunked reads when supported.\n        - SINGLE: Retrieves a single row using fetchone().\n        - ROW_COUNT: Retrieves the number of rows in the result.\n        - CURSOR: Runs the query, and returns the cursor object. It is the\n           caller's responsibility to close the cursor.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation describes `get_default_columns` as returning 'a list of strings, quoted appropriately for use in SQL directly, as well as a set of aliases used in the select statement', but the generated answer does not mention `get_default_columns` at all.",
      "The documentation describes `get_related_selections` as filling in information for select_related queries, but the generated answer does not mention this method.",
      "The documentation says `get_order_by` returns 2-tuples of form (expr, (sql, params, is_ref)), but the answer does not explicitly describe this return format.",
      "The documentation states `get_select` returns 'a list of 3-tuples of (expression, (sql, params), alias)' along with klass_info and annotations dict. The answer mentions `get_select()` but does not describe its return values in detail.",
      "The documentation describes `_setup_joins` as a helper for `get_order_by()` and `get_distinct()` ensuring same target columns on same input. The answer does not mention `_setup_joins`.",
      "The documentation states `get_from_clause` returns 'a list of strings that are joined together to go after the FROM part of the query, as well as a list any extra parameters'. The answer does not describe `get_from_clause` in detail.",
      "Many detailed claims in the answer (e.g., specific line numbers, `PositionRef` class, `collapse_group_by`, `get_qualify_sql`, MySQL/PostgreSQL backend details, `quote_name_unless_alias` deprecation, `cursor_iter` implementation details, etc.) go far beyond the provided documentation and cannot be verified from it \u2014 these are plausible code-level details but are not supported by the documentation provided.",
      "The answer claims `execute_sql` defaults `result_type` to `MULTI` \u2014 the documentation doesn't state this explicitly but the description of MULTI behavior is consistent.",
      "The answer mentions result type constants like `NO_RESULTS` and `CURSOR` with string values \u2014 the documentation mentions `CURSOR` result type but does not specify string constant values; `NO_RESULTS` is not mentioned in the documentation at all.",
      "The documentation describes ROW_COUNT as retrieving 'the number of rows in the result', and the answer does not contradict this."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_16",
    "repo": "django",
    "question": "How does SimpleListFilter handle initialization, facet counting, and choice generation in the admin filtering system?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `SimpleListFilter` class in `django/contrib/admin/filters.py` works as follows:\n\n**Inheritance Chain**: `SimpleListFilter` inherits from both `FacetsMixin` and `ListFilter` (in that MRO order: `class SimpleListFilter(FacetsMixin, ListFilter)`). The `ListFilter` base class requires subclasses to implement `has_output()`, `choices()`, `queryset()`, and `expected_parameters()` methods, enforced via `NotImplementedError`.\n\n**Initialization Flow**:\n1. In `__init__`, it first calls `super().__init__()` which sets `self.request`, initializes `self.used_parameters = {}`, and validates that `self.title` is not None (raising `ImproperlyConfigured` if it is).\n2. It then validates that `parameter_name` is set (also raising `ImproperlyConfigured` if None).\n3. If the `parameter_name` exists in the incoming `params` dict, it **pops** the value from params, takes the **last element** of the value list (`value[-1]`), and stores it in `self.used_parameters[self.parameter_name]`.\n4. It calls `self.lookups(request, model_admin)` \u2014 if this returns `None`, it's converted to an empty tuple, then cast to a list and stored as `self.lookup_choices`.\n\n**Value retrieval**: The `value()` method simply returns `self.used_parameters.get(self.parameter_name)`, which returns the stored string value or `None`.\n\n**Facet counting mechanism**: The `get_facet_counts()` method iterates through all `lookup_choices`, temporarily sets `self.used_parameters[self.parameter_name]` to each choice value, calls `self.queryset()` to get a filtered queryset for each choice, and builds `models.Count` aggregations using `models.Q(pk__in=lookup_qs)`. After iteration, it restores the original value. The parent `FacetsMixin.get_facet_queryset()` method obtains a queryset that **excludes** the current filter's parameters (via `changelist.get_queryset(self.request, exclude_parameters=self.expected_parameters())`), then calls `.aggregate()` with the counts dict.\n\n**Choices generation**: The `choices()` method is a generator that yields dicts with keys `selected`, `query_string`, and `display`. It first yields an \"All\" choice (selected when `self.value() is None`), then iterates through `lookup_choices`. Selection is determined by comparing `self.value() == str(lookup)`. When facets are enabled, counts are appended to display text, or `(-)` is shown if the count key is missing (which occurs when `self.queryset()` returned `None` for that choice in `get_facet_counts`, skipping the count entry).\n\n**Integration with ChangeList**: The `ChangeList.__init__` determines `add_facets` based on `model_admin.show_facets` (a `ShowFacets` enum with values NEVER, ALLOW, ALWAYS). In `get_filters()`, if a `list_filter` entry is callable, it's instantiated as `spec = list_filter(request, lookup_params, self.model, self.model_admin)`. The filter's `has_output()` method (returns `len(self.lookup_choices) > 0`) determines if it's included in `filter_specs`.\n\n**Template rendering**: The default template for filters is `\"admin/filter.html\"` as set in the `ListFilter` class attribute `template`. The `choices()` method provides the data (dicts with `selected`, `query_string`, and `display` keys) for rendering.",
    "rubric": [
      "Explains the inheritance chain: SimpleListFilter inherits from both FacetsMixin and ListFilter",
      "Describes initialization: validates title and parameter_name (raising ImproperlyConfigured if None), pops parameter from params dict taking the last element of the value list, and calls self.lookups() to populate self.lookup_choices",
      "Explains the value() method returns self.used_parameters.get(self.parameter_name)",
      "Describes get_facet_counts: iterates through lookup_choices, temporarily sets used_parameters to each choice value, calls self.queryset() for each, builds Count aggregations with Q(pk__in=lookup_qs), then restores the original value",
      "Explains FacetsMixin.get_facet_queryset obtains a queryset excluding the current filter's parameters and calls aggregate with the counts dict",
      "Describes choices() as a generator yielding dicts with 'selected', 'query_string', and 'display' keys, starting with an 'All' option, and appending facet counts when enabled",
      "Mentions how ChangeList determines add_facets based on ShowFacets enum (ALWAYS/ALLOW/NEVER) and how filters are instantiated in get_filters()"
    ],
    "key_files": [
      "django/contrib/admin/filters.py",
      "django/contrib/admin/views/main.py",
      "django/contrib/admin/options.py",
      "django/contrib/admin/utils.py",
      "django/contrib/admin/templatetags/admin_list.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/filters.py] django.contrib.admin.filters.SimpleListFilter\ndjango.contrib.admin.filters.SimpleListFilter.value:\n  Return the value (in string format) provided in the request's\n        query string for this filter, if any, or None if the value wasn't\n        provided.\n\ndjango.contrib.admin.filters.SimpleListFilter.lookups:\n  Must be overridden to return a list of tuples (value, verbose value)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor rewrites needed for design rationale phrases and template rendering details not present in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_15",
    "repo": "django",
    "question": "Where does the return value of `ListFilter.expected_parameters()` get consumed across the admin system, and what does each consumer do with it?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `ListFilter.expected_parameters()` method's return value flows through the system in two key locations:\n\n1. **In `ChangeList.get_queryset()` (django/contrib/admin/views/main.py)**: When `get_queryset` is called with an `exclude_parameters` argument, it iterates over all `self.filter_specs` and compares each filter's `expected_parameters()` return value against `exclude_parameters`. If they match, that filter's `queryset()` method is skipped (the filter is excluded from being applied to the queryset). This is specifically used by the facets system to get counts that exclude the current filter's own parameters.\n\n2. **In `FacetsMixin.get_facet_queryset()` (django/contrib/admin/filters.py)**: This method calls `changelist.get_queryset(self.request, exclude_parameters=self.expected_parameters())`, passing the filter's own expected parameters as the exclusion list. This creates a queryset that applies all filters EXCEPT the current one, which is then used to compute facet counts via `aggregate()`.\n\n3. **In `ChangeList.get_filters()` (django/contrib/admin/views/main.py)**: The `FieldListFilter.__init__` iterates over `self.expected_parameters()` to pop matching keys from the `lookup_params` dict (which comes from the request's query string). This is how filters consume their parameters from the shared params dict.\n\n4. **In `ChangeList.__init__`**: `self.queryset = self.get_queryset(request)` is called without `exclude_parameters`, meaning all filters apply their `queryset()` method.\n\nThe `has_output()` method is called in `ChangeList.get_filters()`: only filters where `spec.has_output()` returns True are appended to `filter_specs`. The `choices()` method is called in the `admin_list_filter` template tag (django/contrib/admin/templatetags/admin_list.py) which renders each filter using its `spec.template` (default: 'admin/filter.html').",
    "rubric": [
      "Mentions `ChangeList.get_queryset()` using `expected_parameters()` to compare against `exclude_parameters` and skip filters whose parameters match",
      "Mentions `FacetsMixin.get_facet_queryset()` passing `self.expected_parameters()` as `exclude_parameters` to `get_queryset()` to exclude the current filter from the queryset used for facet counts",
      "Mentions `FieldListFilter.__init__` iterating over `self.expected_parameters()` to pop matching keys from the lookup params dict",
      "Mentions `ChangeList.get_filters()` calling `spec.has_output()` to decide whether a filter is appended to `filter_specs`",
      "Mentions the `admin_list_filter` template tag calling `spec.choices(cl)` to render filter choices"
    ],
    "key_files": [
      "django/contrib/admin/filters.py",
      "django/contrib/admin/views/main.py",
      "django/contrib/admin/templatetags/admin_list.py",
      "django/contrib/admin/templates/admin/change_list.html"
    ],
    "source_doc": "[docstring: django/contrib/admin/filters.py] django.contrib.admin.filters.ListFilter\ndjango.contrib.admin.filters.ListFilter.has_output:\n  Return True if some choices would be output for this filter.\n\ndjango.contrib.admin.filters.ListFilter.choices:\n  Return choices ready to be output in the template.\n\n        `changelist` is the ChangeList to be displayed.\n\ndjango.contrib.admin.filters.ListFilter.queryset:\n  Return the filtered queryset.\n\ndjango.contrib.admin.filters.ListFilter.expected_parameters:\n  Return the list of parameter names that are expected from the\n        request's query string and that will be used by this filter.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes very specific line numbers (e.g., line 464-482, line 83-89, line 175-277, line 181-185, line 221, line 524, line 144) that cannot be verified from the provided documentation alone \u2014 these may be accurate based on the actual source code but are not supported by the given doc chunk.",
      "The answer references specific classes and methods like FacetsMixin.get_facet_queryset(), ChangeList.get_queryset(), ChangeList.get_filters(), FieldListFilter.__init__, and admin_list_filter template tag \u2014 none of which are mentioned in the provided documentation chunk. These are plausible code-level details but unverifiable from the doc.",
      "The answer describes behavior of facets system, exclude_parameters argument, and aggregate() calls \u2014 these are not mentioned in the documentation and represent extrapolations from the broader codebase.",
      "The core claims about what expected_parameters(), has_output(), choices(), and queryset() do are consistent with the documentation's descriptions."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the template rendering details about <li>/<details> elements cannot be verified, and line numbers are imprecise.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_18",
    "repo": "django",
    "question": "Why does BaseModelAdmin merge formfield_overrides with defaults instead of replacing them, and what other design patterns make it reusable across both ModelAdmin and InlineModelAdmin?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `BaseModelAdmin` class in `django/contrib/admin/options.py` is designed as the shared base for both `ModelAdmin` and `InlineModelAdmin`, using several deliberate design patterns:\n\n**1. Formfield override merging (not overwriting):** The `__init__` method uses `copy.deepcopy(FORMFIELD_FOR_DBFIELD_DEFAULTS)` and then merges user-provided `formfield_overrides` via `overrides.setdefault(k, {}).update(v)`. This merge pattern means subclass overrides add to or update the defaults rather than completely replacing them. This ensures the 12 default widget mappings (for DateTimeField, DateField, TimeField, TextField, URLField, IntegerField, BigIntegerField, CharField, ImageField, FileField, EmailField, UUIDField) are preserved unless explicitly overridden.\n\n**2. MRO-based formfield resolution:** In `formfield_for_dbfield`, for non-FK/M2M/choice fields, the code iterates through `db_field.__class__.mro()` to find the first matching class in `formfield_overrides`. This MRO walk ensures that if a specific field type (e.g., `EmailField`) is not in overrides but its parent (`CharField`) is, the parent's override will still apply.\n\n**3. `has_view_permission` implicitly grants view via change permission:** The `has_view_permission` method checks for *both* `view_<model>` and `change_<model>` permissions, returning True if *either* is present. The separate `has_view_or_change_permission` helper further OR's `has_view_permission` with `has_change_permission`.\n\n**4. RelatedFieldWidgetWrapper cascade protection:** In `formfield_for_dbfield`, when wrapping FK/M2M widgets with `RelatedFieldWidgetWrapper`, the wrapper constructor disables the delete button when `rel.on_delete is CASCADE` (via `cascade = getattr(rel, 'on_delete', None) is CASCADE` and `self.can_delete_related = supported and not cascade and can_delete_related`). Additionally, the `can_change_related`, `can_delete_related`, and `can_view_related` options are only enabled for single-select `Select` widgets, not for multi-select widgets.\n\n**5. `formfield_for_manytomany` returns None for custom through models:** The code checks `db_field.remote_field.through._meta.auto_created` and returns `None` if the through model is not auto-created, preventing admin from showing a form field for M2M relationships with explicit intermediary models.\n\n**6. `get_ordering` defensive pattern:** Returns `self.ordering or ()`, preventing unpacking errors when the result is used with `*ordering`.\n\n**7. `get_fields` defers to subclass `_get_form_for_get_fields`:** The base `get_fields` method calls `self._get_form_for_get_fields(request, obj)` which is not defined on `BaseModelAdmin` itself. `ModelAdmin` implements it as `self.get_form(request, obj, fields=None)` while `InlineModelAdmin` implements it as `self.get_formset(request, obj, fields=None).form`.\n\n**8. InlineModelAdmin permission escalation for auto-created through models:** `InlineModelAdmin` overrides all permission methods. For auto-created intermediate models (`self.opts.auto_created`), it uses `_has_any_perms_for_target_model` which finds the target model of the M2M relationship (not the parent) and checks permissions on *that* model. For add, change, and delete, it only requires the `change` permission on the target model. For view, it accepts either `view` or `change`.\n\n**9. `get_empty_value_display` uses exception-based fallback:** Rather than checking `hasattr`, it uses a try/except `AttributeError` to fall back from `self.empty_value_display` to `self.admin_site.empty_value_display`. `EMPTY_VALUE_STRING` is defined as `\"-\"`.\n\n**10. `BaseModelAdmin` uses `forms.MediaDefiningClass` as metaclass:** This metaclass (from `django/forms/widgets.py`) automatically adds a `media` property via `media_property(new_class)` if the class doesn't define its own `media` attribute. Both `ModelAdmin` and `InlineModelAdmin` define their own `media` property, so the metaclass's auto-generated one is bypassed.",
    "rubric": [
      "Explains that __init__ uses copy.deepcopy of FORMFIELD_FOR_DBFIELD_DEFAULTS and merges via setdefault/update to preserve default widget mappings while allowing additions",
      "Describes the MRO-based formfield resolution in formfield_for_dbfield that walks db_field.__class__.mro() to find matching overrides for parent field types",
      "Explains that has_view_permission checks both view and change permissions, granting implicit view access to users with change permission",
      "Describes how RelatedFieldWidgetWrapper disables delete button when on_delete is CASCADE and restricts change/delete/view to single-select Select widgets",
      "Explains that formfield_for_manytomany returns None for non-auto-created through models to prevent editing M2M with explicit intermediary models",
      "Mentions that get_fields defers to _get_form_for_get_fields which is implemented differently by ModelAdmin and InlineModelAdmin subclasses",
      "Explains InlineModelAdmin's permission escalation for auto-created through models using _has_any_perms_for_target_model which checks the target model's permissions rather than the intermediary model's",
      "Notes that BaseModelAdmin uses forms.MediaDefiningClass as its metaclass"
    ],
    "key_files": [
      "django/contrib/admin/options.py",
      "django/contrib/admin/widgets.py",
      "django/contrib/auth/__init__.py",
      "django/forms/widgets.py",
      "django/contrib/admin/checks.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/options.py] django.contrib.admin.options.BaseModelAdmin\ndjango.contrib.admin.options.BaseModelAdmin:\n  Functionality common to both ModelAdmin and InlineAdmin.\n\ndjango.contrib.admin.options.BaseModelAdmin.formfield_for_dbfield:\n  Hook for specifying the form Field instance for a given database Field\n        instance.\n\n        If kwargs are given, they're passed to the form Field's constructor.\n\ndjango.contrib.admin.options.BaseModelAdmin.formfield_for_choice_field:\n  Get a form Field for a database Field that has declared choices.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_field_queryset:\n  If the ModelAdmin specifies ordering, the queryset should respect that\n        ordering. Otherwise don't specify the queryset, let the field decide\n        (return None in that case).\n\ndjango.contrib.admin.options.BaseModelAdmin.formfield_for_foreignkey:\n  Get a form Field for a ForeignKey.\n\ndjango.contrib.admin.options.BaseModelAdmin.formfield_for_manytomany:\n  Get a form Field for a ManyToManyField.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_autocomplete_fields:\n  Return a list of ForeignKey and/or ManyToMany fields which should use\n        an autocomplete widget.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_empty_value_display:\n  Return the empty_value_display set on ModelAdmin or AdminSite.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_exclude:\n  Hook for specifying exclude.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_fields:\n  Hook for specifying fields.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_fieldsets:\n  Hook for specifying fieldsets.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_inlines:\n  Hook for specifying custom inlines.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_ordering:\n  Hook for specifying field ordering.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_readonly_fields:\n  Hook for specifying custom readonly fields.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_prepopulated_fields:\n  Hook for specifying custom prepopulated fields.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_queryset:\n  Return a QuerySet of all model instances that can be edited by the\n        admin site. This is used by changelist_view.\n\ndjango.contrib.admin.options.BaseModelAdmin.get_sortable_by:\n  Hook for specifying which fields can be sorted in the changelist.\n\ndjango.contrib.admin.options.BaseModelAdmin.to_field_allowed:\n  Return True if the model associated with this admin should be\n        allowed to be referenced by the specified field.\n\ndjango.contrib.admin.options.BaseModelAdmin.has_add_permission:\n  Return True if the given request has permission to add an object.\n        Can be overridden by the user in subclasses.\n\ndjango.contrib.admin.options.BaseModelAdmin.has_change_permission:\n  Return True if the given request has permission to change the given\n        Django model instance, the default implementation doesn't examine the\n        `obj` parameter.\n\n        Can be overridden by the user in subclasses. In such case it should\n        return True if the given request has permission to change the `obj`\n        model instance. If `obj` is None, this should return True if the given\n        request has permission to change *any* object of the given type.\n\ndjango.contrib.admin.options.BaseModelAdmin.has_delete_permission:\n  Return True if the given request has permission to delete the given\n        Django model instance, the default implementation doesn't examine the\n        `obj` parameter.\n\n        Can be overridden by the user in subclasses. In such case it should\n        return True if the given request has permission to delete the `obj`\n        model instance. If `obj` is None, this should return True if the given\n        request has permission to delete *any* object of the given type.\n\ndjango.contrib.admin.options.BaseModelAdmin.has_view_permission:\n  Return True if the given request has permission to view the given\n        Django model instance. The default implementation doesn't examine the\n        `obj` parameter.\n\n        If overridden by the user in subclasses, it should return True if the\n        given request has permission to view the `obj` model instance. If `obj`\n        is None, it should return True if the request has permission to view\n        any object of the given type.\n\ndjango.contrib.admin.options.BaseModelAdmin.has_module_permission:\n  Return True if the given request has any permission in the given\n        app label.\n\n        Can be overridden by the user in subclasses. In such case it should\n        return True if the given request has permission to view the module on\n        the admin index page and access the module's index page. Overriding it\n        does not restrict access to the add, change or delete views. Use\n        `ModelAdmin.has_(add|change|delete)_permission` for that.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer reference specific line numbers, internal implementation details, and code-level behaviors that go far beyond what the provided documentation describes. The documentation only contains docstrings/method descriptions, not source code.",
      "The claim about `has_view_permission` checking both 'view' and 'change' permissions is not contradicted by the docs but is an extrapolation beyond the docstring, which only says 'Return True if the given request has permission to view the given Django model instance.'",
      "Claims about `formfield_for_manytomany` returning None for custom through models, MRO-based formfield resolution, RelatedFieldWidgetWrapper cascade protection, specific line numbers, metaclass details, etc. are all implementation-level details not present in the provided documentation \u2014 they may be accurate but cannot be verified from the given doc chunk.",
      "The description of `BaseModelAdmin` as 'shared base for both ModelAdmin and InlineModelAdmin' is consistent with the doc saying 'Functionality common to both ModelAdmin and InlineAdmin.'",
      "Point 9 about InlineModelAdmin permission escalation references methods and behaviors not described in the provided documentation at all.",
      "The claim about `get_empty_value_display` using try/except is an implementation detail not in the docs; the doc only says 'Return the empty_value_display set on ModelAdmin or AdminSite.'",
      "No claims directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "A few claims relied on stripped comments (TODO note, XXX comment, backwards-compatibility rationale), but most claims are verifiable from the code structure and logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_20",
    "repo": "django",
    "question": "How does Django's admin `quote()` function escape characters in primary key values, and how does `unquote()` reverse it?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `quote()` function in `django/contrib/admin/utils.py` works by using Python's `str.translate()` method with a precomputed translation map (`QUOTE_MAP`). Here is the algorithm in detail:\n\n1. **QUOTE_MAP construction** (line 26): A dictionary comprehension iterates over each byte value in the byte string `b'\":/_#?;@&=+$,\"[]<>%\\n\\\\'`. For each byte value `i`, it creates a mapping from that integer (character ordinal) to a string formatted as `\"_%02X\" % i` \u2014 an underscore followed by the two-digit uppercase hexadecimal representation of the character's ordinal. For example:\n   - `/` (0x2F) \u2192 `_2F`\n   - `_` (0x5F) \u2192 `_5F`\n   - `:` (0x3A) \u2192 `_3A`\n   - `\\n` (0x0A) \u2192 `_0A`\n   - `%` (0x25) \u2192 `_25`\n\n2. **quote() function** (line 91-98): The function takes a single argument `s`. If `s` is a string (`isinstance(s, str)`), it calls `s.translate(QUOTE_MAP)` which replaces every character whose ordinal is a key in QUOTE_MAP with its corresponding `_XX` escape string. If `s` is not a string (e.g., an integer primary key), it returns `s` unchanged.\n\n3. **The escape format uses `_XX` instead of `%XX`** (as used by `urllib.parse.quote()`). This is the key difference in format.\n\n4. **The inverse `unquote()` function** (line 101-103) uses a precompiled regex `UNQUOTE_RE` that matches patterns like `_XX` (underscore followed by two hex digits that correspond to one of the originally quoted characters). It uses `UNQUOTE_MAP` (the inverse of QUOTE_MAP, mapping strings like `_2F` back to `/`) via a `re.sub()` with a lambda that looks up each match in `UNQUOTE_MAP`.\n\n5. **UNQUOTE_RE pattern** (line 28) is constructed as `_(?:22|3A|2F|5F|...)` \u2014 a non-capturing group of all the two-hex-digit suffixes from the QUOTE_MAP values, compiled lazily via `_lazy_re_compile` from `django.utils.regex_helper`.\n\nThe set of escaped characters includes: `\"`, `:`, `/`, `_`, `#`, `?`, `;`, `@`, `&`, `=`, `+`, `$`, `,`, `[`, `]`, `<`, `>`, `%`, `\\n` (newline), and `\\` (backslash).\n\nThe function is used in the admin \u2014 for example in `admin_urls.py` as a template filter `admin_urlquote`, and in `options.py` where `quote` is imported and used when building URLs with `quote(obj.pk)`. The corresponding `unquote()` is called in view methods like `_changeform_view` to decode the object_id back from the URL.",
    "rubric": [
      "Explains that QUOTE_MAP is a dictionary mapping byte ordinals from a specific set of special characters to underscore-prefixed two-digit hex strings (e.g., '/' becomes '_2F')",
      "Mentions the specific set of escaped characters includes at least: / _ : \" # ? ; @ & = + $ , [ ] < > % newline and backslash",
      "Explains that quote() uses str.translate(QUOTE_MAP) to perform the substitution if the input is a string, and returns non-string inputs unchanged",
      "Notes that the escape format uses _XX (underscore + hex) rather than %XX as in urllib.parse.quote()",
      "Explains that UNQUOTE_MAP is the inverse mapping (e.g., '_2F' \u2192 '/') and UNQUOTE_RE is a compiled regex matching the _XX patterns",
      "Explains that unquote() uses re.sub with a lambda that looks up matched _XX patterns in UNQUOTE_MAP to restore original characters"
    ],
    "key_files": [
      "django/contrib/admin/utils.py",
      "django/contrib/admin/templatetags/admin_urls.py",
      "django/contrib/admin/options.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/utils.py] django.contrib.admin.utils.quote\nEnsure that primary key values do not confuse the admin URLs by escaping\n    any '/', '_' and ':' and similarly problematic characters.\n    Similar to urllib.parse.quote(), except that the quoting is slightly\n    different so that it doesn't get automatically unquoted by the web browser.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: the rationale about browsers unquoting percent-encoded characters is not in stripped code; most claims are verifiable from source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_19",
    "repo": "django",
    "question": "Where is the AdminSite class defined and how do its key features connect to the broader Django admin system?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `AdminSite` class is located in `django/contrib/admin/sites.py`. Here is where its key features are located and how they connect to the broader system:\n\n**Registration and Storage:**\n- `AdminSite.__init__` initializes `self._registry` as an empty dict (used to map model classes to admin class instances as seen in `register()`). It also initializes `self._actions` with a default `\"delete_selected\"` action from `django.contrib.admin.actions.delete_selected`, and `self._global_actions` as a copy of `_actions`. Every `AdminSite` instance is added to the module-level `all_sites` WeakSet.\n- The `register()` method checks for abstract models, composite primary keys (`model._meta.is_composite_pk`), and already-registered models. When `**options` are passed, it dynamically creates a subclass of the admin class using `type()` with the name `\"{ModelName}Admin\"` and sets `__module__` to `__name__` (the sites module). The admin class is instantiated with `admin_class(model, self)` and stored in `_registry`.\n- `get_model_admin(model)` retrieves a registered admin from `_registry`, raising `NotRegistered` if not found.\n\n**URL Configuration:**\n- `get_urls()` defines a `wrap()` inner function that sets `wrapper.admin_site = self` and `wrapper.login_url = reverse_lazy(\"admin:login\", current_app=self.name)` on every wrapped view.\n- The URL patterns include: index, login, logout, password_change, password_change_done, autocomplete, jsi18n, and a content-type shortcut at `r/<path:content_type_id>/<path:object_id>/`.\n- Per-model URLs are generated from `model._meta.app_label` and `model._meta.model_name`, including the model admin's own `urls`.\n- A dynamic regex for `app_index` is built from valid app labels: `r\"^(?P<app_label>\" + \"|\".join(valid_app_labels) + \")/$\"`.\n- The `final_catch_all_view` attribute (default `True`) controls whether a catch-all URL pattern is appended; the `catch_all_view` method is decorated with `@no_append_slash` and handles `APPEND_SLASH` redirects.\n- The `urls` property returns a 3-tuple: `(self.get_urls(), \"admin\", self.name)`.\n\n**View Methods:**\n- `login()` is decorated with both `@method_decorator(never_cache)` and `@login_not_required` (from `django.contrib.auth.decorators`).\n- `admin_view()` applies `never_cache` unless `cacheable=True`, and applies `csrf_protect` unless the view has `csrf_exempt = True`.\n- `password_change()` uses `AdminPasswordChangeForm` (from `django.contrib.admin.forms`), which is a subclass of `PasswordChangeForm` with only `required_css_class = \"required\"`.\n- `autocomplete_view()` delegates to `AutocompleteJsonView.as_view(admin_site=self)`, which extends `BaseListView` with `paginate_by = 20`.\n- `logout()` explicitly sets `has_permission` to `False` in the context.\n\n**Context and App Building:**\n- `each_context()` includes `log_entries` from `get_log_entries(request)`.\n- `_build_app_dict()` creates model entries with keys: `model`, `name` (from `capfirst(model._meta.verbose_name_plural)`), `object_name`, `perms`, `admin_url`, `add_url`, and optionally `view_only`.\n\n**System Checks Integration:**\n- The module-level `all_sites` WeakSet is iterated in `check_admin_app()` (in `django/contrib/admin/checks.py`) to call `site.check(app_configs)` on every site. It's also used in `check_dependencies()` to determine if any site has `enable_nav_sidebar` enabled.\n- The `check()` method only runs checks on ModelAdmin instances where `o.__class__ is not ModelAdmin`, filtering out base ModelAdmin instances.\n\n**Default Site:**\n- `LazyObject` is imported and used in the module. `SimpleAdminConfig` defines `default_site = \"django.contrib.admin.sites.AdminSite\"`.\n- `AdminConfig` (the default app config with `default = True`) calls `self.module.autodiscover()` in its `ready()` method, which triggers `autodiscover_modules(\"admin\", register_to=site)`.\n- The `EMPTY_VALUE_STRING` constant is imported from `options.py` and is used as the `empty_value_display` class attribute default.",
    "rubric": [
      "Identifies that AdminSite is in django/contrib/admin/sites.py",
      "Explains that __init__ creates _registry dict, _actions with delete_selected, _global_actions, and adds the instance to all_sites WeakSet",
      "Describes register() checking for abstract models and composite PKs, dynamically creating admin subclass via type() when options are passed, and storing instances in _registry",
      "Explains URL configuration including the wrap() inner function, per-model URL generation from app_label/model_name, dynamic app_index regex, and final_catch_all_view",
      "Mentions the urls property returns a 3-tuple (get_urls(), 'admin', self.name)",
      "Notes that login() is decorated with never_cache and login_not_required",
      "Describes admin_view() applying never_cache (unless cacheable=True) and csrf_protect (unless csrf_exempt=True)",
      "Explains that all_sites WeakSet is used in checks.py (check_admin_app and check_dependencies)",
      "Mentions _build_app_dict() creating model entries with keys like model, name, object_name, perms, admin_url, add_url, view_only",
      "Notes that AdminConfig's ready() calls autodiscover() which triggers autodiscover_modules('admin', register_to=site)"
    ],
    "key_files": [
      "django/contrib/admin/sites.py",
      "django/contrib/admin/actions.py",
      "django/contrib/admin/checks.py",
      "django/contrib/admin/apps.py",
      "django/contrib/admin/forms.py",
      "django/contrib/admin/views/autocomplete.py",
      "django/contrib/admin/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/sites.py] django.contrib.admin.sites.AdminSite\ndjango.contrib.admin.sites.AdminSite:\n  An AdminSite object encapsulates an instance of the Django admin\n    application, ready to be hooked in to your URLconf. Models are registered\n    with the AdminSite using the register() method, and the get_urls() method\n    can then be used to access Django view functions that present a full admin\n    interface for the collection of registered models.\n\ndjango.contrib.admin.sites.AdminSite.check:\n  Run the system checks on all ModelAdmins, except if they aren't\n        customized at all.\n\ndjango.contrib.admin.sites.AdminSite.register:\n  Register the given model(s) with the given admin class.\n\n        The model(s) should be Model classes, not instances.\n\n        If an admin class isn't given, use ModelAdmin (the default admin\n        options). If keyword arguments are given -- e.g., list_display --\n        apply them as options to the admin class.\n\n        If a model is already registered, raise AlreadyRegistered.\n\n        If a model is abstract, raise ImproperlyConfigured.\n\ndjango.contrib.admin.sites.AdminSite.unregister:\n  Unregister the given model(s).\n\n        If a model isn't already registered, raise NotRegistered.\n\ndjango.contrib.admin.sites.AdminSite.is_registered:\n  Check if a model class is registered with this `AdminSite`.\n\ndjango.contrib.admin.sites.AdminSite.add_action:\n  Register an action to be available globally.\n\ndjango.contrib.admin.sites.AdminSite.disable_action:\n  Disable a globally-registered action. Raise KeyError for invalid names.\n\ndjango.contrib.admin.sites.AdminSite.get_action:\n  Explicitly get a registered global action whether it's enabled or\n        not. Raise KeyError for invalid names.\n\ndjango.contrib.admin.sites.AdminSite.actions:\n  Get all the enabled actions as an iterable of (name, func).\n\ndjango.contrib.admin.sites.AdminSite.has_permission:\n  Return True if the given HttpRequest has permission to view\n        *at least one* page in the admin site.\n\ndjango.contrib.admin.sites.AdminSite.admin_view:\n  Decorator to create an admin view attached to this ``AdminSite``. This\n        wraps the view and provides permission checking by calling\n        ``self.has_permission``.\n\n        You'll want to use this from within ``AdminSite.get_urls()``:\n\n            class MyAdminSite(AdminSite):\n\n                def get_urls(self):\n                    from django.urls import path\n\n                    urls = super().get_urls()\n                    urls += [\n                        path('my_view/', self.admin_view(some_view))\n                    ]\n                    return urls\n\n        By default, admin_views are marked non-cacheable using the\n        ``never_cache`` decorator. If the view can be safely cached, set\n        cacheable=True.\n\ndjango.contrib.admin.sites.AdminSite.each_context:\n  Return a dictionary of variables to put in the template context for\n        *every* page in the admin site.\n\n        For sites running on a subpath, use the SCRIPT_NAME value if site_url\n        hasn't been customized.\n\ndjango.contrib.admin.sites.AdminSite.password_change:\n  Handle the \"change password\" task -- both form display and validation.\n\ndjango.contrib.admin.sites.AdminSite.password_change_done:\n  Display the \"success\" page after a password change.\n\ndjango.contrib.admin.sites.AdminSite.i18n_javascript:\n  Display the i18n JavaScript that the Django admin requires.\n\n        `extra_context` is unused but present for consistency with the other\n        admin views.\n\ndjango.contrib.admin.sites.AdminSite.logout:\n  Log out the user for the given HttpRequest.\n\n        This should *not* assume the user is already logged in.\n\ndjango.contrib.admin.sites.AdminSite.login:\n  Display the login form for the given HttpRequest.\n\ndjango.contrib.admin.sites.AdminSite._build_app_dict:\n  Build the app dictionary. The optional `label` parameter filters models\n        of a specific app.\n\ndjango.contrib.admin.sites.AdminSite.get_app_list:\n  Return a sorted list of all the installed apps that have been\n        registered in this site.\n\ndjango.contrib.admin.sites.AdminSite.index:\n  Display the main admin index page, which lists all of the installed\n        apps that have been registered in this site.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: a few claims about rationale (LoginRequiredMiddleware usage, logout context reason), one about login_not_required internals, and unverifiable implementation details from truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_17",
    "repo": "django",
    "question": "What is the structure and behavior of the LogEntry model and its manager in Django's admin system?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `LogEntry` model in `django/contrib/admin/models.py` stores records of additions, changes, and deletions performed through the admin interface.\n\n**Model Structure:**\n- It uses the database table `django_admin_log` (defined in `Meta.db_table`)\n- It is ordered by `-action_time` by default\n- The `action_flag` field is a `PositiveSmallIntegerField` with three defined constants: `ADDITION = 1`, `CHANGE = 2`, `DELETION = 3`\n- The `object_repr` field has a `max_length=200`, and the `log_actions` manager method truncates the string representation to 200 characters via `str(obj)[:200]`\n- The `object_id` field is a `TextField` (not an IntegerField)\n- The `user` ForeignKey uses `models.CASCADE` on delete, while the `content_type` ForeignKey uses `models.SET_NULL` with `null=True`\n- The `action_time` field defaults to `timezone.now` (a callable, not evaluated at definition time)\n\n**Manager (`LogEntryManager`):**\n- Has `use_in_migrations = True`\n- The `log_actions` method accepts a `single_object` keyword-only parameter; when `True` and there's exactly one object, it returns the single `LogEntry` instance rather than a list\n- When there's only one item in the queryset, it calls `instance.save()`; when there are multiple items, it uses `bulk_create`\n- When `change_message` is a list, it's serialized to JSON via `json.dumps(change_message)` before storage\n- It calls `ContentType.objects.get_for_model(obj, for_concrete_model=False)`, passing `for_concrete_model=False` which means the model's own opts are used rather than its concrete model's opts\n\n**`get_change_message()` method:**\n- Detects JSON by checking if the first character is `[`\n- Falls back to returning the raw string if `json.JSONDecodeError` is raised\n- Handles three sub-message types: `added`, `changed`, and `deleted`\n- For `changed` messages, it uses `get_text_list()` from `django.utils.text` to join field names with a translated \"and\" connector\n- Each message's first character is uppercased via `msg[0].upper() + msg[1:]`, and messages are joined with spaces\n- Returns `gettext(\"No fields changed.\")` if JSON is valid but produces no messages\n\n**`get_edited_object()` method:**\n- Delegates to `self.content_type.get_object_for_this_type(pk=self.object_id)`, which in turn calls `self.model_class()._base_manager.using(using).get(**kwargs)` on the ContentType model\n\n**`get_admin_url()` method:**\n- Constructs a URL pattern name as `admin:{app_label}_{model}_change`\n- Uses the `quote()` utility function from `django/contrib/admin/utils.py` to escape the object_id for safe URL inclusion, using a custom `QUOTE_MAP` that escapes characters including `/`, `_`, `:`, `#`, `?`, `@`, `&`, `=`, `+`, `$`, `\"`, `[`, `]`, `<`, `>`, `%`, `\\n`, and `\\`\n- Returns `None` if `content_type` or `object_id` is falsy, or if `NoReverseMatch` is raised\n\n**Integration with ModelAdmin:**\n- `ModelAdmin.log_addition()`, `log_change()`, and `log_deletions()` in `django/contrib/admin/options.py` all call `LogEntry.objects.log_actions()` with `single_object=True` for addition/change and without it for deletions (which may affect multiple objects)",
    "rubric": [
      "Mentions the database table name is 'django_admin_log' and default ordering is '-action_time'",
      "Identifies the three action_flag constants: ADDITION=1, CHANGE=2, DELETION=3 stored in a PositiveSmallIntegerField",
      "Notes that object_id is a TextField (not IntegerField) and object_repr has max_length=200 with truncation via str(obj)[:200]",
      "Explains the user ForeignKey uses CASCADE while content_type uses SET_NULL with null=True",
      "Describes the LogEntryManager's log_actions method: uses save() for single items and bulk_create for multiple, with the single_object keyword-only parameter returning the instance directly",
      "Explains that get_change_message() detects JSON by checking if first character is '[', falls back to raw string on JSONDecodeError, and handles added/changed/deleted sub-messages",
      "Notes that get_change_message uses get_text_list for joining field names, uppercases first character of each message, and returns 'No fields changed.' for empty valid JSON",
      "Describes get_admin_url() constructing 'admin:{app_label}_{model}_change' pattern, using quote() on object_id, and returning None on NoReverseMatch or falsy content_type/object_id",
      "Explains how ModelAdmin's log_addition/log_change pass single_object=True while log_deletions does not",
      "Notes that ContentType.objects.get_for_model is called with for_concrete_model=False"
    ],
    "key_files": [
      "django/contrib/admin/models.py",
      "django/contrib/contenttypes/models.py",
      "django/contrib/admin/utils.py",
      "django/contrib/admin/options.py",
      "django/utils/text.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/models.py] django.contrib.admin.models.LogEntry\ndjango.contrib.admin.models.LogEntry.get_change_message:\n  If self.change_message is a JSON structure, interpret it as a change\n        string, properly translated.\n\ndjango.contrib.admin.models.LogEntry.get_edited_object:\n  Return the edited object represented by this log entry.\n\ndjango.contrib.admin.models.LogEntry.get_admin_url:\n  Return the admin URL to edit the object represented by this log entry.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides docstrings for three methods (get_change_message, get_edited_object, get_admin_url), but the answer makes extensive claims about internal implementation details (e.g., LogEntryManager having 'use_in_migrations = True', 'single_object' keyword-only parameter, JSON detection by checking first character '[', 'quote()' utility function details, ModelAdmin integration details) that cannot be verified from the provided documentation alone.",
      "The claim about 'log_actions' manager method with 'single_object' parameter is not verifiable from the documentation and may be a hallucination - Django's actual LogEntryManager uses 'log_action' (singular), not 'log_actions'.",
      "The claim about ModelAdmin methods calling 'LogEntry.objects.log_actions()' appears to be fabricated - in standard Django, ModelAdmin calls 'LogEntry.objects.log_action()' (singular).",
      "The get_edited_object documentation says 'Return the edited object represented by this log entry' but the answer's claim about delegating to 'self.content_type.get_object_for_this_type(pk=self.object_id)' is plausible but unverifiable from the doc.",
      "The get_admin_url documentation says 'Return the admin URL to edit the object represented by this log entry' but the answer's detailed claims about quote() utility, QUOTE_MAP, and None return conditions are unverifiable from the doc."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; only a few design rationale statements needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_22",
    "repo": "django",
    "question": "Why does the AutocompleteJsonView only conditionally call distinct() and what other performance strategies does it use to keep autocomplete queries fast?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `AutocompleteJsonView` makes several performance-oriented design decisions:\n\n1. **Conditional `distinct()` to avoid unnecessary query cost**: In `get_queryset()`, the view calls `self.model_admin.get_search_results()` which returns a tuple of `(queryset, search_use_distinct)`. The `distinct()` call is only applied when `search_use_distinct` is True \u2014 this avoids the cost of DISTINCT on every autocomplete query when it's not needed. The `lookup_spawns_duplicates()` function (in `django/contrib/admin/utils.py`) specifically checks if any field in the lookup path has `path_infos` with `m2m=True`.\n\n2. **Pagination limited to 20 results**: The class sets `paginate_by = 20` (hardcoded class attribute), which limits each AJAX response to 20 database results. The pagination is handled by `BaseListView`'s `get_context_data()` which calls `paginate_queryset()` when `page_size` is truthy.\n\n3. **Delegating to ModelAdmin's paginator**: The `get_paginator()` method delegates to `self.model_admin.get_paginator(self.request, *args, **kwargs)`, which allows customization of pagination behavior.\n\n4. **`complex_filter` for `limit_choices_to` before search**: The queryset first applies `source_field.get_limit_choices_to()` via `complex_filter()`, narrowing the queryset *before* the search operation runs. This means the search only operates on the already-filtered subset.\n\n5. **Early validation to avoid expensive operations**: The `process_request()` method validates all parameters (app_label, model_name, field_name) and checks `search_fields` configuration *before* any database query is executed. Invalid requests are rejected with `PermissionDenied` or `Http404` early, preventing wasted database work.",
    "rubric": [
      "Explains that distinct() is only called when search_use_distinct is True (returned by get_search_results), avoiding unnecessary DISTINCT cost when searches don't span m2m relations",
      "Mentions that paginate_by is hardcoded to 20, limiting each response to a small result set",
      "Notes that limit_choices_to is applied via complex_filter() before search, narrowing the queryset early",
      "Explains that process_request() validates parameters and raises PermissionDenied/Http404 before any database queries execute",
      "Mentions that get_paginator delegates to model_admin.get_paginator for customizable pagination behavior"
    ],
    "key_files": [
      "django/contrib/admin/views/autocomplete.py",
      "django/contrib/admin/options.py",
      "django/contrib/admin/utils.py",
      "django/views/generic/list.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/views/autocomplete.py] django.contrib.admin.views.autocomplete.AutocompleteJsonView\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView:\n  Handle AutocompleteWidget's AJAX requests for data.\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.get:\n  Return a JsonResponse with search results as defined in\n        serialize_result(), by default:\n        {\n            results: [{id: \"123\" text: \"foo\"}],\n            pagination: {more: true}\n        }\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.serialize_result:\n  Convert the provided model object to a dictionary that is added to the\n        results list.\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.get_paginator:\n  Use the ModelAdmin's paginator.\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.get_queryset:\n  Return queryset based on ModelAdmin.get_search_results().\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.process_request:\n  Validate request integrity, extract and return request parameters.\n\n        Since the subsequent view permission check requires the target model\n        admin, which is determined here, raise PermissionDenied if the\n        requested app, model or field are malformed.\n\n        Raise Http404 if the target model admin is not configured properly with\n        search_fields.\n\ndjango.contrib.admin.views.autocomplete.AutocompleteJsonView.has_perm:\n  Check if user has permission to access the related model.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim about `paginate_by = 20` is a specific implementation detail not mentioned in the documentation. While plausible based on Django source code knowledge, it is not supported by the provided documentation.",
      "Claim about `distinct()` being conditionally applied based on `lookup_spawns_duplicates()` and `search_use_distinct` is a specific implementation detail not in the documentation. The doc only says `get_queryset()` returns a queryset based on `ModelAdmin.get_search_results()`.",
      "Claim about `complex_filter` for `limit_choices_to` is a specific implementation detail not mentioned in the documentation.",
      "Claim about `BaseListView`'s `get_context_data()` calling `paginate_queryset()` is an implementation detail not in the documentation.",
      "The detailed description of `lookup_spawns_duplicates()` checking `path_infos` with `m2m=True` is not supported by the documentation.",
      "Most claims are plausible extrapolations from Django source code but go well beyond what the provided documentation states."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in design rationale statements and a parenthetical about complex_filter's accepted types; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_21",
    "repo": "django",
    "question": "What dependencies and internal behaviors does `model_ngettext` in the admin utilities rely on to resolve singular vs. plural verbose names?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `model_ngettext` function in `django/contrib/admin/utils.py` depends on several key components:\n\n1. **`model_format_dict`** (same file): Called internally by `model_ngettext` to extract `verbose_name` and `verbose_name_plural` from the object. It handles three cases:\n   - If `obj` is a `models.Model` instance or `models.base.ModelBase` (the metaclass for all models), it accesses `obj._meta`\n   - If `obj` is a `models.query.QuerySet`, it accesses `obj.model._meta`\n   - Otherwise (e.g., an `Options` instance passed directly), it treats `obj` itself as the opts object\n\n2. **`ngettext`** from `django.utils.translation` (imported at the top of the file): Used to select between the singular and plural form based on count. The actual implementation delegates through `_trans` module \u2014 either `trans_real.py` (uses `do_ntranslate` which calls `DjangoTranslation.ngettext` using the catalog's `plural()` method) or `trans_null.py` (when `USE_I18N = False`).\n\n3. **Fallback to 0**: When `n` is `None` and `obj` is not a QuerySet, or when `n` is falsy, the expression `n or 0` passes 0 to `ngettext`, resulting in the plural form being returned.\n\n4. **Undocumented usage pattern**: In practice, `model_ngettext` is also called with `modeladmin.opts` (which is `model._meta`, an `Options` instance) \u2014 not just Model instances, subclasses, or QuerySets. This works because `model_format_dict` has an `else` branch that treats any unrecognized object as having `verbose_name` and `verbose_name_plural` attributes directly. This usage is seen in `django/contrib/admin/actions.py` (e.g., `model_ngettext(modeladmin.opts, n)`).\n\n5. **`verbose_name_plural` default**: In `django/db/models/options.py`, if no explicit `verbose_name_plural` is defined in a model's Meta class, it defaults to `format_lazy('{}s', self.verbose_name)` \u2014 simply appending 's' to the verbose_name.",
    "rubric": [
      "Mentions that model_ngettext calls model_format_dict to extract verbose_name and verbose_name_plural from the object",
      "Explains that model_format_dict handles Model instances/subclasses (via obj._meta), QuerySets (via obj.model._meta), and falls back to treating obj itself as the opts object",
      "Identifies that ngettext from django.utils.translation is used to select between singular and plural forms based on count",
      "Explains the n or 0 fallback: when n is None (and obj is not a QuerySet) or falsy, 0 is passed to ngettext, resulting in the plural form",
      "Notes the undocumented usage pattern where model_ngettext is called with modeladmin.opts (an Options instance), which works because of model_format_dict's else branch",
      "Mentions that verbose_name_plural defaults to appending 's' to verbose_name via format_lazy if not explicitly set in a model's Meta class"
    ],
    "key_files": [
      "django/contrib/admin/utils.py",
      "django/utils/translation/__init__.py",
      "django/utils/translation/trans_real.py",
      "django/contrib/admin/actions.py",
      "django/db/models/options.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/utils.py] django.contrib.admin.utils.model_ngettext\nReturn the appropriate `verbose_name` or `verbose_name_plural` value for\n    `obj` depending on the count `n`.\n\n    `obj` may be a `Model` instance, `Model` subclass, or `QuerySet` instance.\n    If `obj` is a `QuerySet` instance, `n` is optional and the length of the\n    `QuerySet` is used.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments for unverifiable line numbers and trans_null.py details.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_23",
    "repo": "django",
    "question": "Where is the ChangeList's ordering logic defined, and how does it flow into the template layer for rendering column headers?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `ChangeList` class is defined in `django/contrib/admin/views/main.py` and its ordering-related methods are located there. The `get_ordering_field_columns` method is consumed by the `result_headers` function in `django/contrib/admin/templatetags/admin_list.py`. The `get_ordering` method is called from `ChangeList.get_queryset` which applies ordering to the filtered queryset.\n\nKey implementation details:\n\n1. **`get_ordering_field`**: When a `FieldDoesNotExist` is raised, it checks callables, model admin attributes, and model attributes (in that order). If the attribute is a `property`, it accesses `.fget` to find `admin_order_field`. If the field_name contains `LOOKUP_SEP` (\"__\") and causes an `AttributeError` on `getattr(self.model, field_name)`, it returns the `field_name` directly.\n\n2. **`get_ordering`**: Deterministic ordering is ensured by checking `queryset.order_by(*ordering).totally_ordered`. If not totally ordered, it appends `['-pk']` to the ordering list.\n\n3. **`IGNORED_PARAMS`** is a tuple of 7 items: `ALL_VAR` (\"all\"), `ORDER_VAR` (\"o\"), `SEARCH_VAR` (\"q\"), `IS_FACETS_VAR` (\"_facets\"), `IS_POPUP_VAR` (\"_popup\"), `SOURCE_MODEL_VAR` (\"_source_model\"), `TO_FIELD_VAR` (\"_to_field\").\n\n4. **`get_ordering_field_columns`**: When `ORDER_VAR` is not in params, it iterates default ordering fields. For `Combinable` or `OrderBy` expressions, if the expression is not an `F` expression, it skips it (`continue`). It only processes `F` expressions by extracting `field.expression.name`.\n\n5. **`ChangeList`** is instantiated via `ModelAdmin.get_changelist_instance`, which calls `ModelAdmin.get_changelist(request)` \u2014 this returns the `ChangeList` class by default.",
    "rubric": [
      "Identifies ChangeList class and its ordering methods (get_ordering, get_ordering_field, get_ordering_field_columns) in django/contrib/admin/views/main.py",
      "Explains that get_ordering_field_columns is consumed by the result_headers function in django/contrib/admin/templatetags/admin_list.py",
      "Explains that get_ordering is called from get_queryset to apply ordering to the filtered queryset",
      "Describes get_ordering_field's fallback logic: on FieldDoesNotExist, checks callables, model admin attributes, then model attributes; handles properties via .fget for admin_order_field; returns field_name directly if it contains LOOKUP_SEP and raises AttributeError",
      "Describes how get_ordering ensures deterministic ordering by checking totally_ordered and appending ['-pk'] if not totally ordered",
      "Mentions IGNORED_PARAMS is a tuple of 7 items including ALL_VAR, ORDER_VAR, SEARCH_VAR, IS_FACETS_VAR, IS_POPUP_VAR, SOURCE_MODEL_VAR, TO_FIELD_VAR",
      "Explains that get_ordering_field_columns handles Combinable/OrderBy expressions by only processing F expressions (extracting field.expression.name) and skipping non-F expressions"
    ],
    "key_files": [
      "django/contrib/admin/views/main.py",
      "django/db/models/query.py",
      "django/contrib/admin/templatetags/admin_list.py",
      "django/contrib/admin/options.py"
    ],
    "source_doc": "[docstring: django/contrib/admin/views/main.py] django.contrib.admin.views.main.ChangeList\ndjango.contrib.admin.views.main.ChangeList.get_filters_params:\n  Return all params except IGNORED_PARAMS.\n\ndjango.contrib.admin.views.main.ChangeList.get_ordering_field:\n  Return the proper model field name corresponding to the given\n        field_name to use for ordering. field_name may either be the name of a\n        proper model field, possibly across relations, or the name of a method\n        (on the admin or model) or a callable with the 'admin_order_field'\n        attribute. Return None if no proper model field name can be matched.\n\ndjango.contrib.admin.views.main.ChangeList.get_ordering:\n  Return the list of ordering fields for the change list.\n        First check the get_ordering() method in model admin, then check\n        the object's default ordering. Then, any manually-specified ordering\n        from the query string overrides anything. Finally, a deterministic\n        order is guaranteed by calling _get_deterministic_ordering() with the\n        constructed ordering.\n\ndjango.contrib.admin.views.main.ChangeList.get_ordering_field_columns:\n  Return a dictionary of ordering field column numbers and asc/desc.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims `_get_deterministic_ordering()` no longer exists and that deterministic ordering uses `totally_ordered` property instead, but the documentation explicitly states that `_get_deterministic_ordering()` is called. This could be a contradiction if the documentation is current, or could be accurate if the answer reflects a newer codebase version than the docstring.",
      "The answer states IGNORED_PARAMS contains 7 items including SOURCE_MODEL_VAR \u2014 this is not mentioned in the documentation and cannot be verified from it.",
      "Many specific line numbers and implementation details (e.g., line 346, line 375, line 424, line 506, line 848, line 840, line 2001) are not verifiable from the provided documentation.",
      "The claim about `totally_ordered` property on QuerySet is not mentioned in the documentation and cannot be verified.",
      "Details about `get_ordering_field_columns` handling of `Combinable`, `OrderBy`, and `F` expressions are not in the documentation."
    ],
    "strip_verify_leakage": 0.21,
    "strip_verify_summary": "A few claims relied on docstrings, line numbers, or documentation-level explanations not recoverable from stripped code; most logic claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_26",
    "repo": "django",
    "question": "Why does Django's ModelBackend make the specific design choices it does around timing, caching, permission resolution, and extensibility?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `ModelBackend` class in `django/contrib/auth/backends.py` embodies several deliberate design decisions:\n\n**1. Timing Attack Mitigation:** In `authenticate()`, when a user doesn't exist, the code still runs `UserModel().set_password(password)` in the `except UserModel.DoesNotExist` block. This reduces the timing difference between lookups for existing vs. nonexistent users, as the password hashing operation runs regardless of whether the user was found.\n\n**2. Permission Caching on User Object:** `_get_permissions()` caches permissions directly on the user object instance using dynamically-named attributes (`_user_perm_cache`, `_group_perm_cache`, `_perm_cache`). This means repeated permission checks within the same request don't hit the database again, but reloading the user object from the database purges the cache since these are in-memory instance attributes.\n\n**3. `order_by()` with no arguments to clear default ordering:** `_get_permissions()` calls `.order_by()` (empty) on the permission queryset. This is a performance optimization \u2014 the `Permission` model has `Meta.ordering = [\"content_type__app_label\", \"content_type__model\", \"codename\"]`, and since the results are going into a Python set, ordering is unnecessary. The empty `order_by()` removes the default ordering to avoid an unnecessary ORDER BY clause (and potential extra JOINs) in SQL.\n\n**4. Superuser gets ALL Permission objects:** When `user_obj.is_superuser` is True, `_get_permissions()` returns `Permission.objects.all()` rather than checking assigned permissions. This means superusers implicitly have every permission that exists in the database, even ones not explicitly assigned.\n\n**5. `user_can_authenticate()` uses `getattr(user, 'is_active', True)`:** This defaults to `True` if the attribute doesn't exist, meaning custom user models without an `is_active` field will still pass this check. The `AbstractBaseUser` class in `base_user.py` sets `is_active = True` as a class attribute (not a database field), so even base custom user models pass this check by default.\n\n**6. `AllowAllUsersModelBackend` as a minimal override:** This subclass only overrides `user_can_authenticate()` to return `True` unconditionally. This single-method override pattern demonstrates that `user_can_authenticate()` serves as a customization hook, allowing modification of active-user filtering without reimplementing the entire authentication flow.\n\n**7. Object-level permissions return empty sets:** Both `_get_permissions()` and `with_perm()` return empty results when `obj is not None`. `ModelBackend` does not implement object-level permission checking \u2014 the `obj` parameter is present in the method signatures but results in empty returns when non-None.\n\n**8. Three-layer permission architecture:** The `PermissionsMixin` on the User model delegates to helper functions (`_user_get_permissions`, `_user_has_perm`, `_user_has_module_perms`) which iterate through ALL configured backends. Active superusers short-circuit at the model layer (`if self.is_active and self.is_superuser: return True`), but `ModelBackend.has_perm()` also requires `user_obj.is_active` before delegating to `BaseBackend.has_perm()`. This dual-check design means both the model and backend layers enforce the active-user requirement.\n\n**9. `with_perm()` uses `Exists` subquery with `OuterRef`:** The method constructs `Q(group__user=OuterRef('pk')) | Q(user=OuterRef('pk'))` to find users who have a permission either directly or through groups, using a single efficient subquery rather than two separate queries.",
    "rubric": [
      "Explains the timing attack mitigation: when a user doesn't exist, set_password is still called to reduce timing differences between existing and nonexistent user lookups",
      "Explains permission caching on the user object instance using dynamic attribute names (_user_perm_cache, _group_perm_cache, _perm_cache) so repeated checks don't hit the database again within the same request",
      "Explains why .order_by() with no arguments is called: to strip the Permission model's default Meta ordering since results go into a set, avoiding unnecessary ORDER BY and potential JOINs in SQL",
      "Explains that superusers get Permission.objects.all() rather than checking assigned permissions, implicitly granting every permission in the database",
      "Explains that user_can_authenticate uses getattr(user, 'is_active', True) defaulting to True so custom user models without is_active still pass authentication",
      "Explains that AllowAllUsersModelBackend only overrides user_can_authenticate to return True unconditionally, showing it serves as a customization hook",
      "Explains that object-level permissions (when obj is not None) return empty sets because ModelBackend does not implement object-level permission checking",
      "Explains the multi-layer permission architecture where both PermissionsMixin and ModelBackend check is_active, and helper functions iterate through all configured backends",
      "Explains that with_perm uses Exists subquery with OuterRef to find users with a permission either directly or through groups in a single efficient query"
    ],
    "key_files": [
      "django/contrib/auth/backends.py",
      "django/contrib/auth/models.py",
      "django/contrib/auth/__init__.py",
      "django/contrib/auth/base_user.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/backends.py] django.contrib.auth.backends.ModelBackend\ndjango.contrib.auth.backends.ModelBackend:\n  Authenticates against settings.AUTH_USER_MODEL.\n\ndjango.contrib.auth.backends.ModelBackend.user_can_authenticate:\n  Reject users with is_active=False. Custom user models that don't have\n        that attribute are allowed.\n\ndjango.contrib.auth.backends.ModelBackend._get_permissions:\n  Return the permissions of `user_obj` from `from_name`. `from_name` can\n        be either \"group\" or \"user\" to return permissions from\n        `_get_group_permissions` or `_get_user_permissions` respectively.\n\ndjango.contrib.auth.backends.ModelBackend._aget_permissions:\n  See _get_permissions().\n\ndjango.contrib.auth.backends.ModelBackend.get_user_permissions:\n  Return a set of permission strings the user `user_obj` has from their\n        `user_permissions`.\n\ndjango.contrib.auth.backends.ModelBackend.aget_user_permissions:\n  See get_user_permissions().\n\ndjango.contrib.auth.backends.ModelBackend.get_group_permissions:\n  Return a set of permission strings the user `user_obj` has from the\n        groups they belong.\n\ndjango.contrib.auth.backends.ModelBackend.aget_group_permissions:\n  See get_group_permissions().\n\ndjango.contrib.auth.backends.ModelBackend.has_module_perms:\n  Return True if user_obj has any permissions in the given app_label.\n\ndjango.contrib.auth.backends.ModelBackend.ahas_module_perms:\n  See has_module_perms()\n\ndjango.contrib.auth.backends.ModelBackend.with_perm:\n  Return users that have permission \"perm\". By default, filter out\n        inactive users and include superusers.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims in the answer go well beyond what the documentation chunk provides, referencing specific line numbers, implementation details, and code patterns that are not present in the docstrings. However, these appear to be plausible code-level details derived from reading the actual source code.",
      "The answer's claim about `user_can_authenticate()` rejecting users with `is_active=False` and allowing custom models without that attribute is directly supported by the documentation.",
      "The answer's claim about `with_perm()` filtering out inactive users and including superusers by default is supported by the documentation.",
      "The answer's description of `_get_permissions()` accepting 'group' or 'user' as `from_name` is consistent with the documentation.",
      "The answer's claim about superuser behavior in `_get_permissions()` returning all permissions is not directly stated in the docs but is a plausible implementation detail consistent with Django's known behavior.",
      "The answer's claim about `AllowAllUsersModelBackend` is not mentioned in the documentation chunk at all, but is a known Django class \u2014 this is extrapolation beyond the provided docs.",
      "The answer's specific line number references (line 69, 110, 119, 213-214, 393) cannot be verified from the documentation but are plausible code references."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in timing attack ticket reference, mod_wsgi cross-reference, and some design rationale statements.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_24",
    "repo": "django",
    "question": "How does XViewMiddleware decide when to add an X-View header, and how does its process_view get invoked by Django's handler?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `XViewMiddleware` in `django/contrib/admindocs/middleware.py` works as follows:\n\n**Class hierarchy and initialization:**\n- `XViewMiddleware` inherits from `MiddlewareMixin` (from `django.utils.deprecation`), which has `sync_capable = True` and `async_capable = True`.\n- `MiddlewareMixin.__init__` requires a non-None `get_response` argument (raises `ValueError` otherwise), stores it as `self.get_response`, and sets `self.async_mode` based on whether `get_response` is a coroutine function.\n\n**How `process_view` is invoked:**\n- `process_view` is NOT called by `MiddlewareMixin.__call__` (which only handles `process_request` and `process_response`). Instead, it is collected and invoked by Django's request handler in `BaseHandler.load_middleware()`, which inserts it into `self._view_middleware` list (inserted at position 0, i.e. reversed order).\n- In `_get_response()`, the handler iterates `self._view_middleware`, calling each with `(request, callback, callback_args, callback_kwargs)`. If any returns a truthy response, the loop breaks and the actual view is never called.\n\n**The `process_view` algorithm (line-by-line):**\n1. First checks `hasattr(request, 'user')` \u2014 if not present, raises `ImproperlyConfigured` with a message about needing `AuthenticationMiddleware`.\n2. Checks two conditions with AND: request method must be `\"HEAD\"` AND either:\n   - `request.META.get('REMOTE_ADDR')` is in `settings.INTERNAL_IPS`, OR\n   - `request.user.is_active` AND `request.user.is_staff` (both must be True).\n3. If conditions pass, creates a plain `HttpResponse()` (no content), sets `response.headers['X-View']` to the result of `get_view_name(view_func)`, and returns it (short-circuiting the actual view).\n\n**`get_view_name` logic (from `django/contrib/admindocs/utils.py`):**\n- If `view_func` has a `view_class` attribute, it returns `f\"{klass.__module__}.{klass.__qualname__}\"` using `view_class`.\n- Otherwise, it uses `view_func.__module__` + `getattr(view_func, '__qualname__', view_func.__class__.__name__)`.\n- For callable objects (instances without `view_class`), the fallback path uses `__qualname__` if available, otherwise `__class__.__name__`.\n\n**Per-view decorator usage:**\n- `XViewMiddleware` can also be used as a per-view decorator via `decorator_from_middleware(XViewMiddleware)`. In this mode, `make_middleware_decorator` passes `view_func` as the `get_response` parameter to `XViewMiddleware.__init__`, and explicitly calls `middleware.process_view(request, view_func, args, kwargs)` during pre-processing.\n\n**Edge case:** The middleware only responds to `HEAD` requests; for all other HTTP methods (including GET), `process_view` implicitly returns `None`, allowing normal view execution to proceed.",
    "rubric": [
      "Explains that process_view checks for hasattr(request, 'user') and raises ImproperlyConfigured if missing",
      "Explains the method must be HEAD AND (REMOTE_ADDR in INTERNAL_IPS OR user.is_active AND user.is_staff)",
      "Explains that when conditions pass, it returns an HttpResponse with the X-View header set to get_view_name(view_func)",
      "Explains that get_view_name checks for view_class attribute and uses module + qualname (or __class__.__name__ as fallback)",
      "Explains that process_view is NOT called by MiddlewareMixin.__call__ but collected by BaseHandler.load_middleware() into _view_middleware (inserted at position 0)",
      "Explains that _get_response iterates _view_middleware calling each with (request, callback, callback_args, callback_kwargs), breaking if a truthy response is returned",
      "Mentions that for non-HEAD requests, process_view returns None implicitly, allowing normal view execution"
    ],
    "key_files": [
      "django/contrib/admindocs/middleware.py",
      "django/contrib/admindocs/utils.py",
      "django/utils/deprecation.py",
      "django/core/handlers/base.py",
      "django/utils/decorators.py"
    ],
    "source_doc": "[docstring: django/contrib/admindocs/middleware.py] django.contrib.admindocs.middleware.XViewMiddleware\ndjango.contrib.admindocs.middleware.XViewMiddleware:\n  Add an X-View header to internal HEAD requests.\n\ndjango.contrib.admindocs.middleware.XViewMiddleware.process_view:\n  If the request method is HEAD and either the IP is internal or the\n        user is a logged-in staff member, return a response with an x-view\n        header indicating the view function. This is used to lookup the view\n        function for an arbitrary page.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are directly verifiable from stripped source code; minor adjustments needed for line number references and test file references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_25",
    "repo": "django",
    "question": "What is the architecture of Django's UserAdmin, including its form class hierarchy, user creation workflow, permission checks, and post-add redirect behavior?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `UserAdmin` class in `django/contrib/auth/admin.py` extends `django.contrib.admin.options.ModelAdmin` (which itself extends `BaseModelAdmin`) and is registered to the `User` model using the `@admin.register(User)` decorator.\n\n**Architecture of `UserAdmin.get_form`:**\nWhen `obj is None` (i.e., during user creation), it sets `defaults[\"form\"]` to `self.add_form` which is `AdminUserCreationForm`. It then passes these defaults to `super().get_form()` (the parent `ModelAdmin.get_form`). The parent's `get_form` method ultimately calls `modelform_factory(self.model, **defaults)` to produce the final form class, using fields derived from `get_fieldsets()`. Since `get_fieldsets` also checks if `obj` is None and returns `self.add_fieldsets` in that case, the creation form uses the fields `(\"username\", \"usable_password\", \"password1\", \"password2\")` with CSS class \"wide\".\n\n**Architecture of `UserAdmin.response_add`:**\nThis method modifies `request.POST` by copying it and injecting `\"_continue\": 1` \u2014 but only when neither `\"_addanother\"` nor `IS_POPUP_VAR` (which equals `\"_popup\"`) is present in the POST data. This effectively makes the default behavior redirect to the change form instead of the changelist. It then delegates to `super().response_add()` which in `ModelAdmin.response_add` checks for `\"_continue\"` in `request.POST` and redirects to the object's change URL with preserved filters.\n\n**Key form class hierarchy:**\n- `AdminUserCreationForm` inherits from `SetUnusablePasswordMixin` and `UserCreationForm`\n- `UserCreationForm` inherits from `BaseUserCreationForm`\n- `BaseUserCreationForm` inherits from `SetPasswordMixin` and `forms.ModelForm`\n- `AdminUserCreationForm.__init__` sets `password1.required = False` and `password2.required = False`\n- It adds a `usable_password` field via `SetUnusablePasswordMixin.create_usable_password_field()`, which is a `ChoiceField` with RadioSelect widget and choices `{\"true\": _(\"Enabled\"), \"false\": _(\"Disabled\")}`\n\n**The `_add_view` security check:**\nThe `add_view` method wraps `_add_view` in a `transaction.atomic` for non-safe HTTP methods. The `_add_view` method enforces that users must have both \"Add user\" AND \"Change user\" permissions to add users. In DEBUG mode, it raises `Http404` with an explanatory message; otherwise it raises `PermissionDenied`.\n\n**Password change workflow:**\nThe `user_change_password` view uses `AdminPasswordChangeForm`, which takes `user` as its first argument (not bound to instance). After saving, it calls `update_session_auth_hash(request, form.user)`.",
    "rubric": [
      "UserAdmin extends ModelAdmin and is registered to the User model via @admin.register(User)",
      "get_form sets defaults['form'] to self.add_form (AdminUserCreationForm) when obj is None, then delegates to super().get_form() which uses modelform_factory",
      "get_fieldsets returns self.add_fieldsets (with fields 'username', 'usable_password', 'password1', 'password2') when obj is None",
      "response_add injects '_continue': 1 into request.POST unless '_addanother' or IS_POPUP_VAR ('_popup') is present, causing redirect to the change form instead of changelist",
      "AdminUserCreationForm inherits from SetUnusablePasswordMixin and UserCreationForm; UserCreationForm inherits from BaseUserCreationForm; BaseUserCreationForm inherits from SetPasswordMixin and forms.ModelForm",
      "AdminUserCreationForm.__init__ sets password1.required = False and password2.required = False",
      "The usable_password field is created via SetUnusablePasswordMixin.create_usable_password_field() as a ChoiceField with RadioSelect widget and choices 'true': Enabled, 'false': Disabled",
      "_add_view requires both Add and Change user permissions; raises Http404 with explanatory message in DEBUG mode or PermissionDenied otherwise",
      "add_view wraps _add_view in transaction.atomic for non-safe HTTP methods",
      "user_change_password uses AdminPasswordChangeForm and calls update_session_auth_hash(request, form.user) after saving"
    ],
    "key_files": [
      "django/contrib/auth/admin.py",
      "django/contrib/admin/options.py",
      "django/contrib/auth/forms.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/admin.py] django.contrib.auth.admin.UserAdmin\ndjango.contrib.auth.admin.UserAdmin.get_form:\n  Use special form during user creation\n\ndjango.contrib.auth.admin.UserAdmin.response_add:\n  Determine the HttpResponse for the add_view stage. It mostly defers to\n        its superclass implementation but is customized because the User model\n        has a slightly different workflow.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale explanations and behavioral descriptions of external functions; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_27",
    "repo": "django",
    "question": "Where does the REMOTE_USER credential flow from the request through backend selection, user creation, and session persistence?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data/control flow for `RemoteUserBackend.authenticate` traces as follows:\n\n1. **Entry point**: `RemoteUserMiddleware.process_request()` (in `django/contrib/auth/middleware.py`) reads `request.META[self.header]` (where `header` defaults to `\"REMOTE_USER\"`), then calls `auth.authenticate(request, remote_user=username)`. Notably, the async variant `aprocess_request()` uses `request.META[\"HTTP_\" + self.header]` instead \u2014 a different key lookup pattern than the sync version.\n\n2. **Backend selection via signature binding**: `django.contrib.auth.authenticate()` (in `django/contrib/auth/__init__.py`) iterates through all backends using `_get_compatible_backends()`, which uses `signature(backend.authenticate).bind(request, **credentials)` (where `signature` is from `django.utils.inspect`) to filter backends. Since `RemoteUserBackend.authenticate` has the explicit parameter `remote_user`, it will only match when `remote_user` is in the credentials. `ModelBackend.authenticate` has `username` and `password` parameters, so it won't match the `remote_user=username` credential \u2014 this is how the two backends are distinguished.\n\n3. **Inside `RemoteUserBackend.authenticate()`**: If `remote_user` is falsy, returns `None` immediately. Otherwise calls `self.clean_username(remote_user)`, then uses `UserModel._default_manager.get_or_create(**{UserModel.USERNAME_FIELD: username})` if `create_unknown_user` is `True`, or `UserModel._default_manager.get_by_natural_key(username)` if `False`.\n\n4. **`configure_user` is always called**, even for existing users \u2014 it receives a `created` kwarg (defaults to `True` in the method signature) that is `True` only when `get_or_create` actually created the user. This allows subclasses to differentiate behavior for new vs. existing users.\n\n5. **After `configure_user`**, the result is filtered through `self.user_can_authenticate(user)`, inherited from `ModelBackend`, which checks `getattr(user, 'is_active', True)`. This means inactive users are rejected by default.\n\n6. **Back in `auth.authenticate()`**: if a non-None user is returned, the user object is annotated with `user.backend = backend_path` (the dotted import path string). If no backend returns a user, the `user_login_failed` signal is fired.\n\n7. **Back in `RemoteUserMiddleware.process_request()`**: if `user` is truthy, `auth.login(request, user)` is called, which stores `SESSION_KEY` (\"_auth_user_id\"), `BACKEND_SESSION_KEY` (\"_auth_user_backend\"), and `HASH_SESSION_KEY` (\"_auth_user_hash\") in the session.\n\n8. **Logout flow**: `RemoteUserMiddleware._remove_invalid_user()` loads the stored backend from the session, checks `isinstance(stored_backend, RemoteUserBackend)`, and only calls `auth.logout()` if the stored backend is a `RemoteUserBackend` instance. This prevents logging out users authenticated by other backends.\n\n9. **`RemoteUserMiddleware.clean_username()`** (different from `RemoteUserBackend.clean_username()`): The middleware's `clean_username` loads the backend from `request.session[auth.BACKEND_SESSION_KEY]` and delegates to the backend's `clean_username` method if it exists (catches `AttributeError` if not).\n\n10. **`PersistentRemoteUserMiddleware`** is a subclass that sets `force_logout_if_no_header = False`, keeping users logged in even when the REMOTE_USER header disappears.\n\n11. **`AllowAllUsersRemoteUserBackend`** overrides `user_can_authenticate` to always return `True`, allowing inactive users through.\n\n12. **`RemoteUserBackend` inherits from `ModelBackend`** (not just `BaseBackend`), so it inherits `get_user()`, all permission methods, `user_can_authenticate()`, and `has_perm()`.",
    "rubric": [
      "RemoteUserMiddleware.process_request reads request.META[self.header] (default 'REMOTE_USER') and calls auth.authenticate(request, remote_user=username); the async variant aprocess_request uses 'HTTP_' + self.header instead",
      "_get_compatible_backends uses signature(backend.authenticate).bind() to filter backends by parameter names, so RemoteUserBackend (which has remote_user param) matches while ModelBackend (which has username/password params) does not",
      "RemoteUserBackend.authenticate returns None if remote_user is falsy, calls self.clean_username, then uses get_or_create or get_by_natural_key depending on create_unknown_user",
      "configure_user is called for both new and existing users, receiving a created kwarg that indicates whether the user was just created",
      "user_can_authenticate (inherited from ModelBackend) checks getattr(user, 'is_active', True) to reject inactive users",
      "auth.authenticate annotates the returned user with user.backend = backend_path (dotted import string); fires user_login_failed signal if no backend returns a user",
      "Back in middleware, auth.login stores SESSION_KEY, BACKEND_SESSION_KEY, and HASH_SESSION_KEY in the session",
      "_remove_invalid_user loads the stored backend from session, checks isinstance(stored_backend, RemoteUserBackend), and only calls logout if it is a RemoteUserBackend instance",
      "The middleware's clean_username loads the backend from session and delegates to the backend's clean_username method (catching AttributeError)",
      "PersistentRemoteUserMiddleware sets force_logout_if_no_header = False; AllowAllUsersRemoteUserBackend overrides user_can_authenticate to return True"
    ],
    "key_files": [
      "django/contrib/auth/backends.py",
      "django/contrib/auth/middleware.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/backends.py] django.contrib.auth.backends.RemoteUserBackend\ndjango.contrib.auth.backends.RemoteUserBackend:\n  This backend is to be used in conjunction with the ``RemoteUserMiddleware``\n    found in the middleware module of this package, and is used when the server\n    is handling authentication outside of Django.\n\n    By default, the ``authenticate`` method creates ``User`` objects for\n    usernames that don't already exist in the database. Subclasses can disable\n    this behavior by setting the ``create_unknown_user`` attribute to\n    ``False``.\n\ndjango.contrib.auth.backends.RemoteUserBackend.authenticate:\n  The username passed as ``remote_user`` is considered trusted. Return\n        the ``User`` object with the given username. Create a new ``User``\n        object if ``create_unknown_user`` is ``True``.\n\n        Return None if ``create_unknown_user`` is ``False`` and a ``User``\n        object with the given username is not found in the database.\n\ndjango.contrib.auth.backends.RemoteUserBackend.aauthenticate:\n  See authenticate().\n\ndjango.contrib.auth.backends.RemoteUserBackend.clean_username:\n  Perform any cleaning on the \"username\" prior to using it to get or\n        create the user object. Return the cleaned username.\n\n        By default, return the username unchanged.\n\ndjango.contrib.auth.backends.RemoteUserBackend.configure_user:\n  Configure a user and return the updated user.\n\n        By default, return the user unmodified.\n\ndjango.contrib.auth.backends.RemoteUserBackend.aconfigure_user:\n  See configure_user()",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 4 claims 'configure_user is always called, even for existing users' - the documentation says 'Configure a user and return the updated user. By default, return the user unmodified.' It doesn't explicitly state whether it's called for existing users or only new ones. This is a code-level detail that goes beyond the doc but could be a hallucination if incorrect.",
      "Point 9 describes 'RemoteUserMiddleware.clean_username()' delegating to the backend's clean_username - the documentation only describes RemoteUserBackend.clean_username, not the middleware's version. This is plausible but unverifiable from the given docs.",
      "Point 12 claims 'RemoteUserBackend inherits from ModelBackend' - the documentation doesn't explicitly state the inheritance hierarchy, though it mentions user_can_authenticate being inherited. This is a code-level detail beyond the doc scope.",
      "Point 1 claims the async variant uses 'HTTP_' + self.header - this is a specific implementation detail not covered in the documentation and could be incorrect.",
      "Many claims about internal implementation details (thread safety comments, signal firing, session keys, etc.) go well beyond what the documentation covers but are plausible code-level details."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Only one claim (about a comment explaining thread safety rationale for get_or_create) was pure doc leakage; one minor correction on inspect.signature vs django.utils.inspect.signature and a line number reference removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_28",
    "repo": "django",
    "question": "How does Django's AbstractBaseUser handle password storage, verification, hash upgrades, timing attack mitigation, and session authentication?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `AbstractBaseUser` class in `django/contrib/auth/base_user.py` implements the core authentication logic for Django's user model system. Here's how the system is designed:\n\n## Password Management\n\n**`set_password(raw_password)`** calls `make_password(raw_password)` from `django/contrib/auth/hashers.py` and stores the raw password in `self._password` (a transient attribute). When `save()` is called, if `_password` is not None, it triggers `password_validation.password_changed(self._password, self)` to notify validators, then clears `_password`.\n\n**`check_password(raw_password)`** defines an internal `setter` closure that re-hashes the password (for hash upgrade scenarios). Critically, this setter sets `self._password = None` so the password hash upgrade is NOT treated as a password change (skipping `password_changed()` on save). The method calls the module-level `check_password()` function in `hashers.py`, which delegates to `verify_password()`. That function uses `identify_hasher(encoded)` to determine the algorithm from the stored hash, compares it against the preferred (default) hasher, and determines if an upgrade is needed. If the password is correct AND must be updated, the setter is called to transparently upgrade the hash.\n\n**Timing attack mitigation**: In `verify_password()`, if the password is `None` or the hash is unusable (starts with `!`), or if the hasher can't be identified, it sets `fake_runtime = True` and runs `make_password(get_random_string(UNUSABLE_PASSWORD_SUFFIX_LENGTH))` (where UNUSABLE_PASSWORD_SUFFIX_LENGTH is 40) to create a fake computation before returning `False, False`.\n\n**`set_unusable_password()`** calls `make_password(None)`, which returns `UNUSABLE_PASSWORD_PREFIX` (`\"!\"`) + 40 random characters.\n\n**`is_password_usable(encoded)`** returns `True` unless the encoded password starts with `\"!\"`.\n\n## Session Auth Hash\n\n**`get_session_auth_hash()`** delegates to `_get_session_auth_hash()`, which uses `salted_hmac()` from `django/utils/crypto.py` with:\n- key_salt: `\"django.contrib.auth.models.AbstractBaseUser.get_session_auth_hash\"` (note: references `models` not `base_user` in the key_salt string)\n- value: `self.password`\n- algorithm: `\"sha256\"`\n- secret: defaults to `settings.SECRET_KEY` (when None is passed)\n\nThe `salted_hmac` function derives a key by hashing `key_salt + secret` with the specified algorithm, then uses that derived key in `hmac.new()` with the password as the message.\n\n**`get_session_auth_fallback_hash()`** is a generator that yields HMAC hashes computed with each secret in `settings.SECRET_KEY_FALLBACKS`.\n\n## Session Verification Flow\n\nIn `django/contrib/auth/__init__.py`, `get_user(request)` retrieves the user from the session and verifies the stored `HASH_SESSION_KEY` (`\"_auth_user_hash\"`) against `user.get_session_auth_hash()` using `constant_time_compare`. If verification fails with the current secret, it tries each fallback hash via `get_session_auth_fallback_hash()`. If a fallback matches, it cycles the session key and updates the stored hash to the current one. If nothing matches, the session is flushed and the user is set to `AnonymousUser()`.\n\n## Async Support\n\n**`acheck_password(raw_password)`** uses an async setter (which calls `await self.asave(update_fields=[\"password\"])`) and delegates to the module-level `acheck_password()`, which wraps `verify_password` via `sync_to_async(thread_sensitive=False)` and awaits the setter if needed.\n\n## Class Hierarchy\n\n`AbstractBaseUser` extends `models.Model` and is abstract. The `get_username()` method dynamically reads the attribute named by `USERNAME_FIELD`. The `normalize_username()` classmethod applies NFKC Unicode normalization to string usernames.",
    "rubric": [
      "Explains that set_password() calls make_password() and stores the raw password in _password, which triggers password_changed() on save() then clears _password",
      "Explains that check_password() defines a setter closure that re-hashes and sets _password=None so hash upgrades are not treated as password changes",
      "Explains that verify_password() identifies the hasher from the stored hash and compares against the preferred/default hasher to determine if an upgrade is needed",
      "Explains the timing attack mitigation: when password is None, hash is unusable, or hasher can't be identified, a fake make_password computation with a random 40-char string is run before returning False",
      "Explains that set_unusable_password() calls make_password(None) which returns '!' prefix plus 40 random characters, and is_password_usable checks for the '!' prefix",
      "Explains that get_session_auth_hash() uses salted_hmac with key_salt referencing 'django.contrib.auth.models.AbstractBaseUser.get_session_auth_hash', the password field as value, sha256 algorithm, and SECRET_KEY as default secret",
      "Explains that get_session_auth_fallback_hash() yields HMAC hashes for each secret in SECRET_KEY_FALLBACKS",
      "Explains the session verification flow in get_user(): compares stored hash against current auth hash, tries fallback hashes if no match (cycling the session key on fallback match), or flushes the session if nothing matches"
    ],
    "key_files": [
      "django/contrib/auth/base_user.py",
      "django/contrib/auth/hashers.py",
      "django/utils/crypto.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/base_user.py] django.contrib.auth.base_user.AbstractBaseUser\ndjango.contrib.auth.base_user.AbstractBaseUser.get_username:\n  Return the username for this User.\n\ndjango.contrib.auth.base_user.AbstractBaseUser.is_anonymous:\n  Always return False. This is a way of comparing User objects to\n        anonymous users.\n\ndjango.contrib.auth.base_user.AbstractBaseUser.is_authenticated:\n  Always return True. This is a way to tell if the user has been\n        authenticated in templates.\n\ndjango.contrib.auth.base_user.AbstractBaseUser.check_password:\n  Return a boolean of whether the raw_password was correct. Handles\n        hashing formats behind the scenes.\n\ndjango.contrib.auth.base_user.AbstractBaseUser.acheck_password:\n  See check_password().\n\ndjango.contrib.auth.base_user.AbstractBaseUser.has_usable_password:\n  Return False if set_unusable_password() has been called for this user.\n\ndjango.contrib.auth.base_user.AbstractBaseUser.get_session_auth_hash:\n  Return an HMAC of the password field.",
    "verification_verdict": "pass",
    "verification_issues": [
      "The documentation only provides docstrings/summaries for AbstractBaseUser methods, while the answer goes into extensive implementation detail (e.g., timing attack mitigation, session verification flow, async support, key rotation). These details are not present in the provided documentation but are plausible extrapolations from Django source code knowledge.",
      "The answer mentions set_password() which is not listed in the documentation excerpt, but is a real Django method \u2014 this is an extrapolation beyond the doc scope.",
      "The answer's description of is_anonymous ('Always return False') and is_authenticated ('Always return True') matches the documentation exactly.",
      "The answer's description of check_password matches the documentation's description of returning a boolean for raw_password correctness and handling hashing formats.",
      "The answer's description of acheck_password as delegating to check_password is consistent with the doc's 'See check_password()'.",
      "The answer's description of has_usable_password is consistent with the doc's 'Return False if set_unusable_password() has been called'.",
      "The answer's description of get_session_auth_hash as returning an HMAC of the password field matches the documentation.",
      "The answer's description of get_username matches the documentation's 'Return the username for this User'."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only the AbstractUser inheritance details from unshown models.py and some design rationale phrases needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_29",
    "repo": "django",
    "question": "What is Django's AuthenticationForm, how does it validate credentials, and how does it integrate with LoginView and the auth backends?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `AuthenticationForm` in Django is a form (inheriting directly from `forms.Form`, not `ModelForm`) that handles username/password authentication. It defines two fields: a `username` field using `UsernameField` (which performs NFKC Unicode normalization in its `to_python()` method and sets `autocapitalize: none` and `autocomplete: username` widget attributes) and a `password` field with `strip=False` and `autocomplete: current-password`.\n\nKey architectural details:\n\n1. **`__init__` method**: Stores the `request` parameter as `self.request` and initializes `self.user_cache = None`. It dynamically sets the username field's `max_length` from `UserModel._meta.get_field(UserModel.USERNAME_FIELD).max_length` (defaulting to 254 if None), and sets the label to `capfirst(self.username_field.verbose_name)` only if the field's label is currently None.\n\n2. **`clean()` method**: Decorated with `@sensitive_variables()` (with no arguments). It calls `django.contrib.auth.authenticate(self.request, username=username, password=password)`. If authentication returns None, it raises the error from `get_invalid_login_error()`. If authentication succeeds, it calls `self.confirm_login_allowed(self.user_cache)`.\n\n3. **`authenticate()` function** (in `django/contrib/auth/__init__.py`): Iterates through compatible backends (checking signature compatibility), calls `backend.authenticate(request, **credentials)`, and annotates the returned user with `user.backend = backend_path`. If all backends fail, it fires the `user_login_failed` signal with cleaned (sanitized) credentials.\n\n4. **`confirm_login_allowed()`**: Simply checks `user.is_active` and raises a `ValidationError` with code `'inactive'` if False. The `AdminAuthenticationForm` subclass (in `django/contrib/admin/forms.py`) extends this by calling `super().confirm_login_allowed(user)` first, then additionally checking `user.is_staff`.\n\n5. **`get_user()` method**: Simply returns `self.user_cache` \u2014 the user object cached during `clean()`.\n\n6. **`get_invalid_login_error()`**: Returns a `ValidationError` with code `'invalid_login'` and params `{'username': self.username_field.verbose_name}`, allowing the error message to include the username field's verbose name.\n\n7. **Integration with `LoginView`**: The `LoginView.get_form_kwargs()` injects `request` into form kwargs. After successful validation, `LoginView.form_valid()` calls `auth_login(self.request, form.get_user())` which persists the authentication in the session.\n\n8. **Backend interaction**: The default `ModelBackend.authenticate()` uses `UserModel._default_manager.get_by_natural_key(username)` and if the user doesn't exist, it still runs `UserModel().set_password(password)` in the except block. The backend's `user_can_authenticate()` method checks `getattr(user, \"is_active\", True)`, rejecting inactive users by default.",
    "rubric": [
      "AuthenticationForm inherits from forms.Form (not ModelForm) and defines username (UsernameField) and password fields",
      "UsernameField performs NFKC Unicode normalization in to_python() and sets autocapitalize/autocomplete widget attributes",
      "__init__ stores request and initializes user_cache to None, dynamically sets username max_length from the User model's USERNAME_FIELD (defaulting to 254)",
      "clean() is decorated with @sensitive_variables() and calls django.contrib.auth.authenticate() with request, username, and password",
      "If authenticate returns None, get_invalid_login_error() raises a ValidationError with code 'invalid_login'; otherwise confirm_login_allowed() is called",
      "confirm_login_allowed() checks user.is_active and raises ValidationError with code 'inactive' if False",
      "AdminAuthenticationForm extends confirm_login_allowed to additionally check user.is_staff",
      "get_user() returns self.user_cache, the user object cached during clean()",
      "LoginView.get_form_kwargs() injects request into form kwargs; form_valid() calls auth_login(request, form.get_user())",
      "The authenticate() function iterates backends, annotates user.backend on success, and fires user_login_failed signal on failure",
      "ModelBackend.authenticate() uses get_by_natural_key and runs set_password on a dummy user if the user doesn't exist (timing attack mitigation); user_can_authenticate checks is_active"
    ],
    "key_files": [
      "django/contrib/auth/forms.py",
      "django/contrib/auth/__init__.py",
      "django/contrib/auth/views.py",
      "django/contrib/auth/backends.py",
      "django/contrib/admin/forms.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/forms.py] django.contrib.auth.forms.AuthenticationForm\ndjango.contrib.auth.forms.AuthenticationForm:\n  Base class for authenticating users. Extend this to get a form that accepts\n    username/password logins.\n\ndjango.contrib.auth.forms.AuthenticationForm.__init__:\n  The 'request' parameter is set for custom auth use by subclasses.\n        The form data comes in via the standard 'data' kwarg.\n\ndjango.contrib.auth.forms.AuthenticationForm.confirm_login_allowed:\n  Controls whether the given User may log in. This is a policy setting,\n        independent of end-user authentication. This default behavior is to\n        allow login by active users, and reject login by inactive users.\n\n        If the given user cannot log in, this method should raise a\n        ``ValidationError``.\n\n        If the given user may log in, this method should return None.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for design rationale descriptions like 'timing-attack mitigation' and decorator semantics.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_30",
    "repo": "django",
    "question": "Why does Django's PasswordResetForm use so many layered security mechanisms \u2014 double email matching, exception swallowing, special PK encoding, and HMAC token generation?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `PasswordResetForm` in Django implements several deliberate design decisions:\n\n**1. Double-defense against Unicode homograph attacks in `get_users()`:**\nThe `get_users()` method applies TWO layers of email matching. First, it queries the database with `__iexact` lookup (`email_field_name__iexact`). Then, it applies a second Python-level check using `_unicode_ci_compare()`, which performs NFKC normalization and `casefold()` comparison. The tests `test_user_email_unicode_collision` and `test_user_email_unicode_collision_nonexistent` verify this: if 'mike@example.org' and 'm\u0131ke@example.org' both exist, a reset for 'm\u0131ke@example.org' only sends to that exact user. If only 'mike@example.org' exists, requesting 'm\u0131ke@example.org' sends zero emails.\n\n**2. Exception swallowing with logging in `send_mail()`:**\nThe `send_mail()` method wraps `email_message.send()` in a `try/except Exception` block. When sending fails, instead of crashing, it logs the failure via `logger.exception()` to the `\"django.contrib.auth\"` logger, including the user's **primary key** (not the email) in the message: `\"Failed to send password reset email to %s\", context[\"user\"].pk`.\n\n**3. UID encoding uses `value_to_string` on the PK field, not just `str(user.pk)`:**\nIn `save()`, the user's primary key is encoded as `force_bytes(UserModel._meta.pk.value_to_string(user))` before being base64-encoded. This uses the model field's serialization method rather than a simple string conversion.\n\n**4. Token invalidation mechanism:**\nThe `default_token_generator` (a `PasswordResetTokenGenerator` instance from `django/contrib/auth/tokens.py`) generates tokens using an HMAC that incorporates the user's password hash, last_login timestamp, email, and pk. Since the hash includes `user.password` and `last_login`, the token is invalidated when the password changes or when the user logs in. The token also respects `settings.PASSWORD_RESET_TIMEOUT`. The hash is truncated by taking every other character (`hexdigest()[::2]`).\n\n**5. `get_users()` returns a generator, not a list:**\nThe method returns a generator expression that lazily evaluates the `has_usable_password()` and `_unicode_ci_compare()` checks. This means the database query (`active_users`) is evaluated once, but the Python-level filtering is done lazily as the `save()` method iterates over users.\n\n**6. `domain_override` sets both `site_name` and `domain` to the same value:**\nWhen `domain_override` is provided to `save()`, both `site_name` and `domain` are set to that string (`site_name = domain = domain_override`). Without it, `get_current_site(request)` is used.\n\n**7. Subject newline stripping:**\nIn `send_mail()`, after rendering the subject template, all newlines are removed with `\"\".join(subject.splitlines())`.",
    "rubric": [
      "Explains the double-defense against Unicode homograph attacks: database __iexact query followed by Python-level NFKC normalization + casefold comparison via _unicode_ci_compare()",
      "Mentions that this prevents confusable characters (e.g., 'mike' vs 'm\u0131ke') from triggering password resets for the wrong user, and sends zero emails if only the non-matching user exists",
      "Explains that send_mail() wraps email sending in try/except Exception and logs failures via logger.exception() instead of crashing, logging the user's PK (not email)",
      "Notes that the UID encoding uses the PK field's value_to_string() method rather than a simple str(user.pk) conversion",
      "Explains the token generation uses HMAC incorporating user.password, last_login, email, and pk \u2014 so the token auto-invalidates when the password changes or user logs in",
      "Mentions that the HMAC hash is truncated by taking every other character (hexdigest()[::2]) to shorten the URL",
      "Notes that get_users() returns a generator (lazy evaluation) rather than a list",
      "Mentions that subject newlines are stripped and that domain_override sets both site_name and domain to the same value"
    ],
    "key_files": [
      "django/contrib/auth/forms.py",
      "django/contrib/auth/tokens.py",
      "django/contrib/auth/base_user.py",
      "django/contrib/auth/views.py",
      "tests/auth_tests/test_forms.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/forms.py] django.contrib.auth.forms.PasswordResetForm\ndjango.contrib.auth.forms.PasswordResetForm.send_mail:\n  Send a django.core.mail.EmailMultiAlternatives to `to_email`.\n\ndjango.contrib.auth.forms.PasswordResetForm.get_users:\n  Given an email, return matching user(s) who should receive a reset.\n\n        This allows subclasses to more easily customize the default policies\n        that prevent inactive users and users with unusable passwords from\n        resetting their password.\n\ndjango.contrib.auth.forms.PasswordResetForm.save:\n  Generate a one-use only link for resetting password and send it to the\n        user.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 7 self-contradicts: states newlines are removed with `\"\\n\".join(subject.splitlines())` then corrects itself to `\"\".join(subject.splitlines())`. The actual Django implementation uses `\"\\n\".join(subject.splitlines())` which joins lines with a single newline (not removes them) \u2014 but the stated intent of removing newlines is incorrect if `\"\\n\".join()` is used. However, looking at actual Django code, it uses `' '.join(subject.splitlines())` or similar. This self-contradiction and potential inaccuracy is notable.",
      "Most claims (Unicode homograph defense, exception swallowing with logging, UID encoding via value_to_string, token generation details, generator return type, domain_override behavior, subject newline stripping) go far beyond what is stated in the provided documentation. These are plausible code-level implementation details but cannot be verified from the given docstrings alone.",
      "No claims directly contradict the documentation \u2014 the docstrings are very minimal and the answer's claims about send_mail sending EmailMultiAlternatives and get_users returning matching users and save generating one-use links are all consistent with the docs."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about design rationale (Unicode TR36 reference, security reasoning, RFC justification) rely on documentation and were removed or rewritten; code-verifiable claims about the actual implementation were kept.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_31",
    "repo": "django",
    "question": "Where does the mod_wsgi authentication handler mitigate timing attacks for nonexistent or inactive users, and how does that flow through the password hashing system?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `_get_user` function is located in `django/contrib/auth/handlers/modwsgi.py` at line 7. It is called exclusively by the `check_password` function in the same file, which serves as the mod_wsgi authentication check handler.\n\nThe data flow works as follows:\n\n1. `_get_user` uses `UserModel._default_manager.get_by_natural_key(username)` to look up the user. The `UserModel` is obtained at module level via `auth.get_user_model()`, which resolves to `django_apps.get_model(settings.AUTH_USER_MODEL, require_ready=False)` (in `django/contrib/auth/__init__.py`).\n\n2. `get_by_natural_key` is defined in `BaseUserManager` (in `django/contrib/auth/base_user.py`) and performs `self.get(**{self.model.USERNAME_FIELD: username})`. For the default `User` model, `USERNAME_FIELD` is `\"username\"` (defined in `django/contrib/auth/models.py`).\n\n3. When the user is not found or is inactive, `UserModel().set_password(\"\")` is called, which creates a throwaway unsaved user instance and calls `set_password` from `AbstractBaseUser`. This in turn calls `make_password(raw_password)` from `django/contrib/auth/hashers.py`, which invokes `get_hasher('default')` to get the first hasher from `settings.PASSWORD_HASHERS`, then calls `hasher.encode(password, salt)`. This results in the same hashing work occurring whether or not the user exists.\n\n4. The caller `check_password` in `modwsgi.py` wraps the call with `db.reset_queries()` and `db.close_old_connections()` in a try/finally block. It returns `None` if user doesn't exist, `True` if password is correct, `False` if incorrect.\n\n5. In the test file `tests/auth_tests/test_handlers.py`, `test_check_password_fake_runtime` verifies the timing-consistent behavior by mocking `hasher.encode` and asserting it is called exactly once for every case \u2014 existing user with right/wrong password, inactive user, user with unusable password, and nonexistent user.",
    "rubric": [
      "Identifies `_get_user` in the mod_wsgi handler as the location where timing attack mitigation occurs",
      "Explains that `_get_user` calls `UserModel().set_password(\"\")` on a throwaway instance when the user is None (not found or inactive)",
      "Describes the user lookup via `UserModel._default_manager.get_by_natural_key(username)`, which uses `USERNAME_FIELD` in `BaseUserManager`",
      "Explains that `set_password` calls `make_password`, which invokes the default hasher's `encode` method to perform the same hashing work as a real check",
      "Mentions the `check_password` function in the same module as the caller, which wraps the call with `db.reset_queries()` and `db.close_old_connections()`",
      "Notes the test (`test_check_password_fake_runtime`) verifies `hasher.encode` is called exactly once for all cases including nonexistent/inactive users"
    ],
    "key_files": [
      "django/contrib/auth/handlers/modwsgi.py",
      "django/contrib/auth/base_user.py",
      "django/contrib/auth/hashers.py",
      "tests/auth_tests/test_handlers.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/handlers/modwsgi.py] django.contrib.auth.handlers.modwsgi._get_user\nReturn the UserModel instance for `username`.\n\n    If no matching user exists, or if the user is inactive, return None, in\n    which case the default password hasher is run to mitigate timing attacks.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage: references to Django ticket #20760, mod_wsgi protocol rationale, and request/response cycle explanation are not in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_33",
    "repo": "django",
    "question": "What are the full implementation details of PBKDF2PasswordHasher, including its dependencies on the crypto module, salt generation, verification, and update logic?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `PBKDF2PasswordHasher` class in `django/contrib/auth/hashers.py` extends `BasePasswordHasher` and is the default password hasher in Django's `PASSWORD_HASHERS` setting (listed first in `django/conf/global_settings.py`).\n\n**Key implementation details:**\n\n1. **Algorithm identifier**: `\"pbkdf2_sha256\"`, with `iterations = 1_500_000` and `digest = hashlib.sha256`.\n\n2. **Encoding format**: The `encode()` method produces strings in the format `\"pbkdf2_sha256$<iterations>$<salt>$<base64_hash>\"`. It calls `self._check_encode_args(password, salt)` (inherited from `BasePasswordHasher`) to validate that password is not None and salt doesn't contain `$`. The password and salt are converted to strings via `force_str()`, then passed to `django.utils.crypto.pbkdf2()`.\n\n3. **The `pbkdf2()` dependency** (in `django/utils/crypto.py`): This function delegates to `hashlib.pbkdf2_hmac()`. When called from `PBKDF2PasswordHasher.encode()`, `dklen` is not passed (defaults to 0), which is converted to `None` via `dklen = dklen or None`, causing `hashlib.pbkdf2_hmac` to use the digest's natural output length. The raw bytes are then base64-encoded in the hasher's `encode()` method.\n\n4. **Salt generation**: Inherited from `BasePasswordHasher.salt()`, which uses `salt_entropy = 128` bits. The salt length is calculated as `math.ceil(128 / math.log2(62))` = 22 characters, drawn from `RANDOM_STRING_CHARS` (62 characters: a-z, A-Z, 0-9) using `secrets.choice()`.\n\n5. **Verification**: `verify()` decodes the stored hash to extract salt and iterations, re-encodes the candidate password, and uses `constant_time_compare()` (from `django.utils.crypto`) which delegates to `secrets.compare_digest(force_bytes(val1), force_bytes(val2))`.\n\n6. **must_update()**: Returns True if either the stored iterations differ from the current `self.iterations` value, OR if the salt has insufficient entropy (checked via `must_update_salt()` which compares `len(salt) * math.log2(len(RANDOM_STRING_CHARS))` against `salt_entropy=128`).\n\n7. **harden_runtime()**: Computes `extra_iterations = self.iterations - decoded['iterations']` and, if positive, calls `self.encode(password, decoded['salt'], extra_iterations)` to close the timing gap.\n\n8. **Subclass**: `PBKDF2SHA1PasswordHasher` inherits from `PBKDF2PasswordHasher`, overriding only `algorithm = \"pbkdf2_sha1\"` and `digest = hashlib.sha1`. It reuses all other methods.\n\n9. **The `get_hashers()` function** uses `@functools.lru_cache` and is invalidated via the `reset_hashers` signal receiver listening for `setting_changed` on `\"PASSWORD_HASHERS\"`.",
    "rubric": [
      "Identifies algorithm as 'pbkdf2_sha256' with iterations=1_500_000 and digest=hashlib.sha256",
      "Explains encode() format is 'pbkdf2_sha256$<iterations>$<salt>$<base64_hash>' and calls _check_encode_args before hashing",
      "Traces into django.utils.crypto.pbkdf2() showing it calls hashlib.pbkdf2_hmac() with dklen converted from 0 to None (digest's natural output length)",
      "Explains salt generation: inherited from BasePasswordHasher.salt() using salt_entropy=128 bits, computing ceil(128/log2(62))=22 characters from RANDOM_STRING_CHARS via secrets.choice()",
      "Describes verify() decoding the stored hash, re-encoding with extracted salt and iterations, then using constant_time_compare (which delegates to secrets.compare_digest)",
      "Explains must_update() returns True if iterations differ from self.iterations OR if salt has insufficient entropy (via must_update_salt checking len(salt)*log2(len(RANDOM_STRING_CHARS)) < 128)",
      "Describes harden_runtime() computing extra_iterations = self.iterations - decoded['iterations'] and running encode() with those extra iterations if positive",
      "Mentions PBKDF2SHA1PasswordHasher subclass overriding only algorithm='pbkdf2_sha1' and digest=hashlib.sha1",
      "Notes PBKDF2PasswordHasher is the default hasher (first in PASSWORD_HASHERS in global_settings.py)",
      "Mentions get_hashers() uses @functools.lru_cache and is invalidated by reset_hashers signal receiver on PASSWORD_HASHERS setting changes"
    ],
    "key_files": [
      "django/contrib/auth/hashers.py",
      "django/utils/crypto.py",
      "django/conf/global_settings.py",
      "django/utils/encoding.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/hashers.py] django.contrib.auth.hashers.PBKDF2PasswordHasher\ndjango.contrib.auth.hashers.PBKDF2PasswordHasher:\n  Secure password hashing using the PBKDF2 algorithm (recommended)\n\n    Configured to use PBKDF2 + HMAC + SHA256.\n    The result is a 64 byte binary string. Iterations may be changed\n    safely but you must rename the algorithm if you change SHA256.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states 'The result is a 64 byte binary string', but the answer claims the natural output length is 32 bytes for SHA-256. However, the answer refers to the raw bytes before base64 encoding, and 32 bytes base64-encoded would be 44 characters, not 64 bytes. The documentation's '64 byte binary string' likely refers to the hex-encoded output (32 bytes * 2 = 64 hex characters) or could be referring to the base64-encoded hash. This is a potential discrepancy between the answer's claim of 32 bytes and the documentation's claim of 64 bytes.",
      "Most implementation details (iterations count, salt generation specifics, exact function delegation paths, signal handling, etc.) go well beyond what the documentation provides and cannot be verified from it, though they are plausible code-level details."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are directly verifiable from the stripped source code; only minor wording about '32 bytes for SHA-256' and exact must_update_salt comparison were slightly adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_32",
    "repo": "django",
    "question": "How does Django's password hashing framework generate salts, validate inputs, and implement runtime hardening across its different hashers?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `BasePasswordHasher` class in `django/contrib/auth/hashers.py` implements a password hashing framework with the following key implementation details:\n\n**Salt Generation Algorithm (`salt()` method):**\nThe salt is generated by computing the number of characters needed to achieve at least `salt_entropy` bits (default 128) of entropy. The formula is `math.ceil(self.salt_entropy / math.log2(len(RANDOM_STRING_CHARS)))`. Since `RANDOM_STRING_CHARS` contains 62 characters (`abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789`), this results in `math.ceil(128 / 5.954...) = 22` characters. The actual random string is generated by `get_random_string()` from `django/utils/crypto.py`, which uses `secrets.choice()` for each character.\n\n**Encode Argument Validation (`_check_encode_args()`):**\nBefore encoding, the base class provides `_check_encode_args(self, password, salt)` which raises `TypeError` if password is None, and `ValueError` if salt is falsy or contains a `$` character. The salt is coerced through `force_str()` before checking for `$`.\n\n**PBKDF2PasswordHasher Implementation:**\n- Uses `hashlib.sha256` digest and 1,500,000 iterations by default\n- `encode()` calls Django's `pbkdf2()` wrapper which delegates to `hashlib.pbkdf2_hmac(digest().name, password, salt, iterations, dklen)` from `django/utils/crypto.py`\n- The hash is base64-encoded and stored as `algorithm$iterations$salt$hash`\n- `verify()` re-encodes the password with the same salt and iterations, then uses `constant_time_compare()` (which wraps `secrets.compare_digest`)\n- `harden_runtime()` computes `extra_iterations = self.iterations - decoded['iterations']` and runs `encode()` with those extra iterations if positive\n- `must_update()` returns True if either the iteration count doesn't match or the salt has insufficient entropy (checked via `must_update_salt()`)\n\n**BCryptSHA256PasswordHasher Implementation:**\n- Overrides `salt()` to use `bcrypt.gensalt(self.rounds)` (default 12 rounds) instead of the base class method\n- Pre-hashes the password with SHA256 (hex-encoded via `binascii.hexlify`) before passing to bcrypt\n- `harden_runtime()` uses logarithmic scaling: `diff = 2**(self.rounds - int(rounds)) - 1`, then calls `encode()` that many times\n\n**Argon2PasswordHasher** defines `harden_runtime()` as a no-op (`pass`).\n\n**`verify_password()` timing behavior:**\nWhen the password is None, the encoded hash is unusable, or the hasher cannot be identified, the function runs `make_password(get_random_string(UNUSABLE_PASSWORD_SUFFIX_LENGTH))` before returning `(False, False)`. The `harden_runtime()` is only called when `is_correct` is False, the hasher hasn't changed, and `must_update` is True.\n\n**`must_update_salt()` function:**\nChecks if the salt's entropy is below the expected threshold: `len(salt) * math.log2(len(RANDOM_STRING_CHARS)) < expected_entropy`.",
    "rubric": [
      "Explains salt generation: computes char_count via math.ceil(salt_entropy / math.log2(len(RANDOM_STRING_CHARS))), yielding 22 characters from a 62-char alphabet for 128-bit entropy, using secrets.choice via get_random_string",
      "Describes _check_encode_args: raises TypeError if password is None, raises ValueError if salt is falsy or contains '$' (after force_str coercion)",
      "Explains PBKDF2PasswordHasher: uses hashlib.sha256, 1,500,000 iterations, stores as algorithm$iterations$salt$hash with base64-encoded hash, verify uses constant_time_compare (wrapping secrets.compare_digest)",
      "Explains PBKDF2 harden_runtime: computes extra_iterations = self.iterations - decoded['iterations'] and runs encode with those extra iterations if positive",
      "Explains PBKDF2 must_update: returns True if iterations don't match current setting or if salt has insufficient entropy via must_update_salt()",
      "Explains BCryptSHA256PasswordHasher: overrides salt() to use bcrypt.gensalt(self.rounds), pre-hashes password with SHA256 via binascii.hexlify before bcrypt",
      "Explains BCrypt harden_runtime: uses logarithmic scaling diff = 2**(self.rounds - int(rounds)) - 1, calls encode that many times",
      "Notes Argon2PasswordHasher's harden_runtime is a no-op (pass)",
      "Explains verify_password timing defense: runs make_password with random string when password is None, encoded is unusable, or hasher can't be identified; harden_runtime only called when is_correct is False, hasher unchanged, and must_update is True",
      "Explains must_update_salt: checks len(salt) * math.log2(len(RANDOM_STRING_CHARS)) < expected_entropy"
    ],
    "key_files": [
      "django/contrib/auth/hashers.py",
      "django/utils/crypto.py",
      "django/conf/global_settings.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/hashers.py] django.contrib.auth.hashers.BasePasswordHasher\ndjango.contrib.auth.hashers.BasePasswordHasher:\n  Abstract base class for password hashers\n\n    When creating your own hasher, you need to override algorithm,\n    verify(), encode() and safe_summary().\n\n    PasswordHasher objects are immutable.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.salt:\n  Generate a cryptographically secure nonce salt in ASCII with an entropy\n        of at least `salt_entropy` bits.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.verify:\n  Check if the given password is correct.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.encode:\n  Create an encoded database value.\n\n        The result is normally formatted as \"algorithm$salt$hash\" and\n        must be fewer than 128 characters.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.decode:\n  Return a decoded database value.\n\n        The result is a dictionary and should contain `algorithm`, `hash`, and\n        `salt`. Extra keys can be algorithm specific like `iterations` or\n        `work_factor`.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.safe_summary:\n  Return a summary of safe values.\n\n        The result is a dictionary and will be used where the password field\n        must be displayed to construct a safe representation of the password.\n\ndjango.contrib.auth.hashers.BasePasswordHasher.harden_runtime:\n  Bridge the runtime gap between the work factor supplied in `encoded`\n        and the work factor suggested by this hasher.\n\n        Taking PBKDF2 as an example, if `encoded` contains 20000 iterations and\n        `self.iterations` is 30000, this method should run password through\n        another 10000 iterations of PBKDF2. Similar approaches should exist\n        for any hasher that has a work factor. If not, this method should be\n        defined as a no-op to silence the warning.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (line numbers, specific iteration counts, specific function implementations) that are not present in the provided documentation. These could be accurate based on the actual source code, but cannot be verified from the given documentation alone.",
      "The documentation states encode() result 'must be fewer than 128 characters' - the answer doesn't contradict this but doesn't mention this constraint either.",
      "The documentation says 'When creating your own hasher, you need to override algorithm, verify(), encode() and safe_summary()' - the answer doesn't mention that 'algorithm' needs to be overridden as an attribute.",
      "The answer claims PBKDF2 uses 1,500,000 iterations by default - this cannot be verified from the documentation provided, though it's a plausible code-level detail.",
      "The answer claims BCrypt uses 12 rounds by default - cannot be verified from docs but is plausible.",
      "The answer references 'verify_password()' function but the documentation only mentions 'verify()' method on BasePasswordHasher.",
      "The harden_runtime() description in the answer (PBKDF2 example with extra iterations) is consistent with the documentation's description of bridging the runtime gap."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale explanations (timing attack prevention, bcrypt truncation workaround, harden_runtime purpose) are documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_34",
    "repo": "django",
    "question": "Why does `get_default_username` have so many early returns and guard checks before its database query?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `get_default_username` function in `django/contrib/auth/management/__init__.py` employs several performance optimizations to avoid unnecessary work, particularly expensive database queries:\n\n1. **Lazy model import inside the function body**: `from django.contrib.auth import models as auth_app` is placed inside the function rather than at module level. Since `django/contrib/auth/apps.py` imports `create_permissions` and `rename_permissions_after_model_rename` from this same management module at import time, importing auth models at module level would risk circular imports or premature model loading.\n\n2. **The `check_db` parameter defaults to `True` but can be set to `False`** to entirely skip the database query.\n\n3. **Short-circuit evaluation**: `if check_db and default_username:` \u2014 the `and default_username` part ensures that if the username string is empty (after NFKD normalization, ASCII encoding, space removal, and lowercasing), the function skips the database query entirely and just returns the empty string. This avoids a pointless `User.objects.get(username='')` call.\n\n4. **Multiple early `return ''` exits** before the database query: if `User._meta.swapped` is true, if `UnicodeDecodeError` occurs during normalization, or if the username fails field validator checks. Each of these prevents the expensive DB hit from ever executing.\n\n5. **The database query itself uses `.get()` rather than `.filter().exists()`** \u2014 it catches `User.DoesNotExist` to determine the username is available (the common/happy path), only returning empty string in the `else` clause when the user IS found. This is semantically correct since `.get()` will raise the exception on the common path (username not taken), avoiding result set construction overhead.",
    "rubric": [
      "Mentions the lazy/deferred import of auth models inside the function body to avoid circular imports (since apps.py imports from the same management module at import time)",
      "Explains the `check_db` parameter allows callers to skip the database query entirely",
      "Explains the short-circuit `if check_db and default_username` avoids querying the DB when the username is empty after normalization",
      "Identifies early returns for swapped User model, UnicodeDecodeError during normalization, or validation failure \u2014 each preventing unnecessary DB hits",
      "Notes that the DB lookup uses .get() catching DoesNotExist (the happy/common path) rather than .filter().exists(), returning empty string only if the user already exists"
    ],
    "key_files": [
      "django/contrib/auth/management/__init__.py",
      "django/contrib/auth/apps.py",
      "django/contrib/auth/models.py",
      "django/contrib/auth/validators.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/management/__init__.py] django.contrib.auth.management.get_default_username\nTry to determine the current system user's username to use as a default.\n\n    :param check_db: If ``True``, requires that the username does not match an\n        existing ``auth.User`` (otherwise returns an empty string).\n    :param database: The database where the unique check will be performed.\n    :returns: The username, or an empty string if no username can be\n        determined or the suggested username is already taken.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage from a stripped comment explaining the lazy import rationale and a reference to test code not in the stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_35",
    "repo": "django",
    "question": "Where is `_validate_username` defined and called in the createsuperuser command, and how does error handling differ between interactive and non-interactive mode?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `_validate_username` method is defined in `django/contrib/auth/management/commands/createsuperuser.py` within the `Command` class, which inherits from `django.core.management.base.BaseCommand`. It is called in three distinct locations within the same file's `handle` method:\n\n1. In interactive mode, when a username is provided via command-line options (validates before prompting)\n2. In interactive mode, inside the `while username is None` loop (validates after each user prompt)\n3. In non-interactive mode, after resolving the username from options or environment variable\n\nThe key behavioral difference: in interactive mode, validation failure writes the error to `self.stderr` and sets `username = None` to re-prompt the user. In non-interactive mode, validation failure raises a `CommandError`, which terminates the command.\n\n`_validate_username` relies on the `username_is_unique` cached property, which checks both `self.username_field.unique` and the model's `total_unique_constraints` for single-field UniqueConstraints. The uniqueness check uses `BaseUserManager.get_by_natural_key()` (defined in `django/contrib/auth/base_user.py`), which performs `self.get(**{self.model.USERNAME_FIELD: username})`. The method also calls `self.username_field.clean(username, None)` which invokes `Field.clean()` from `django/db/models/fields/__init__.py`.",
    "rubric": [
      "_validate_username is defined in the Command class in the createsuperuser management command, which inherits from BaseCommand",
      "It is called in three places within the handle method: (1) when username is provided via command-line options in interactive mode, (2) inside the while loop after prompting in interactive mode, (3) in non-interactive mode after resolving the username",
      "In interactive mode, validation failure writes the error to self.stderr and sets username = None to re-prompt the user",
      "In non-interactive mode, validation failure raises a CommandError, terminating the command",
      "The username_is_unique cached property checks both self.username_field.unique and single-field UniqueConstraints from total_unique_constraints",
      "The uniqueness check uses get_by_natural_key(), which performs a get() lookup using the model's USERNAME_FIELD",
      "The method also calls self.username_field.clean(username, None) to validate the field value"
    ],
    "key_files": [
      "django/contrib/auth/management/commands/createsuperuser.py",
      "django/contrib/auth/base_user.py",
      "django/db/models/options.py",
      "django/db/models/fields/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/management/commands/createsuperuser.py] django.contrib.auth.management.commands.createsuperuser.Command\ndjango.contrib.auth.management.commands.createsuperuser.Command.get_input_data:\n  Override this method if you want to customize data inputs or\n        validation exceptions.\n\ndjango.contrib.auth.management.commands.createsuperuser.Command._validate_username:\n  Validate username. If invalid, return a string error message.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive specific details (line numbers, method calls, cached properties, behavioral differences) that are not supported by the provided documentation, which only contains brief docstrings.",
      "The documentation only states that _validate_username should 'Validate username. If invalid, return a string error message.' The answer's claims about specific line numbers, call locations, and behavioral differences cannot be verified from the provided documentation.",
      "While the claims may be accurate based on the actual source code, they go far beyond what the documentation snippet can verify."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments needed for truncated implementation details.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_37",
    "repo": "django",
    "question": "What is the architecture of RemoteUserMiddleware and PersistentRemoteUserMiddleware, including their class hierarchy and how they handle missing headers?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "`PersistentRemoteUserMiddleware` is a minimal subclass of `RemoteUserMiddleware` defined in `django/contrib/auth/middleware.py`. Its entire body is a single attribute override: `force_logout_if_no_header = False` (compared to `True` in the parent `RemoteUserMiddleware`). This is the mechanism by which it keeps users authenticated even when the `REMOTE_USER` header is absent from subsequent requests.\n\n**Architecture and class hierarchy:**\n\n1. **`RemoteUserMiddleware`** (the parent) does NOT inherit from `MiddlewareMixin`. It is a standalone middleware class that directly defines `__init__`, `__call__`, `__acall__`, `process_request`, and `aprocess_request`. It sets `sync_capable = True` and `async_capable = True` as class attributes, and detects async mode via `iscoroutinefunction(get_response)` in `__init__`.\n\n2. **`PersistentRemoteUserMiddleware`** inherits from `RemoteUserMiddleware` and only overrides `force_logout_if_no_header = False`.\n\n3. The `force_logout_if_no_header` flag is checked in `process_request` (sync) and `aprocess_request` (async). When the header key lookup raises `KeyError`, the middleware checks `self.force_logout_if_no_header`. If `True` (default in `RemoteUserMiddleware`), it calls `_remove_invalid_user()` on authenticated users, which calls `auth.logout()` if the stored backend is an instance of `RemoteUserBackend`. If `False` (in `PersistentRemoteUserMiddleware`), it simply returns without logging out.\n\n4. **Notable async difference**: The sync `process_request` reads the header via `request.META[self.header]`, while the async `aprocess_request` reads it via `request.META[\"HTTP_\" + self.header]`. This means the async path prepends `\"HTTP_\"` to the header name.\n\n5. **`_remove_invalid_user`** and its async counterpart `_aremove_invalid_user` perform a safety check: they only call `auth.logout()` if the stored session backend is an instance of `RemoteUserBackend` (imported from `django.contrib.auth.backends`). If the backend fails to load (raises `ImportError`), they also call logout.\n\n6. **`RemoteUserBackend`** (in `backends.py`) extends `ModelBackend` (which extends `BaseBackend`). It has `create_unknown_user = True` by default, and its `authenticate()` method takes `remote_user` as a keyword argument (not username/password). It uses `UserModel._default_manager.get_or_create()` for user creation. The `configure_user()` method returns the user unmodified by default.\n\n7. The `clean_username` method on `RemoteUserMiddleware` delegates to the backend stored in the session key `auth.BACKEND_SESSION_KEY`, calling `backend.clean_username()` if it exists and silently catching `AttributeError` if it doesn't.\n\n8. When authentication succeeds, both sync and async paths call `auth.login(request, user)` / `auth.alogin(request, user)` to persist the user in the session.\n\n9. The `RemoteUserMiddleware.header` default is `\"REMOTE_USER\"`.",
    "rubric": [
      "States that PersistentRemoteUserMiddleware is a subclass of RemoteUserMiddleware that only overrides force_logout_if_no_header to False",
      "Explains that RemoteUserMiddleware does NOT inherit from MiddlewareMixin but is a standalone middleware class with __init__, __call__, __acall__, process_request, and aprocess_request",
      "Mentions that RemoteUserMiddleware sets sync_capable = True and async_capable = True, and detects async via iscoroutinefunction(get_response)",
      "Explains that force_logout_if_no_header is checked when the header key raises KeyError: if True, calls _remove_invalid_user; if False, simply returns",
      "Notes the async difference: sync process_request uses request.META[self.header] while aprocess_request uses request.META['HTTP_' + self.header]",
      "Describes that _remove_invalid_user only calls auth.logout if the stored backend is an instance of RemoteUserBackend (or if it fails to load with ImportError)",
      "Identifies RemoteUserBackend as extending ModelBackend (which extends BaseBackend), with create_unknown_user = True and authenticate() taking remote_user as argument",
      "Mentions that the default header value is 'REMOTE_USER'"
    ],
    "key_files": [
      "django/contrib/auth/middleware.py",
      "django/contrib/auth/backends.py",
      "tests/auth_tests/test_remote_user.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/middleware.py] django.contrib.auth.middleware.PersistentRemoteUserMiddleware\ndjango.contrib.auth.middleware.PersistentRemoteUserMiddleware:\n  Middleware for web-server provided authentication on logon pages.\n\n    Like RemoteUserMiddleware but keeps the user authenticated even if\n    the ``request.META`` key is not found in the request. Useful for\n    setups when the external authentication is only expected to happen\n    on some \"logon\" URL and the rest of the application wants to use\n    Django's authentication mechanism.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims go far beyond what the documentation snippet covers, but they appear to be plausible code-level details about Django's actual implementation rather than contradictions of the provided documentation.",
      "Claim about async path prepending 'HTTP_' to the header name (point 4) is a very specific implementation detail that cannot be verified from the documentation \u2014 this could be a hallucination if incorrect in the actual source code, but it does not contradict the provided doc.",
      "The specific line numbers cited (e.g., line 264, line 149, line 192) cannot be verified from the documentation provided."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only line numbers and a 'thread-safe' characterization needed minor adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_39",
    "repo": "django",
    "question": "Where does control flow go in `UserPassesTestMixin.dispatch()` when the test fails, and how does the redirect URL get constructed?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "In `UserPassesTestMixin`, the data/control flow during `dispatch()` follows this path:\n\n1. **`dispatch(self, request, *args, **kwargs)`** (line 132 of `django/contrib/auth/mixins.py`) is the entry point, called when a request hits the view.\n\n2. It calls **`self.get_test_func()()`** (line 133) \u2014 note the double parentheses: `get_test_func()` returns a callable (by default `self.test_func`), which is then immediately invoked. This two-step indirection allows subclasses to override `get_test_func()` to return a different callable.\n\n3. If the test function returns a falsy value, control flows to **`self.handle_no_permission()`** (line 135), which is inherited from `AccessMixin` (line 47).\n\n4. Inside `handle_no_permission()` (lines 47\u201365), there's a branching point:\n   - If `self.raise_exception` is `True` **or** `self.request.user.is_authenticated` is `True`, it raises `PermissionDenied` with the message from `get_permission_denied_message()`.\n   - Otherwise (unauthenticated user without `raise_exception`), it builds the redirect URL by calling `self.get_login_url()` (which falls back to `settings.LOGIN_URL`), compares the scheme/netloc of the login URL vs. the current request URL, and calls `redirect_to_login()` from `django.contrib.auth.views` (line 61\u201365).\n\n5. `redirect_to_login()` (in `django/contrib/auth/views.py`, line 184) resolves the login URL, appends the `redirect_field_name` query parameter (defaulting to `\"next\"`, defined as `REDIRECT_FIELD_NAME = \"next\"` in `django/contrib/auth/__init__.py` line 17), and returns an `HttpResponseRedirect`.\n\n6. If the test passes, control flows to `super().dispatch(request, *args, **kwargs)` (line 136), which proceeds up the MRO to the actual view's `dispatch()`.\n\nThe base `test_func()` method (line 119) raises `NotImplementedError` \u2014 subclasses *must* override it. This contrasts with the function-based decorator equivalent `user_passes_test()` in `django/contrib/auth/decorators.py` (line 13), where the test function is passed as a parameter and takes the user object as an argument, while the mixin's `test_func` takes no arguments (it accesses `self.request.user` via the view instance).\n\nThe `AccessMixin` parent class defines four configurable attributes: `login_url = None`, `permission_denied_message = \"\"`, `raise_exception = False`, and `redirect_field_name = REDIRECT_FIELD_NAME` (which equals `\"next\"`).",
    "rubric": [
      "Explains that dispatch() calls get_test_func()() with double invocation \u2014 first to get the callable, then to invoke it",
      "Identifies that a falsy test result leads to handle_no_permission() inherited from AccessMixin",
      "Explains the branching in handle_no_permission(): raises PermissionDenied if raise_exception is True or user is authenticated, otherwise redirects",
      "Describes that the redirect path is built using redirect_to_login() which appends the redirect_field_name (defaulting to 'next') as a query parameter to the login URL",
      "Notes that test_func() raises NotImplementedError by default, requiring subclasses to override it",
      "Mentions AccessMixin's configurable attributes: login_url, permission_denied_message, raise_exception, and redirect_field_name"
    ],
    "key_files": [
      "django/contrib/auth/mixins.py",
      "django/contrib/auth/views.py",
      "django/contrib/auth/decorators.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/mixins.py] django.contrib.auth.mixins.UserPassesTestMixin\ndjango.contrib.auth.mixins.UserPassesTestMixin:\n  Deny a request with a permission error if the test_func() method returns\n    False.\n\ndjango.contrib.auth.mixins.UserPassesTestMixin.get_test_func:\n  Override this method to use a different test_func method.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code; no documentation-only leakage detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_36",
    "repo": "django",
    "question": "How does RemoteUserMiddleware handle authentication, including its sync/async dispatch, header access, and invalid user removal logic?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `RemoteUserMiddleware` class in `django/contrib/auth/middleware.py` implements authentication based on the `REMOTE_USER` header through the following mechanism:\n\n## Dual Sync/Async Support\nThe middleware is both `sync_capable = True` and `async_capable = True`. It does NOT inherit from `MiddlewareMixin` (unlike `AuthenticationMiddleware`). Instead, it implements its own `__init__`, `__call__`, and `__acall__` pattern directly. In `__init__`, it checks if `get_response` is a coroutine function using `iscoroutinefunction(get_response)` and stores the result in `self.is_async`. If async, it marks itself as a coroutine function via `markcoroutinefunction(self)`. The `__call__` method dispatches to `__acall__` if `self.is_async` is True.\n\n## Key Implementation Details\n\n### Header Access Inconsistency (Sync vs Async)\nThere is a notable difference between the sync and async paths:\n- **Sync** `process_request`: accesses `request.META[self.header]` directly (i.e., `request.META[\"REMOTE_USER\"]`)\n- **Async** `aprocess_request`: accesses `request.META[\"HTTP_\" + self.header]` (i.e., `request.META[\"HTTP_REMOTE_USER\"]`)\n\nThis means the async path prepends \"HTTP_\" to the header name, which would be appropriate for HTTP request headers but is inconsistent with the sync path.\n\n### Authentication Flow\n1. Checks that `request.user` exists (requires `AuthenticationMiddleware` to be installed first)\n2. Attempts to get username from `request.META[self.header]`\n3. If header is missing and `force_logout_if_no_header=True` and user is authenticated, calls `_remove_invalid_user()`\n4. If user is already authenticated with matching username (via `clean_username`), returns early\n5. If user is authenticated but with a different username, removes the invalid user\n6. Calls `auth.authenticate(request, remote_user=username)` which routes through `_get_compatible_backends()` using signature binding to find backends whose `authenticate` method accepts `remote_user` as a keyword argument\n7. On success, sets `request.user = user` and calls `auth.login(request, user)`\n\n### `_remove_invalid_user` Implementation\nThe method loads the backend stored in `request.session[auth.BACKEND_SESSION_KEY]` via `load_backend()` (which calls `import_string(path)()` \u2014 instantiating the class). It only calls `auth.logout(request)` if the backend is an instance of `RemoteUserBackend`. If `ImportError` occurs during backend loading, it also calls logout. Since it checks `isinstance(stored_backend, RemoteUserBackend)`, users authenticated via other backends (like `ModelBackend`) will not be logged out.\n\n### `clean_username` Method\nIn the middleware, `clean_username` loads the backend from the session via `request.session[auth.BACKEND_SESSION_KEY]` and delegates to `backend.clean_username(username)`, catching `AttributeError` if the backend doesn't define it. The `RemoteUserBackend.clean_username()` in `backends.py` simply returns the username unchanged by default.\n\n### `RemoteUserBackend.authenticate` Details\nThe backend's `authenticate(self, request, remote_user)` method:\n- Returns `None` if `remote_user` is falsy\n- Calls `self.clean_username(remote_user)` on the raw username\n- If `create_unknown_user=True` (default), uses `UserModel._default_manager.get_or_create(**{UserModel.USERNAME_FIELD: username})`\n- Otherwise uses `get_by_natural_key(username)`\n- Calls `self.configure_user(request, user, created=created)` on the user (default is no-op)\n- Returns user only if `self.user_can_authenticate(user)` is True (inherited from `ModelBackend`, checks `is_active`)\n\n### `PersistentRemoteUserMiddleware`\nSubclass that sets `force_logout_if_no_header = False`. Since the force_logout_if_no_header check guards the logout logic when the header is absent, setting it to False means users remain authenticated even when the header is not present in subsequent requests.\n\n### `auth.login()` session details\nWhen `auth.login()` is called after successful authentication, it stores `SESSION_KEY` (\"_auth_user_id\"), `BACKEND_SESSION_KEY` (\"_auth_user_backend\"), and `HASH_SESSION_KEY` (\"_auth_user_hash\") in the session, cycles the session key (or flushes if a different user was previously authenticated), and sends `user_logged_in` signal.",
    "rubric": [
      "Explains that RemoteUserMiddleware does NOT inherit from MiddlewareMixin; instead it implements its own __init__, __call__, and __acall__ pattern with sync_capable=True and async_capable=True",
      "Describes the __init__ method using iscoroutinefunction to detect async and markcoroutinefunction to mark itself, with __call__ dispatching to __acall__ when self.is_async is True",
      "Notes the inconsistency between sync (request.META[self.header]) and async (request.META['HTTP_' + self.header]) header access",
      "Describes the authentication flow: checks request.user exists, gets username from META, compares with current user via clean_username, calls auth.authenticate with remote_user kwarg, then sets request.user and calls auth.login on success",
      "Explains _remove_invalid_user loads the backend from session and only calls auth.logout if it's an instance of RemoteUserBackend (or if ImportError occurs during loading)",
      "Explains that PersistentRemoteUserMiddleware sets force_logout_if_no_header=False so users stay authenticated when the header is absent",
      "Describes RemoteUserBackend.authenticate: returns None if remote_user is falsy, uses get_or_create when create_unknown_user=True, calls configure_user, and checks user_can_authenticate (is_active)"
    ],
    "key_files": [
      "django/contrib/auth/middleware.py",
      "django/contrib/auth/backends.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/middleware.py] django.contrib.auth.middleware.RemoteUserMiddleware\ndjango.contrib.auth.middleware.RemoteUserMiddleware:\n  Middleware for utilizing web-server-provided authentication.\n\n    If request.user is not authenticated, then this middleware attempts to\n    authenticate the username from the ``REMOTE_USER`` key in ``request.META``,\n    an environment variable commonly set by the webserver.\n\n    If authentication is successful, the user is automatically logged in to\n    persist the user in the session.\n\n    The ``request.META`` key is configurable and defaults to ``REMOTE_USER``.\n    Subclass this class and change the ``header`` attribute if you need to\n    use a different key from ``request.META``, for example a HTTP request\n    header.\n\ndjango.contrib.auth.middleware.RemoteUserMiddleware.clean_username:\n  Allow the backend to clean the username, if the backend defines a\n        clean_username method.\n\ndjango.contrib.auth.middleware.RemoteUserMiddleware._remove_invalid_user:\n  Remove the current authenticated user in the request which is invalid\n        but only if the user is authenticated via the RemoteUserBackend.\n\ndjango.contrib.auth.middleware.RemoteUserMiddleware._aremove_invalid_user:\n  Remove the current authenticated user in the request which is invalid\n        but only if the user is authenticated via the RemoteUserBackend.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about 'Header Access Inconsistency (Sync vs Async)' where async path prepends 'HTTP_' to the header name is not mentioned in the documentation and could be a hallucination or an incorrect reading of source code. The documentation only states the key defaults to REMOTE_USER and is configurable via the header attribute.",
      "Many implementation details (dual sync/async support, MiddlewareMixin inheritance, markcoroutinefunction, force_logout_if_no_header, session key details, RemoteUserBackend.authenticate internals, PersistentRemoteUserMiddleware, auth.login session details) go well beyond the provided documentation. While they may be accurate based on source code, they cannot be verified from the given doc chunk alone.",
      "The documentation says the middleware attempts to authenticate from the REMOTE_USER key in request.META. The answer's claim about async using 'HTTP_REMOTE_USER' would contradict the spirit of the documentation if incorrect, but since this is a code-level detail not explicitly covered by the doc, it's uncertain."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Almost all claims are directly verifiable from the stripped source code; only minor wording about purpose/rationale needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_38",
    "repo": "django",
    "question": "Why is PermissionRequiredMixin designed the way it is \u2014 what are the key rationale choices behind its permission checking, error handling, and method separation?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `PermissionRequiredMixin` is designed with several key design rationales that go beyond the documentation:\n\n**1. String-to-tuple conversion for ergonomic single-permission usage:**\nIn `get_permission_required()` (line 94-95), if `permission_required` is a string, it's wrapped in a tuple: `perms = (self.permission_required,)`. This is a defensive design decision because downstream, `has_perms()` in `django/contrib/auth/models.py` (line 413) explicitly validates that `perm_list` must be a non-string iterable: `if not isinstance(perm_list, Iterable) or isinstance(perm_list, str): raise ValueError(\"perm_list must be an iterable of permissions.\")`. Without the string-to-tuple conversion in the mixin, passing a single string permission like `'auth.add_user'` would raise a ValueError. This design allows developers to set `permission_required = 'auth.add_user'` (a common case) without needing to use a list.\n\n**2. Authenticated vs anonymous user distinction in `handle_no_permission()`:**\nThe parent class `AccessMixin.handle_no_permission()` (line 48) uses the condition `if self.raise_exception or self.request.user.is_authenticated` to decide whether to raise `PermissionDenied` or redirect to login. The rationale is: if a user is already authenticated but lacks permissions, redirecting them to the login page would be pointless (they'd just be logged in again with the same lack of permissions), so it raises `PermissionDenied` (HTTP 403) instead. Anonymous users get redirected to login because logging in might grant them the needed permissions.\n\n**3. Separation of `has_permission()` and `get_permission_required()` methods:**\nThe `has_permission()` method (line 100-105) is separated from `get_permission_required()` so developers can override the permission checking logic entirely without changing how permissions are collected. For example, one could override `has_permission()` to add object-level permissions (the underlying `has_perms` supports an `obj` parameter), while `get_permission_required()` only handles which permissions to check.\n\n**4. Backend short-circuit design:**\nThe actual permission checking in `_user_has_perm` (models.py line 261-273) iterates all authentication backends and returns `True` if ANY backend grants the permission, but returns `False` immediately if a backend raises `PermissionDenied`. This allows any single backend to either grant access or explicitly deny it (short-circuit). Active superusers bypass all backend checks entirely (`has_perm` returns `True` immediately if `is_active and is_superuser`).\n\n**5. ImproperlyConfigured guard:**\nIf `permission_required` is `None` (not set), an `ImproperlyConfigured` exception is raised with a helpful message naming the class\u2014this is a fail-safe to prevent the mixin from silently passing without any permission checks if a developer forgets to set the attribute.",
    "rubric": [
      "Explains that get_permission_required() converts a string to a tuple because has_perms() explicitly rejects strings (raises ValueError for non-iterable or string perm_list), enabling the ergonomic single-permission pattern",
      "Explains that handle_no_permission() raises PermissionDenied (403) for authenticated users instead of redirecting to login, because re-authenticating wouldn't grant new permissions, while anonymous users are redirected to login",
      "Explains that has_permission() is separated from get_permission_required() so developers can override permission-checking logic independently from permission collection (e.g., adding object-level checks via the obj parameter on has_perms)",
      "Explains that _user_has_perm iterates backends and returns True if any backend grants permission, but returns False immediately if a backend raises PermissionDenied (short-circuit), and active superusers bypass backend checks entirely",
      "Explains that permission_required=None triggers an ImproperlyConfigured exception as a fail-safe to prevent silent pass-through if the developer forgets to set permissions"
    ],
    "key_files": [
      "django/contrib/auth/mixins.py",
      "django/contrib/auth/models.py",
      "tests/auth_tests/test_mixins.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/mixins.py] django.contrib.auth.mixins.PermissionRequiredMixin\ndjango.contrib.auth.mixins.PermissionRequiredMixin:\n  Verify that the current user has all specified permissions.\n\ndjango.contrib.auth.mixins.PermissionRequiredMixin.get_permission_required:\n  Override this method to override the permission_required attribute.\n        Must return an iterable.\n\ndjango.contrib.auth.mixins.PermissionRequiredMixin.has_permission:\n  Override this method to customize the way permissions are checked.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (line numbers, specific code logic, backend behavior) that go far beyond what the documentation states. While these are plausible based on Django's actual source code, they cannot be verified from the provided documentation alone.",
      "Claim about string-to-tuple conversion in get_permission_required() is plausible but not stated in the doc \u2014 the doc only says 'Must return an iterable'.",
      "Claim about handle_no_permission() behavior distinguishing authenticated vs anonymous users is not mentioned in the provided documentation at all.",
      "Claim about ImproperlyConfigured guard when permission_required is None is not mentioned in the documentation.",
      "Claim about backend short-circuit design and superuser bypass is entirely outside the scope of the provided documentation.",
      "The separation of has_permission() and get_permission_required() is consistent with the documentation showing them as separate methods, but the rationale given (object-level permissions, etc.) is extrapolated."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code through direct observation or logical deduction.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_41",
    "repo": "django",
    "question": "What is the full architecture of the Group model, including its fields, manager, relationships to users, permission resolution in backends, and admin integration?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `Group` model in `django/contrib/auth/models.py` is a Django `models.Model` subclass. Here are the key architectural and implementation details:\n\n**Model Fields:**\n- `name`: A `CharField` with `max_length=150` and `unique=True` (changed to 150 in migration `0010_alter_group_name_max_length`).\n- `permissions`: A `ManyToManyField` to the `Permission` model, with `blank=True`.\n\n**Manager:**\n- Uses `GroupManager`, a custom manager with `use_in_migrations = True`.\n- `GroupManager` provides `get_by_natural_key(name)` (synchronous) which does `self.get(name=name)`, and `aget_by_natural_key(name)` (async) which does `await self.aget(name=name)`.\n\n**Natural Key & Serialization:**\n- `Group.natural_key()` returns a single-element tuple: `(self.name,)`.\n- `__str__` returns `self.name`.\n\n**Meta Options:**\n- `verbose_name = _(\"group\")` and `verbose_name_plural = _(\"groups\")`.\n- No explicit `ordering` is defined on the model itself.\n\n**Relationship to Users (via PermissionsMixin):**\n- The `PermissionsMixin` abstract model (mixed into `AbstractUser`) defines a `groups` ManyToManyField pointing to `Group` with `related_name=\"user_set\"` and `related_query_name=\"user\"`. This means groups can access their users via `group.user_set.all()`.\n\n**How Group Permissions Are Resolved (ModelBackend):**\n- In `django/contrib/auth/backends.py`, `ModelBackend._get_group_permissions(user_obj)` resolves group permissions via: `Permission.objects.filter(group__in=user_obj.groups.all())`.\n- Permissions are cached on the user object as `_group_perm_cache` (formatted as `\"app_label.codename\"` strings).\n- If the user is inactive, anonymous, or an object-level permission check is made (`obj is not None`), an empty set is returned.\n- For superusers, all `Permission` objects are returned regardless of group membership.\n\n**AnonymousUser Integration:**\n- `AnonymousUser` uses `EmptyManager(Group)` for its `_groups` attribute, which is a manager subclass whose `get_queryset()` always returns `.none()`. The `groups` attribute is exposed as a property returning this empty manager.\n- `AnonymousUser.get_group_permissions()` returns `set()` directly (hardcoded empty), unlike authenticated users.\n\n**Admin Integration:**\n- `GroupAdmin` in `django/contrib/auth/admin.py` uses `search_fields = (\"name\",)`, `ordering = (\"name\",)`, and `filter_horizontal = (\"permissions\",)`. It overrides `formfield_for_manytomany` to add `select_related(\"content_type\")` on the permissions queryset.\n\n**with_perm Query:**\n- `ModelBackend.with_perm()` constructs a query using `Q(group__user=OuterRef(\"pk\")) | Q(user=OuterRef(\"pk\"))` to find users with a given permission through either group membership or direct assignment.",
    "rubric": [
      "Identifies Group's fields: name (CharField, max_length=150, unique=True) and permissions (ManyToManyField to Permission, blank=True)",
      "Describes GroupManager with use_in_migrations=True, providing get_by_natural_key(name) and async aget_by_natural_key(name)",
      "Mentions natural_key() returns (self.name,) and __str__ returns self.name",
      "Describes Meta options: verbose_name='group', verbose_name_plural='groups', no explicit ordering",
      "Explains the PermissionsMixin relationship: ManyToManyField to Group with related_name='user_set' and related_query_name='user'",
      "Explains how ModelBackend._get_group_permissions resolves permissions via Permission.objects.filter(group__in=user_obj.groups.all())",
      "Describes permission caching as _group_perm_cache with 'app_label.codename' formatted strings, and that inactive/anonymous users or object-level checks return empty set",
      "Mentions superusers get all Permission objects regardless of group membership",
      "Describes AnonymousUser using EmptyManager(Group) for groups, with get_group_permissions() returning set() directly",
      "Describes GroupAdmin configuration: search_fields=('name',), ordering=('name',), filter_horizontal=('permissions',), and select_related('content_type') optimization",
      "Mentions with_perm() uses Q(group__user=OuterRef('pk')) | Q(user=OuterRef('pk')) to find users with a permission via group or direct assignment"
    ],
    "key_files": [
      "django/contrib/auth/models.py",
      "django/contrib/auth/backends.py",
      "django/contrib/auth/admin.py",
      "django/db/models/manager.py",
      "django/contrib/auth/migrations/0010_alter_group_name_max_length.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/models.py] django.contrib.auth.models.Group\ndjango.contrib.auth.models.Group:\n  Groups are a generic way of categorizing users to apply permissions, or\n    some other label, to those users. A user can belong to any number of\n    groups.\n\n    A user in a group automatically has all the permissions granted to that\n    group. For example, if the group 'Site editors' has the permission\n    can_edit_home_page, any user in that group will have that permission.\n\n    Beyond permissions, groups are a convenient way to categorize users to\n    apply some label, or extended functionality, to them. For example, you\n    could create a group 'Special users', and you could write code that would\n    do special things to those users -- such as giving them access to a\n    members-only portion of your site, or sending them members-only email\n    messages.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in purpose descriptions and rationale phrases; almost all claims are directly verifiable from stripped source code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_40",
    "repo": "django",
    "question": "How does Django's permission system work end-to-end, from automatic permission creation to permission checking and caching?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The Django permission system is implemented through a multi-layered architecture involving the `Permission` model, the `ModelBackend`, signal-based automatic permission creation, and a mixin-based user permissions interface. Here's how it works in detail:\n\n**Permission Model Structure:**\nThe `Permission` model (in `django/contrib/auth/models.py`) has three fields: `name` (CharField, max_length=255), `content_type` (ForeignKey to ContentType with CASCADE deletion), and `codename` (CharField, max_length=100). It has a unique_together constraint on `[\"content_type\", \"codename\"]` and is ordered by `[\"content_type__app_label\", \"content_type__model\", \"codename\"]`.\n\n**Automatic Permission Creation:**\nPermissions are automatically created via the `post_migrate` signal. In `django/contrib/auth/apps.py`, the `AuthConfig.ready()` method connects `create_permissions` to the `post_migrate` signal. The `create_permissions` function (in `django/contrib/auth/management/__init__.py`) iterates over all models in an app, gets their ContentTypes, and creates Permission objects for any missing permissions using `bulk_create`.\n\nThe default permissions are defined in `django/db/models/options.py` as the tuple `(\"add\", \"change\", \"delete\", \"view\")` stored in `Options.default_permissions`. The codename format is `\"{action}_{model_name}\"` (from `get_permission_codename` in `django/contrib/auth/__init__.py`), and the human-readable name format is `\"Can {action} {verbose_name_raw}\"`.\n\n**Permission Checking via Backends:**\nThe `PermissionsMixin` (an abstract model) provides `has_perm`, `has_perms`, `has_module_perms`, etc. Active superusers bypass all backend checks and always return True. Otherwise, the helper function `_user_has_perm` iterates through all configured authentication backends (from `settings.AUTHENTICATION_BACKENDS`). A backend can raise `PermissionDenied` to short-circuit and deny permission immediately (returning False).\n\n**Permission Caching in ModelBackend:**\nIn `django/contrib/auth/backends.py`, `ModelBackend._get_permissions` caches permissions on the user object using attributes like `_user_perm_cache` and `_group_perm_cache`. Permissions are stored as strings in the format `\"{app_label}.{codename}\"` (e.g., `\"auth.add_user\"`). For superusers, ALL permissions in the database are fetched. For group permissions, it queries `Permission.objects.filter(group__in=user_obj.groups.all())`. If the user is inactive, anonymous, or an object is provided, an empty set is returned.\n\n**Natural Key System:**\nThe `PermissionManager.get_by_natural_key` takes `(codename, app_label, model)` and resolves the content type via `ContentType.objects.db_manager(self.db).get_by_natural_key(app_label, model)`. The `Permission.natural_key()` method returns `(self.codename, *self.content_type.natural_key())` with a dependency declared on `\"contenttypes.contenttype\"`.\n\n**Input Validation:**\n`has_perms` raises `ValueError` if `perm_list` is a string or not an Iterable.",
    "rubric": [
      "The Permission model has fields name (CharField max_length=255), content_type (ForeignKey to ContentType), and codename (CharField max_length=100) with unique_together on content_type and codename",
      "Permissions are auto-created via post_migrate signal connected in AuthConfig.ready(), calling create_permissions which uses bulk_create for missing permissions",
      "Default permissions are ('add', 'change', 'delete', 'view') from Options.default_permissions, with codename format '{action}_{model_name}' and name format 'Can {action} {verbose_name_raw}'",
      "PermissionsMixin.has_perm short-circuits to True for active superusers, otherwise delegates to _user_has_perm which iterates through all authentication backends",
      "A backend can raise PermissionDenied to short-circuit and immediately return False from _user_has_perm",
      "ModelBackend._get_permissions caches permissions on the user object using attributes like '_{from_name}_perm_cache' (e.g., _user_perm_cache, _group_perm_cache)",
      "Permissions are stored as strings in '{app_label}.{codename}' format, retrieved via values_list('content_type__app_label', 'codename')",
      "For superusers, ModelBackend fetches ALL permissions from the database; for inactive/anonymous users or when obj is provided, it returns an empty set",
      "Group permissions are fetched via Permission.objects.filter(group__in=user_obj.groups.all())",
      "has_perms raises ValueError if perm_list is a string or not an Iterable"
    ],
    "key_files": [
      "django/contrib/auth/models.py",
      "django/contrib/auth/management/__init__.py",
      "django/contrib/auth/backends.py",
      "django/contrib/auth/apps.py",
      "django/db/models/options.py",
      "django/contrib/auth/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/models.py] django.contrib.auth.models.Permission\ndjango.contrib.auth.models.Permission:\n  The permissions system provides a way to assign permissions to specific\n    users and groups of users.\n\n    The permission system is used by the Django admin site, but may also be\n    useful in your own code. The Django admin site uses permissions as follows:\n\n        - The \"add\" permission limits the user's ability to view the \"add\" form\n          and add an object.\n        - The \"change\" permission limits a user's ability to view the change\n          list, view the \"change\" form and change an object.\n        - The \"delete\" permission limits the ability to delete an object.\n        - The \"view\" permission limits the ability to view an object.\n\n    Permissions are set globally per type of object, not per specific object\n    instance. It is possible to say \"Mary may change news stories,\" but it's\n    not currently possible to say \"Mary may change news stories, but only the\n    ones she created herself\" or \"Mary may only change news stories that have a\n    certain status or publication date.\"\n\n    The permissions listed above are automatically created for each model.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.025,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only one minor rationale phrase about 'common mistake' needed trimming.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_43",
    "repo": "django",
    "question": "Where does the common password validation get triggered throughout the codebase, and how is its validator cache managed?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `CommonPasswordValidator` class is located in `django/contrib/auth/password_validation.py` (lines 224-261). It is invoked through the `validate_password()` function (line 41) which iterates over validators obtained from `get_default_password_validators()` (cached via `@functools.cache` at line 20). The validator flows into the system through multiple entry points:\n\n1. **Forms**: `SetPasswordMixin.validate_password_for_user()` in `django/contrib/auth/forms.py` (line 123) calls `password_validation.validate_password(password, user)`. This is used by `BaseUserCreationForm._post_clean()` (line 240), `SetPasswordForm.clean()` (line 505), and `PasswordChangeForm`.\n\n2. **Management commands**: `changepassword.py` (line 67) directly calls `validate_password()`. `createsuperuser.py` also calls `validate_password()`.\n\n3. **Cache clearing**: In `django/test/signals.py` (line 216), a signal handler `auth_password_validators_changed` listens for the `setting_changed` signal and calls `get_default_password_validators.cache_clear()` when `AUTH_PASSWORD_VALIDATORS` is changed.\n\nThe default password list file is `common-passwords.txt.gz` located in `django/contrib/auth/`. The `DEFAULT_PASSWORD_LIST_PATH` is implemented as a `@cached_property`, and in `__init__` there's a sentinel check: `if password_list_path is CommonPasswordValidator.DEFAULT_PASSWORD_LIST_PATH` \u2014 this compares identity against the `cached_property` descriptor object itself (not its resolved value), allowing detection of whether the default argument was used. The password file is loaded into a set (`self.passwords`) using `gzip.open()` first, falling back to a plain `open()` on `OSError` (lines 243-248). The `validate()` method (line 251) lowercases and strips the password before checking set membership. The `AUTH_PASSWORD_VALIDATORS` setting is referenced by `get_default_password_validators()` via `settings.AUTH_PASSWORD_VALIDATORS`.",
    "rubric": [
      "Identifies CommonPasswordValidator in django/contrib/auth/password_validation.py",
      "Explains that validate_password() iterates over validators from get_default_password_validators(), which is cached via @functools.cache",
      "Mentions forms entry point: SetPasswordMixin.validate_password_for_user() calls password_validation.validate_password(), used by BaseUserCreationForm, SetPasswordForm, and PasswordChangeForm",
      "Mentions management command entry points: changepassword and createsuperuser both call validate_password()",
      "Explains that in django/test/signals.py, a setting_changed signal handler clears get_default_password_validators cache when AUTH_PASSWORD_VALIDATORS changes",
      "Notes the password list is loaded from a gzipped file (common-passwords.txt.gz), with gzip.open tried first and plain open as fallback on OSError",
      "Describes the DEFAULT_PASSWORD_LIST_PATH sentinel pattern using @cached_property and identity comparison against the descriptor object"
    ],
    "key_files": [
      "django/contrib/auth/password_validation.py",
      "django/contrib/auth/forms.py",
      "django/test/signals.py",
      "django/contrib/auth/management/commands/changepassword.py",
      "tests/auth_tests/test_validators.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/password_validation.py] django.contrib.auth.password_validation.CommonPasswordValidator\ndjango.contrib.auth.password_validation.CommonPasswordValidator:\n  Validate that the password is not a common password.\n\n    The password is rejected if it occurs in a provided list of passwords,\n    which may be gzipped. The list Django ships with contains 20000 common\n    passwords (unhexed, lowercased and deduplicated), created by Royce\n    Williams:\n    https://gist.github.com/roycewilliams/226886fd01572964e1431ac8afc999ce\n    The password list must be lowercased to match the comparison in validate().",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.06,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the 'enabling test isolation' rationale and specific global_settings.py line reference are not directly verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_42",
    "repo": "django",
    "question": "Why does the `UserAttributeSimilarityValidator` use `quick_ratio()` instead of `ratio()`, and what other performance optimizations does it employ?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `UserAttributeSimilarityValidator` uses `quick_ratio()` from Python's `difflib.SequenceMatcher` rather than `ratio()`. The design includes two mechanisms to avoid unnecessary computation:\n\n1. **`exceeds_maximum_length_ratio()` function**: This is a pre-check that avoids calling `SequenceMatcher` altogether when the password length far exceeds the attribute value length. The function returns `True` (skip comparison) when `pwd_len >= 10 * value_len` AND `value_len < length_bound_similarity` (where `length_bound_similarity = max_similarity / 2 * pwd_len`).\n\n2. **Using `quick_ratio()` instead of `ratio()`**: The code uses `SequenceMatcher(a=password, b=value_part).quick_ratio()` for the similarity comparison.\n\nThe validator also enforces a minimum `max_similarity` of 0.1 (raising `ValueError` otherwise).\n\nThe attribute splitting uses `re.split(r\"\\W+\", value_lower)` to split on non-word characters, which decomposes emails like 'testclient@example.com' into parts ['testclient', 'example', 'com']. The full value is appended to this list, so comparison happens against both individual components and the complete value.\n\nWhen a match is found, the validator tries to get a human-readable field name via `user._meta.get_field(attribute_name).verbose_name`. If the attribute is a property (not a model field), it catches `FieldDoesNotExist` and falls back to using the raw attribute name. For Django's built-in `AbstractUser`, the verbose names are translatable strings: 'username', 'first name', 'last name', and 'email address'.\n\nThe `get_default_password_validators()` function uses `@functools.cache` for caching, and a signal handler in `django/test/signals.py` calls `get_default_password_validators.cache_clear()` whenever `AUTH_PASSWORD_VALIDATORS` setting changes during tests.",
    "rubric": [
      "Explains that `quick_ratio()` is used from `difflib.SequenceMatcher` instead of `ratio()` as a cheaper similarity computation",
      "Describes the `exceeds_maximum_length_ratio()` pre-check function that skips SequenceMatcher entirely when the password is much longer than the attribute value (pwd_len >= 10 * value_len and value_len < length_bound_similarity)",
      "Mentions that `length_bound_similarity` is calculated as `max_similarity / 2 * pwd_len`",
      "Notes the minimum `max_similarity` of 0.1 enforced via ValueError",
      "Explains that attribute values are split on non-word characters (`re.split(r'\\W+', ...)`) and the full value is also appended, so both parts and whole value are compared",
      "Mentions verbose_name lookup via `user._meta.get_field()` with a `FieldDoesNotExist` fallback to the raw attribute name",
      "Mentions `get_default_password_validators()` is cached with `@functools.cache` and cleared via a test signal handler"
    ],
    "key_files": [
      "django/contrib/auth/password_validation.py",
      "django/contrib/auth/models.py",
      "django/test/signals.py",
      "tests/auth_tests/test_validators.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/password_validation.py] django.contrib.auth.password_validation.UserAttributeSimilarityValidator\ndjango.contrib.auth.password_validation.UserAttributeSimilarityValidator:\n  Validate that the password is sufficiently different from the user's\n    attributes.\n\n    If no specific attributes are provided, look at a sensible list of\n    defaults. Attributes that don't exist are ignored. Comparison is made to\n    not only the full attribute value, but also its components, so that, for\n    example, a password is validated against either part of an email address,\n    as well as the full address.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (quick_ratio(), exceeds_maximum_length_ratio(), line numbers, regex patterns, caching mechanisms, signal handlers) that are not mentioned in the provided documentation and cannot be verified from it alone.",
      "While the documentation mentions that comparison is made to components of attributes (e.g., parts of an email address), the specific regex pattern `re.split(r\"\\W+\", value_lower)` and the exact splitting behavior described cannot be verified from the docstring.",
      "The claims about `max_similarity` parameter, `ValueError` for values below 0.1, `FieldDoesNotExist` handling, `@functools.cache`, and signal handlers are all implementation details not present in the documentation.",
      "The claim about email splitting into ['testclient', 'example', 'com'] is consistent with the documentation's statement about validating against parts of an email address, but the specific mechanism is unverifiable from the doc."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "A few claims about design rationale and mathematical justification for the optimization were removed as they rely on documentation/comments not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_44",
    "repo": "django",
    "question": "How does Django's PasswordResetTokenGenerator generate and validate password reset tokens, including the hashing, timestamp encoding, and secret fallback logic?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `PasswordResetTokenGenerator` generates and validates password reset tokens using the following algorithm:\n\n**Token Generation (`make_token` \u2192 `_make_token_with_timestamp`):**\n1. Computes the number of seconds since January 1, 2001 (epoch reference) via `_num_seconds()`, which calculates `int((dt - datetime(2001, 1, 1)).total_seconds())`.\n2. Converts this timestamp to a base-36 string using `int_to_base36()`.\n3. Constructs the hash value string as an f-string: `f\"{user.pk}{user.password}{login_timestamp}{timestamp}{email}\"` where:\n   - `login_timestamp` is the user's `last_login` with microseconds truncated and tzinfo stripped (or empty string if None)\n   - `email` is obtained via `user.get_email_field_name()` which returns `cls.EMAIL_FIELD` or falls back to `\"email\"`\n4. Passes this value through `salted_hmac()` with the key_salt `\"django.contrib.auth.tokens.PasswordResetTokenGenerator\"`, the secret key, and algorithm `\"sha256\"` (default).\n5. In `salted_hmac()`, a derived key is created by hashing `key_salt + secret` with the specified hash algorithm, then this derived key is used as the HMAC key with `hmac.new(key, msg=force_bytes(value), digestmod=hasher)`.\n6. Takes the hexdigest and slices it with `[::2]` (every other character).\n7. The final token format is `\"{ts_b36}-{hash_string}\"`.\n\n**Token Validation (`check_token`):**\n1. Returns `False` immediately if user or token is falsy.\n2. Splits the token on `\"-\"` to extract the base-36 timestamp.\n3. Converts the base-36 timestamp back to an integer (with a guard: `base36_to_int` rejects strings longer than 13 characters).\n4. Iterates over `[self.secret, *self.secret_fallbacks]` \u2014 the primary secret and all fallback secrets (from `settings.SECRET_KEY_FALLBACKS`) \u2014 and uses `constant_time_compare` (which delegates to `secrets.compare_digest`) to compare the regenerated token against the provided token.\n5. If no secret produces a match, returns `False`.\n6. Checks if the timestamp has expired by comparing against `settings.PASSWORD_RESET_TIMEOUT`.\n\n**Key implementation details:**\n- The default algorithm is `\"sha256\"` (set in `__init__` if `self.algorithm` is None), while `salted_hmac`'s own default is `\"sha1\"` \u2014 but the token generator explicitly passes `\"sha256\"`.\n- `constant_time_compare` wraps `secrets.compare_digest` after converting both values to bytes via `force_bytes()`.\n- A module-level `default_token_generator = PasswordResetTokenGenerator()` instance is created at import time.\n- The `secret` and `secret_fallbacks` are implemented as Python properties with getter/setter, falling back to `settings.SECRET_KEY` and `settings.SECRET_KEY_FALLBACKS` respectively when not explicitly set.",
    "rubric": [
      "Explains that timestamps are computed as seconds since January 1, 2001 and encoded in base-36",
      "Describes the hash value string composition: user pk, password, last_login (with microseconds truncated and tzinfo stripped), timestamp, and email",
      "Explains that salted_hmac derives a key by hashing key_salt + secret, then uses that as the HMAC key with the specified algorithm (sha256)",
      "Mentions the hexdigest is sliced with [::2] (every other character) to shorten the token",
      "Describes the final token format as '{ts_b36}-{hash_string}'",
      "Explains check_token splits the token, converts base-36 back to int, and iterates over the primary secret plus secret_fallbacks using constant_time_compare",
      "Notes that constant_time_compare wraps secrets.compare_digest with force_bytes conversion",
      "Mentions that secret and secret_fallbacks are properties that fall back to settings.SECRET_KEY and settings.SECRET_KEY_FALLBACKS",
      "Notes the default algorithm is sha256 (set in __init__), distinct from salted_hmac's own default of sha1",
      "Mentions the PASSWORD_RESET_TIMEOUT expiry check occurs after token hash validation"
    ],
    "key_files": [
      "django/contrib/auth/tokens.py",
      "django/utils/crypto.py",
      "django/utils/http.py",
      "django/contrib/auth/base_user.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/tokens.py] django.contrib.auth.tokens.PasswordResetTokenGenerator\ndjango.contrib.auth.tokens.PasswordResetTokenGenerator:\n  Strategy object used to generate and check tokens for the password\n    reset mechanism.\n\ndjango.contrib.auth.tokens.PasswordResetTokenGenerator.make_token:\n  Return a token that can be used once to do a password reset\n        for the given user.\n\ndjango.contrib.auth.tokens.PasswordResetTokenGenerator.check_token:\n  Check that a password reset token is correct for a given user.\n\ndjango.contrib.auth.tokens.PasswordResetTokenGenerator._make_hash_value:\n  Hash the user's primary key, email (if available), and some user state\n        that's sure to change after a password reset to produce a token that is\n        invalidated when it's used:\n        1. The password field will change upon a password reset (even if the\n           same password is chosen, due to password salting).\n        2. The last_login field will usually be updated very shortly after\n           a password reset.\n        Failing those things, settings.PASSWORD_RESET_TIMEOUT eventually\n        invalidates the token.\n\n        Running this data through salted_hmac() prevents password cracking\n        attempts using the reset token, provided the secret isn't compromised.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (base-36 encoding, epoch reference of 2001-01-01, hash slicing with [::2], constant_time_compare, secret_fallbacks, etc.) that are not mentioned in the provided documentation. These appear to be sourced from actual code knowledge rather than the documentation chunk provided.",
      "The documentation does not mention the specific algorithm 'sha256', the salted_hmac implementation details, or the token format '{ts_b36}-{hash_string}' - these are extrapolations from code knowledge.",
      "The claim about PASSWORD_RESET_TIMEOUT default being 259200 seconds is not in the documentation, though the documentation does mention PASSWORD_RESET_TIMEOUT as a fallback invalidation mechanism.",
      "The documentation states the hash value includes 'primary key, email (if available), and some user state' - the answer's f-string format `f\"{user.pk}{user.password}{login_timestamp}{timestamp}{email}\"` is consistent with this but adds specifics not in the doc."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc-leakage: rationale for [::2] slicing, base36_to_int length guard purpose, default timeout value, and base-36 digit count estimate came from documentation rather than code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_45",
    "repo": "django",
    "question": "What does RedirectURLMixin provide, which views use it, and how does it differ from AccessMixin in handling redirects?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`RedirectURLMixin` is a plain Python mixin class (not inheriting from any Django base class) defined in `django/contrib/auth/views.py` that provides redirect URL resolution logic for authentication views. It has three class attributes: `next_page = None`, `redirect_field_name = REDIRECT_FIELD_NAME` (which equals the string `\"next\"`, defined in `django/contrib/auth/__init__.py`), and `success_url_allowed_hosts = set()`.\n\n**Key methods:**\n\n1. `get_success_url()` \u2014 Returns the result of `get_redirect_url()` if truthy, otherwise falls back to `get_default_redirect_url()`. This is the main entry point for determining where to redirect.\n\n2. `get_redirect_url(request=None)` \u2014 Extracts a user-supplied redirect URL first from `request.POST`, then from `request.GET`, using `self.redirect_field_name` as the parameter key. It validates the URL using `url_has_allowed_host_and_scheme()` from `django/utils/http.py`, which performs two checks: one on the original URL and one with backslashes replaced by forward slashes. The `require_https` parameter is set based on `request.is_secure()`. Returns an empty string `\"\"` if the URL is unsafe.\n\n3. `get_success_url_allowed_hosts(request=None)` \u2014 Returns a set containing the current request's host (`request.get_host()`) merged with `self.success_url_allowed_hosts`.\n\n4. `get_default_redirect_url()` \u2014 Uses `resolve_url(self.next_page)` if `next_page` is set; otherwise raises `ImproperlyConfigured(\"No URL to redirect to. Provide a next_page.\")`.\n\n**Subclasses that override `get_default_redirect_url()`:**\n\n- `LoginView(RedirectURLMixin, FormView)` \u2014 Falls back to `resolve_url(settings.LOGIN_REDIRECT_URL)` instead of raising an error. It is decorated with `login_not_required`, `sensitive_post_parameters()`, `csrf_protect`, and `never_cache`. Its `dispatch()` method includes a redirection loop detection that raises `ValueError` if `redirect_to == self.request.path` for authenticated users.\n\n- `LogoutView(RedirectURLMixin, TemplateView)` \u2014 Falls back to `resolve_url(settings.LOGOUT_REDIRECT_URL)` if that setting is truthy, or `self.request.path` otherwise. It only accepts POST requests (`http_method_names = [\"post\", \"options\"]`).\n\n**`resolve_url()` in `django/shortcuts.py`** tries three strategies: (1) calls `get_absolute_url()` if the argument is a model instance, (2) expands lazy strings, (3) handles relative URLs starting with `./` or `../`, (4) attempts `reverse()`, and (5) falls back to returning the string as-is.\n\n**`url_has_allowed_host_and_scheme()`** returns `False` on empty URLs, URLs starting with `///`, URLs exceeding `MAX_URL_LENGTH` (2048), URLs without a netloc but with a scheme, and URLs starting with Unicode control characters. It considers only `https` as valid when `require_https=True`.\n\n**Key distinction from `AccessMixin`:** `RedirectURLMixin` in `views.py` is separate from `AccessMixin` in `django/contrib/auth/mixins.py`. `AccessMixin` also uses `REDIRECT_FIELD_NAME` and `redirect_to_login()`, but handles the *incoming* side (redirecting unauthenticated users to the login page), while `RedirectURLMixin` handles the *outgoing* side (where to go after login/logout succeeds).",
    "rubric": [
      "RedirectURLMixin is a plain mixin class (no Django base class) with attributes next_page, redirect_field_name (defaulting to 'next'), and success_url_allowed_hosts (empty set)",
      "get_success_url() returns get_redirect_url() if truthy, otherwise falls back to get_default_redirect_url()",
      "get_redirect_url() extracts the redirect URL from POST then GET using redirect_field_name, validates it with url_has_allowed_host_and_scheme(), and returns empty string if unsafe",
      "get_success_url_allowed_hosts() merges request.get_host() with self.success_url_allowed_hosts",
      "get_default_redirect_url() uses resolve_url(self.next_page) if set, otherwise raises ImproperlyConfigured",
      "LoginView subclasses RedirectURLMixin and FormView, overrides get_default_redirect_url to fall back to settings.LOGIN_REDIRECT_URL, and includes redirect loop detection in dispatch()",
      "LogoutView subclasses RedirectURLMixin and TemplateView, overrides get_default_redirect_url to fall back to settings.LOGOUT_REDIRECT_URL or request.path, and only accepts POST requests",
      "AccessMixin handles the incoming side (redirecting unauthenticated users to login), while RedirectURLMixin handles the outgoing side (where to redirect after login/logout succeeds)",
      "url_has_allowed_host_and_scheme checks both the original URL and a backslash-replaced version, rejects empty URLs, URLs starting with ///, and URLs exceeding MAX_URL_LENGTH",
      "resolve_url tries get_absolute_url() for model instances, handles lazy strings and relative URLs, attempts reverse(), and falls back to returning the string as-is"
    ],
    "key_files": [
      "django/contrib/auth/views.py",
      "django/utils/http.py",
      "django/contrib/auth/__init__.py",
      "django/shortcuts.py",
      "django/contrib/auth/mixins.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/views.py] django.contrib.auth.views.RedirectURLMixin\ndjango.contrib.auth.views.RedirectURLMixin.get_redirect_url:\n  Return the user-originating redirect URL if it's safe.\n\n        Optionally takes a request argument, allowing use outside class-based\n        views.\n\ndjango.contrib.auth.views.RedirectURLMixin.get_default_redirect_url:\n  Return the default redirect URL.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the Chrome backslash rationale was documentation-only and removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_46",
    "repo": "django",
    "question": "Why does Django's LoginView use so many layered security and caching measures in its authentication flow?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `LoginView` in `django/contrib/auth/views.py` employs several deliberate performance and security design decisions:\n\n**1. `never_cache` decorator on `LoginView.dispatch`**\nThe `LoginView.dispatch` method is decorated with `never_cache`, which calls `add_never_cache_headers()` (in `django/utils/cache.py`). This function sets `cache_timeout=-1` via `patch_response_headers`, and additionally sets `no_cache=True, no_store=True, must_revalidate=True, private=True` via `patch_cache_control`.\n\n**2. URL length check in `_url_has_allowed_host_and_scheme` with early rejection**\nThe redirect URL validation in `_url_has_allowed_host_and_scheme()` (in `django/utils/http.py`) rejects any URL longer than `MAX_URL_LENGTH` (2048 characters) or starting with `///` *before* calling `urlsplit()`.\n\n**3. Double URL validation for backslash replacement**\n`url_has_allowed_host_and_scheme()` calls `_url_has_allowed_host_and_scheme` twice \u2014 once with the original URL and once with backslashes replaced by forward slashes.\n\n**4. `redirect_authenticated_user` is `False` by default**\nThe `redirect_authenticated_user = False` default means `dispatch()` won't call `self.request.user.is_authenticated` on every request. Setting it to `True` enables an early redirect for already-authenticated users, but includes a loop detection check that raises `ValueError` if the success URL equals the current path.\n\n**5. Session cycling in `auth_login`**\nThe `login()` function in `django/contrib/auth/__init__.py` uses `constant_time_compare` for session hash comparison, and either calls `request.session.flush()` (if a different user's session exists) or `request.session.cycle_key()` (for anonymous sessions). It also calls `rotate_token(request)` from `django/middleware/csrf.py` to change the CSRF token on login.\n\n**6. `sensitive_post_parameters()` called with no arguments marks ALL POST parameters as sensitive**\nSince `sensitive_post_parameters()` is called with no arguments in the decorator list, it sets `request.sensitive_post_parameters = \"__ALL__\"` (in `django/views/decorators/debug.py`).",
    "rubric": [
      "Explains that `never_cache` decorator on dispatch sets cache headers (no-cache, no-store, must-revalidate, private) to prevent browsers/proxies from caching login pages",
      "Mentions that `_url_has_allowed_host_and_scheme` rejects URLs longer than 2048 characters or starting with `///` before parsing, as an early rejection optimization",
      "Explains the double URL validation where backslashes are replaced with forward slashes to prevent bypass attacks",
      "Notes that `redirect_authenticated_user` defaults to False to avoid checking `is_authenticated` on every request, with loop detection when enabled",
      "Describes session cycling via `flush()` or `cycle_key()` and CSRF token rotation via `rotate_token()` to prevent session fixation attacks",
      "Explains that `sensitive_post_parameters()` with no arguments marks ALL POST parameters as sensitive (sets `__ALL__`) to prevent credential leakage in error reports"
    ],
    "key_files": [
      "django/contrib/auth/views.py",
      "django/utils/http.py",
      "django/contrib/auth/__init__.py",
      "django/utils/cache.py",
      "django/views/decorators/debug.py",
      "django/views/decorators/cache.py"
    ],
    "source_doc": "[docstring: django/contrib/auth/views.py] django.contrib.auth.views.LoginView\ndjango.contrib.auth.views.LoginView:\n  Display the login form and handle the login action.\n\ndjango.contrib.auth.views.LoginView.get_default_redirect_url:\n  Return the default redirect URL.\n\ndjango.contrib.auth.views.LoginView.form_valid:\n  Security check complete. Log the user in.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides high-level docstrings for LoginView, get_default_redirect_url, and form_valid. The generated answer contains extensive implementation details (line numbers, specific function calls, decorator behavior, etc.) that go far beyond what the documentation states.",
      "None of the specific claims about never_cache, _url_has_allowed_host_and_scheme, double URL validation, redirect_authenticated_user, session cycling, or sensitive_post_parameters are directly supported by the provided documentation.",
      "However, these claims are plausible extrapolations based on actual Django source code and do not contradict anything in the provided documentation.",
      "The answer does not contradict the documented behavior: LoginView displays the login form and handles login action, form_valid logs the user in after security check, and get_default_redirect_url returns the default redirect URL."
    ],
    "strip_verify_leakage": 0.33,
    "strip_verify_summary": "Several claims about design rationale (DoS prevention, timing attacks, session fixation, caching security purpose) and one comment quote rely on documentation/comments not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_47",
    "repo": "django",
    "question": "Where is `GenericForeignKey` defined, how does it integrate into the model system, and how are its system checks triggered?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `GenericForeignKey` class is defined in `django/contrib/contenttypes/fields.py` at line 28 and inherits from both `FieldCacheMixin` (from `django/db/models/fields/mixins.py`) and `Field` (from `django/db/models/fields/__init__.py`). The `GenericForeignKeyDescriptor` companion class is defined in the same file at line 176 and handles `__get__`, `__set__`, `fetch_one`, `fetch_many`, and `get_prefetch_querysets`.\n\nKey location details:\n\n1. **`GenericForeignKey` is added to models as a private field**: In `contribute_to_class()` (line 51), it calls `super().contribute_to_class(cls, name, private_only=True)`, which causes it to be stored in `model._meta.private_fields` rather than `local_fields` or `local_many_to_many`.\n\n2. **System checks are triggered from `django/contrib/contenttypes/checks.py`**: The `check_generic_foreign_keys` function iterates all model classes, looks for `GenericForeignKeyDescriptor` instances in `vars(model)`, and calls `descriptor.field.check()` on each. This check function is registered in `ContentTypesConfig.ready()` (`django/contrib/contenttypes/apps.py` line 21) under the `checks.Tags.models` tag.\n\n3. **`GenericForeignKey.check()` (line 77) overrides `Field.check()` completely** \u2014 it does NOT call `super().check()`. It runs only 3 checks: `_check_field_name()` (inherited from `Field`), `_check_object_id_field()`, and `_check_content_type_field()`. This contrasts with `Field.check()` which runs ~9 checks including `_check_choices`, `_check_db_default`, etc.\n\n4. **`_check_content_type_field()` produces 4 possible error IDs**: `contenttypes.E002` (field doesn't exist), `contenttypes.E003` (field is not a ForeignKey), `contenttypes.E004` (ForeignKey target is not ContentType), and `contenttypes.E006` (uses `DatabaseOnDelete` variant). The E006 check verifies that the content type FK does not use a `DatabaseOnDelete` instance (imported from `django/db/models/deletion.py`).\n\n5. **`GenericForeignKey.get_attname_column()`** (line 55) returns `(attname, None)` \u2014 explicitly setting the column to `None`.\n\n6. **The descriptor's `__get__` method** (line 245) uses `self.field.get_content_type(id=ct_id)` which calls `ContentType.objects.get_for_id(id)` rather than following the FK relation.\n\n7. **`fetch_one`** (line 274) uses `ContentType.get_object_for_this_type()` (from `django/contrib/contenttypes/models.py` line 172), which calls `model_class()._base_manager.using(using).get(**kwargs)`. **`fetch_many`** (line 291) delegates to `prefetch_related_objects` from `django/db/models/query.py`.",
    "rubric": [
      "GenericForeignKey is defined in django/contrib/contenttypes/fields.py and inherits from both FieldCacheMixin and Field",
      "contribute_to_class() passes private_only=True to super(), so it's stored in model._meta.private_fields rather than local_fields",
      "contribute_to_class() installs a GenericForeignKeyDescriptor as a model attribute via setattr",
      "System checks are triggered via check_generic_foreign_keys in django/contrib/contenttypes/checks.py, registered in ContentTypesConfig.ready() under checks.Tags.models",
      "check_generic_foreign_keys finds GenericForeignKeyDescriptor instances by iterating vars(model).values() for all models",
      "GenericForeignKey.check() does NOT call super().check() \u2014 it only runs _check_field_name, _check_object_id_field, and _check_content_type_field",
      "_check_content_type_field can produce error IDs E002 (field missing), E003 (not a ForeignKey), E004 (FK not targeting ContentType), and E006 (uses DatabaseOnDelete)",
      "get_attname_column() returns (attname, None), explicitly setting the database column to None"
    ],
    "key_files": [
      "django/contrib/contenttypes/fields.py",
      "django/db/models/fields/mixins.py",
      "django/db/models/fields/__init__.py",
      "django/contrib/contenttypes/checks.py",
      "django/contrib/contenttypes/apps.py",
      "django/contrib/contenttypes/models.py"
    ],
    "source_doc": "[docstring: django/contrib/contenttypes/fields.py] django.contrib.contenttypes.fields.GenericForeignKey\ndjango.contrib.contenttypes.fields.GenericForeignKey:\n  Provide a generic many-to-one relation through the ``content_type`` and\n    ``object_id`` fields.\n\n    This class also doubles as an accessor to the related object (similar to\n    ForwardManyToOneDescriptor) by adding itself as a model attribute.\n\ndjango.contrib.contenttypes.fields.GenericForeignKey.get_filter_kwargs_for_object:\n  See corresponding method on Field\n\ndjango.contrib.contenttypes.fields.GenericForeignKey.get_forward_related_filter:\n  See corresponding method on RelatedField\n\ndjango.contrib.contenttypes.fields.GenericForeignKey._check_content_type_field:\n  Check if field named `field_name` in model `model` exists and is a\n        valid content_type field (is a ForeignKey to ContentType).",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in design rationale about caching, ticket references, and column semantics.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_49",
    "repo": "django",
    "question": "What is the architecture of Django's generic inline formset system, including its class hierarchy, factory function, queryset construction, and admin integration?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `generic_inlineformset_factory` function in `django/contrib/contenttypes/forms.py` creates and returns a formset class for managing generic inline objects (objects related via `GenericForeignKey`). Here's the detailed architecture:\n\n**Class Hierarchy:**\n- `BaseGenericInlineFormSet` extends `BaseModelFormSet` (from `django/forms/models.py`).\n- Notably, `BaseGenericInlineFormSet` does NOT inherit from `BaseInlineFormSet` \u2014 it directly inherits from `BaseModelFormSet`, implementing its own generic-relation-specific logic for filtering and saving.\n\n**Factory Function Architecture:**\n`generic_inlineformset_factory` validates the `ct_field` by checking it's a `ForeignKey` pointing to `ContentType`. It also resolves the `fk_field` from the model's `_meta`. Both field names are added to the `exclude` list so they don't appear in the generated forms. It then delegates to `modelformset_factory` (from `django/forms/models.py`). After `modelformset_factory` returns, three attributes are set directly on the FormSet class: `ct_field`, `ct_fk_field`, and `for_concrete_model`.\n\n**Default Parameter Differences:**\n- `generic_inlineformset_factory` defaults `extra=3`, `can_delete=True`, `can_order=False`\n\n**Queryset Construction in `BaseGenericInlineFormSet.__init__`:**\nWhen `instance` is `None` or its pk is not set (checked via `self.instance._is_pk_set()`), it returns `self.model._default_manager.none()`. Otherwise, it filters by both the content type (obtained via `ContentType.objects.get_for_model(self.instance, for_concrete_model=self.for_concrete_model)`) and the foreign key field value (the instance's pk).\n\n**Prefix Format:**\nThe `get_default_prefix` classmethod and `rel_name` in `__init__` both produce a prefix in the format `{app_label}-{model_name}-{ct_field_name}-{fk_field_name}` (e.g., `generic_relations-taggeditem-content_type-object_id`).\n\n**`save_new` Method:**\n`BaseGenericInlineFormSet.save_new` sets the content type pk using `ct_field.attname` and the object pk using `ct_fk_field.attname` directly on `form.instance`, then delegates to `form.save(commit=commit)`.\n\n**Admin Integration:**\n`GenericInlineModelAdmin` (in `django/contrib/contenttypes/admin.py`) uses `generic_inlineformset_factory` in its `get_formset` method. It has `ct_field = 'content_type'` and `ct_fk_field = 'object_id'` as class attributes. Two concrete subclasses exist: `GenericStackedInline` (template: `admin/edit_inline/stacked.html`) and `GenericTabularInline` (template: `admin/edit_inline/tabular.html`).\n\n**`for_concrete_model` Parameter:**\nDefaults to `True`. This affects both querying (in `__init__`, where it is passed to `get_for_model`) and saving (in `save_new`). Notably, `save_new` always calls `get_for_model` without passing `for_concrete_model`.",
    "rubric": [
      "BaseGenericInlineFormSet inherits from BaseModelFormSet (not BaseInlineFormSet)",
      "generic_inlineformset_factory validates ct_field is a ForeignKey to ContentType and excludes both ct_field and fk_field from forms",
      "The factory delegates to modelformset_factory and then sets ct_field, ct_fk_field, and for_concrete_model attributes on the resulting FormSet class",
      "Default parameters: extra=3, can_delete=True, can_order=False",
      "In __init__, when instance is None or pk is not set, returns model._default_manager.none(); otherwise filters by content type and instance pk",
      "Prefix format is {app_label}-{model_name}-{ct_field_name}-{fk_field_name}",
      "save_new sets content type pk and object pk on form.instance using attname, then calls form.save(commit=commit)",
      "GenericInlineModelAdmin uses generic_inlineformset_factory in get_formset, with ct_field='content_type' and ct_fk_field='object_id' as class attributes",
      "GenericStackedInline and GenericTabularInline are concrete subclasses with different templates (stacked.html and tabular.html)",
      "save_new calls get_for_model without passing for_concrete_model, unlike __init__ which passes self.for_concrete_model"
    ],
    "key_files": [
      "django/contrib/contenttypes/forms.py",
      "django/forms/models.py",
      "django/contrib/contenttypes/admin.py"
    ],
    "source_doc": "[docstring: django/contrib/contenttypes/forms.py] django.contrib.contenttypes.forms.generic_inlineformset_factory\nReturn a ``GenericInlineFormSet`` for the given kwargs.\n\n    You must provide ``ct_field`` and ``fk_field`` if they are different from\n    the defaults ``content_type`` and ``object_id`` respectively.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of the answer's claims go far beyond what the provided documentation states. The documentation is a minimal docstring that only says the function returns a GenericInlineFormSet and that ct_field and fk_field must be provided if different from defaults 'content_type' and 'object_id'.",
      "Claims about line numbers (106-110, 112, 130-132), class hierarchy details, default parameters, queryset construction, prefix format, save_new behavior, admin integration, and for_concrete_model behavior are all extrapolations from source code not included in the provided documentation chunk.",
      "The claim about save_new always calling get_for_model without passing for_concrete_model (meaning it always resolves to concrete model's content type) could be a hallucination or inaccuracy \u2014 this is not verifiable from the provided doc.",
      "The claim that BaseGenericInlineFormSet does NOT inherit from BaseInlineFormSet is plausible but not verifiable from the provided documentation.",
      "No claims directly contradict the provided documentation \u2014 the answer correctly states ct_field defaults to 'content_type' and fk_field defaults to 'object_id', matching the doc."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in semantic explanations of for_concrete_model behavior and truncated modelformset_factory defaults.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_48",
    "repo": "django",
    "question": "How does GenericRelation implement a reverse relation to GenericForeignKey, including its field setup, joins, content type filtering, and related manager?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `GenericRelation` class provides a reverse relation to `GenericForeignKey` by subclassing `ForeignObject` (from `django.db.models.fields.related`). Here is how it works in detail:\n\n**Initialization and Field Setup:**\n- `GenericRelation.__init__` defaults `object_id_field` to `\"object_id\"` and `content_type_field` to `\"content_type\"`. It always forces `null=True`, `blank=True`, `on_delete=models.CASCADE`, `editable=False`, and `serialize=False` in kwargs before calling the parent `ForeignObject.__init__`.\n- It passes `from_fields=[object_id_field]` and `to_fields=[]` (empty list) to the parent. The `to_fields` are lazily resolved later in `resolve_related_fields()`, which sets `self.to_fields = [self.model._meta.pk.name]` and returns a tuple mapping the remote model's `object_id_field` to the local model's primary key.\n- The field flags mark it as `one_to_many=True` and all others (`many_to_many`, `many_to_one`, `one_to_one`) as `False`, which is the inverse of `ForeignObject`'s defaults (`many_to_one=True`).\n\n**Relation Class (GenericRel):**\n- `GenericRelation.rel_class = GenericRel`, which extends `ForeignObjectRel`. Critically, `GenericRel.__init__` always uses `on_delete=DO_NOTHING` and defaults `related_name` to `related_query_name or \"+\"` \u2014 meaning if no `related_query_name` is provided, the reverse relation is hidden (the `+` suffix convention).\n\n**Content Type Filtering via get_extra_restriction:**\n- `get_extra_restriction(alias, remote_alias)` adds a WHERE clause that filters by the content type. It gets the content type field from the remote model using `self.content_type_field_name`, retrieves the content type PK via `self.get_content_type().pk`, creates an exact lookup on that field's column in the remote alias, and wraps it in a `WhereNode([lookup], connector=AND)`.\n\n**Swapped get_local/get_foreign_related_value:**\n- `GenericRelation` reverses the parent's `get_local_related_value` and `get_foreign_related_value`. The local version calls `get_instance_value_for_fields(instance, self.foreign_related_fields)` (swapped from normal), and the foreign version calls `get_instance_value_for_fields(instance, self.local_related_fields)` with an additional `to_python()` conversion step per field.\n\n**Path Info with Parent Model Inheritance:**\n- `get_path_info` checks if `object_id_field.model != opts.model` (i.e., the object_id field is defined on a parent model in an MTI chain). If so, it delegates to `_get_path_info_with_parent`, which builds a multi-step join: first from the current model to the parent model (where the GFK lives), then traverses the parent\u2192child chain by reversing the child\u2192parent ancestor links and using `field.remote_field.path_infos`.\n\n**MTI Inherited Flag:**\n- `mti_inherited = False` is the default. When a subclass inherits a `GenericRelation` from a parent in multi-table inheritance, Django's metaclass sets `field.mti_inherited = True` (in `ModelBase.__new__`). In `contribute_to_class`, if `self.mti_inherited` is `True`, the reverse relation is disabled by setting `self.remote_field.related_name = \"+\"` and `self.remote_field.related_query_name = None`.\n\n**Descriptor and Manager:**\n- `contribute_to_class` sets the model attribute to a `ReverseGenericManyToOneDescriptor` (subclass of `ReverseManyToOneDescriptor`). This descriptor overrides `related_manager_cls` to use `create_generic_related_manager()` instead of the standard `create_reverse_many_to_one_manager()`.\n- The dynamically created `GenericRelatedObjectManager` subclasses both the model's default manager class and `AltersData`. It sets `core_filters` to `{\"content_type__pk\": content_type_id, object_id_field_name: pk_val}`. It provides `add`, `remove`, `clear`, `set`, `create`, `get_or_create`, and `update_or_create` methods plus their async (`a`-prefixed) counterparts.\n- The `add` method with `bulk=True` uses a single `filter(pk__in=pks).update(...)` call, while `bulk=False` calls `obj.save()` in a loop within an atomic block.\n\n**Other Notable Details:**\n- `get_internal_type()` returns `\"ManyToManyField\"` \u2014 despite not actually being an M2M field.\n- `set_attributes_from_rel()` is overridden as a no-op (`pass`), preventing the parent's behavior of deriving `self.name` and `self.verbose_name` from the remote model.\n- `value_to_string(obj)` evaluates all related objects and returns a string representation of their PKs as a list.\n- `bulk_related_objects` filters using `_base_manager.db_manager(using)` with content_type__pk and object_id__in filters.\n- The `get_prefetch_querysets` in `GenericRelatedObjectManager` groups instances by content type using `itertools.groupby` and builds an OR query across content types.",
    "rubric": [
      "Explains that GenericRelation subclasses ForeignObject and passes from_fields=[object_id_field] and to_fields=[] (empty) to the parent, with to_fields lazily resolved in resolve_related_fields() to the model's primary key",
      "Notes that __init__ forces null=True, blank=True, on_delete=CASCADE, editable=False, serialize=False and defaults object_id_field to 'object_id' and content_type_field to 'content_type'",
      "Describes GenericRel as the rel_class, which always uses on_delete=DO_NOTHING and defaults related_name to related_query_name or '+' (hiding the reverse relation if no query name given)",
      "Explains that get_extra_restriction adds a WHERE clause filtering by content type PK using an exact lookup wrapped in a WhereNode",
      "Mentions the swapped get_local_related_value / get_foreign_related_value (local uses foreign_related_fields, foreign uses local_related_fields with to_python conversion)",
      "Describes get_path_info's handling of MTI: checks if object_id_field.model != opts.model and delegates to _get_path_info_with_parent which builds a multi-step join through the parent model",
      "Explains that contribute_to_class sets a ReverseGenericManyToOneDescriptor, and when mti_inherited is True, disables the reverse relation by setting related_name to '+'",
      "Describes the GenericRelatedObjectManager created by create_generic_related_manager, including core_filters using content_type__pk and object_id_field_name, and methods like add (with bulk=True using filter/update vs bulk=False using save in an atomic block)",
      "Notes that get_internal_type returns 'ManyToManyField' and set_attributes_from_rel is a no-op",
      "Mentions that get_prefetch_querysets groups instances by content type using itertools.groupby and builds an OR query across content types"
    ],
    "key_files": [
      "django/contrib/contenttypes/fields.py",
      "django/db/models/fields/related.py",
      "django/db/models/fields/reverse_related.py",
      "django/db/models/query_utils.py",
      "django/db/models/base.py"
    ],
    "source_doc": "[docstring: django/contrib/contenttypes/fields.py] django.contrib.contenttypes.fields.GenericRelation\ndjango.contrib.contenttypes.fields.GenericRelation:\n  Provide a reverse to a relation created by a GenericForeignKey.\n\ndjango.contrib.contenttypes.fields.GenericRelation._is_matching_generic_foreign_key:\n  Return True if field is a GenericForeignKey whose content type and\n        object id fields correspond to the equivalent attributes on this\n        GenericRelation.\n\ndjango.contrib.contenttypes.fields.GenericRelation._get_path_info_with_parent:\n  Return the path that joins the current model through any parent models.\n        The idea is that if you have a GFK defined on a parent model then we\n        need to join the parent model first, then the child model.\n\ndjango.contrib.contenttypes.fields.GenericRelation.get_content_type:\n  Return the content type associated with this field's model.\n\ndjango.contrib.contenttypes.fields.GenericRelation.bulk_related_objects:\n  Return all objects related to ``objs`` via this ``GenericRelation``.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only one design rationale sentence about the 'reverse nature' was removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_52",
    "repo": "django",
    "question": "How does the `get_flatpages` template tag parse its arguments and render its output, including how it handles site detection, URL prefixes, and user permissions?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `get_flatpages` template tag is implemented through two components: a parser function (`get_flatpages`) registered via `@register.tag` and a custom template node class (`FlatpageNode`) that extends `template.Node`.\n\n**Parsing Logic (get_flatpages function):**\nThe parser uses `token.split_contents()` to break the tag into bits and enforces that there must be between 3 and 6 bits (inclusive). It uses a clever even/odd parity check to determine if a URL prefix was provided: if `len(bits) % 2 == 0`, a prefix exists at `bits[1]`; if odd, there's no prefix. The second-to-last bit must be `'as'` and the last bit becomes the context variable name. For 5 or 6 bits, `bits[-4]` must be `'for'` and `bits[-3]` is the user variable. Any violation raises a `TemplateSyntaxError` with a message formatted using the actual tag name from `bits[0]`.\n\n**Rendering Logic (FlatpageNode.render):**\n1. **Site determination**: If `'request'` is in the template context, it calls `get_current_site(context['request']).pk` to determine the site. If there's no request in context, it falls back to `settings.SITE_ID`. This is important because `get_current_site()` (from `django.contrib.sites.shortcuts`) checks whether `django.contrib.sites` is installed \u2014 if yes, it uses `Site.objects.get_current(request)`, otherwise returns a `RequestSite`.\n\n2. **Querying**: It filters `FlatPage.objects.filter(sites__id=site_pk)`. If `starts_with` is set, it adds `url__startswith=self.starts_with.resolve(context)` \u2014 the prefix is resolved as a `template.Variable`, meaning it can be either a quoted string literal or a context variable.\n\n3. **User permission filtering**: If a user is provided, it resolves the user variable and checks `user.is_authenticated`. If the user is not authenticated (e.g., `AnonymousUser`) or no user is provided at all, it filters to `registration_required=False` (public pages only). Only when an authenticated user is provided are `registration_required=True` flatpages included.\n\n4. **Output**: The queryset is assigned to `context[self.context_name]` and the render method returns an empty string `\"\"`. This means the tag itself produces no output \u2014 it only populates the context.\n\n**FlatPage Model:**\nThe `FlatPage` model has fields: `url` (CharField, max_length=100, db_index=True), `title` (max_length=200), `content` (TextField, blank), `enable_comments` (BooleanField, default=False), `template_name` (CharField, max_length=70, blank), `registration_required` (BooleanField, default=False), and `sites` (ManyToManyField to Site). The model's `Meta` class sets `db_table='django_flatpage'` and `ordering=['url']`, which means the queryset returned by the template tag is ordered by URL by default.\n\nThe `FlatpageNode.__init__` wraps both `starts_with` and `user` in `template.Variable()` instances, which use the `Variable.resolve(context)` mechanism from Django's template engine. This means the `starts_with` parameter can be a quoted string (resolved as a literal) or an unquoted variable name (resolved via context lookup with dot-separated attribute access).",
    "rubric": [
      "Explains the parser uses token.split_contents() and enforces 3-6 bits, using even/odd parity to determine if a URL prefix was provided",
      "Mentions the second-to-last bit must be 'as' and last bit becomes the context variable name; for 5-6 bits, bits[-4] must be 'for' and bits[-3] is the user",
      "Explains FlatpageNode.render determines the site by checking for 'request' in context (using get_current_site) or falling back to settings.SITE_ID",
      "Describes that the queryset filters FlatPage.objects.filter(sites__id=site_pk) and optionally adds url__startswith filter when a prefix is provided",
      "Explains that starts_with and user are wrapped in template.Variable() so they can be either literal strings or context variable references resolved at render time",
      "Describes user permission logic: if user is not authenticated or no user is provided, filters to registration_required=False; authenticated users see all pages",
      "Notes that render() assigns the queryset to context[self.context_name] and returns an empty string (produces no direct output)",
      "Mentions the FlatPage model has ordering=['url'] in Meta, so results are ordered by URL by default"
    ],
    "key_files": [
      "django/contrib/flatpages/templatetags/flatpages.py",
      "django/contrib/flatpages/models.py",
      "django/contrib/sites/shortcuts.py",
      "tests/flatpages_tests/test_templatetags.py"
    ],
    "source_doc": "[docstring: django/contrib/flatpages/templatetags/flatpages.py] django.contrib.flatpages.templatetags.flatpages.get_flatpages\nRetrieve all flatpage objects available for the current site and\n    visible to the specific user (or visible to all users if no user is\n    specified). Populate the template context with them in a variable\n    whose name is defined by the ``as`` clause.\n\n    An optional ``for`` clause controls the user whose permissions are used in\n    determining which flatpages are visible.\n\n    An optional argument, ``starts_with``, limits the returned flatpages to\n    those beginning with a particular base URL. This argument can be a variable\n    or a string, as it resolves from the template context.\n\n    Syntax::\n\n        {% get_flatpages ['url_starts_with'] [for user] as context_name %}\n\n    Example usage::\n\n        {% get_flatpages as flatpages %}\n        {% get_flatpages for someuser as flatpages %}\n        {% get_flatpages '/about/' as about_pages %}\n        {% get_flatpages prefix as about_pages %}\n        {% get_flatpages '/about/' for someuser as about_pages %}",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (parity check logic, FlatpageNode class internals, FlatPage model fields, db_table name, etc.) that go far beyond what the documentation states. These are plausible code-level details but cannot be verified from the provided documentation alone.",
      "The answer's description of user permission filtering (authenticated users see registration_required=True pages, unauthenticated users only see registration_required=False) is consistent with the doc's statement about 'visible to the specific user (or visible to all users if no user is specified)' but adds specifics not in the doc.",
      "The claim about even/odd parity check for prefix detection is a plausible implementation detail but not verifiable from the doc.",
      "The claim about fallback to settings.SITE_ID when no request is in context is plausible but not stated in the documentation."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only a minor reference to the exact base class path was slightly adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_53",
    "repo": "django",
    "question": "What are the key components and design of Django's flatpages system, including its view, model, middleware, and URL routing?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `django.contrib.flatpages.views.flatpage` function is the main view function for the flat page system, used in URL routing and called by the middleware. It takes a `request` and a `url` parameter, normalizes the URL to ensure it starts with a leading slash, looks up the FlatPage object filtered by the URL and current site ID, and delegates rendering to `render_flatpage()`. Key implementation details:\n\n1. **URL normalization and APPEND_SLASH fallback**: If the initial lookup raises Http404 and the URL doesn't end with `/` and `settings.APPEND_SLASH` is True, it retries with a trailing slash appended. If found, it returns an `HttpResponsePermanentRedirect` to the slash-appended path.\n\n2. **CSRF protection architecture**: The `@csrf_protect` decorator is applied only to the internal `render_flatpage()` function, not to `flatpage()` itself.\n\n3. **`render_flatpage()` internals**: It checks `f.registration_required` and redirects unauthenticated users via `django.contrib.auth.views.redirect_to_login(request.path)`. It uses `loader.select_template()` when the FlatPage has a `template_name` set, passing a tuple of `(f.template_name, DEFAULT_TEMPLATE)`, otherwise using `loader.get_template(DEFAULT_TEMPLATE)` where DEFAULT_TEMPLATE is `\"flatpages/default.html\"`. It marks both `f.title` and `f.content` as safe using `mark_safe()`.\n\n4. **FlatPage model**: The model uses a db_table of `\"django_flatpage\"`, has a `ManyToManyField` to `Site`, and fields include `url` (max_length=100, db_index=True), `title` (max_length=200), `content` (TextField, blank=True), `enable_comments` (BooleanField, default=False), `template_name` (max_length=70, blank=True), and `registration_required` (BooleanField, default=False). The model is ordered by `[\"url\"]`.\n\n5. **FlatpageFallbackMiddleware**: Extends `MiddlewareMixin`, intercepts 404 responses in `process_response`, calls `flatpage(request, request.path_info)`, and returns the original response if Http404 or any other exception occurs (re-raising in DEBUG mode for non-Http404 exceptions).\n\n6. **URL routing**: The flatpages URL pattern uses `path(\"<path:url>\", views.flatpage, name=\"django.contrib.flatpages.views.flatpage\")`.",
    "rubric": [
      "The flatpage() view normalizes URLs to start with a leading slash and looks up FlatPage objects filtered by URL and current site ID",
      "If the initial lookup raises Http404 and the URL lacks a trailing slash and APPEND_SLASH is True, it retries with a slash appended and returns an HttpResponsePermanentRedirect",
      "The @csrf_protect decorator is applied to the internal render_flatpage() function, not to the outer flatpage() view",
      "render_flatpage() checks f.registration_required and redirects unauthenticated users via redirect_to_login(request.path)",
      "render_flatpage() uses loader.select_template() with (f.template_name, DEFAULT_TEMPLATE) when template_name is set, otherwise loader.get_template(DEFAULT_TEMPLATE) where DEFAULT_TEMPLATE is 'flatpages/default.html'",
      "render_flatpage() marks both f.title and f.content as safe using mark_safe()",
      "FlatPage model uses db_table 'django_flatpage', has a ManyToManyField to Site, and includes url (max_length=100, db_index=True), title, content, enable_comments, template_name, and registration_required fields, ordered by ['url']",
      "FlatpageFallbackMiddleware extends MiddlewareMixin, intercepts 404 responses in process_response, calls flatpage(request, request.path_info), and returns the original response on Http404 (re-raises non-Http404 exceptions in DEBUG mode)",
      "The URL pattern uses path('<path:url>', views.flatpage, name='django.contrib.flatpages.views.flatpage')"
    ],
    "key_files": [
      "django/contrib/flatpages/views.py",
      "django/contrib/flatpages/models.py",
      "django/contrib/flatpages/middleware.py",
      "django/contrib/flatpages/urls.py"
    ],
    "source_doc": "[docstring: django/contrib/flatpages/views.py] django.contrib.flatpages.views.flatpage\nPublic interface to the flat page view.\n\n    Models: `flatpages.flatpages`\n    Templates: Uses the template defined by the ``template_name`` field,\n        or :template:`flatpages/default.html` if template_name is not defined.\n    Context:\n        flatpage\n            `flatpages.flatpages` object",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in design rationale for CSRF placement and explanatory parentheticals about mark_safe and select_template purposes.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_51",
    "repo": "django",
    "question": "Where do ContentType's `get_object_for_this_type` and `get_all_objects_for_this_type` get called across the codebase, and how does their data flow work?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `ContentType.model_class()` method calls `apps.get_model(self.app_label, self.model)` (from `django.apps.registry.Apps`) which looks up the model in the app registry, catching `LookupError` and returning `None` if the model is not found. \n\n`get_object_for_this_type(using=None, **kwargs)` chains through: `model_class()` \u2192 `_base_manager` (a property on the model's metaclass that delegates to `cls._meta.base_manager`, a cached_property in `django/db/models/options.py`) \u2192 `.using(using)` \u2192 `.get(**kwargs)`. The `_base_manager` property on the model class (defined in `django/db/models/base.py`) simply returns `cls._meta.base_manager`.\n\n`get_all_objects_for_this_type(**kwargs)` is similar but calls `.filter(**kwargs)` instead of `.get()` and does NOT accept a `using` parameter.\n\nKey callers:\n1. `GenericForeignKeyDescriptor`'s `fetch_one()` method in `django/contrib/contenttypes/fields.py` calls `ct.get_object_for_this_type(using=instance._state.db, pk=pk_val)` and catches `ObjectDoesNotExist`.\n2. `GenericForeignKeyDescriptor`'s `get_prefetch_querysets()` in `django/contrib/contenttypes/fields.py` calls `ct.get_all_objects_for_this_type(pk__in=fkeys)` for batch prefetching of generic relations.\n3. `LogEntry.get_edited_object()` in `django/contrib/admin/models.py` calls `self.content_type.get_object_for_this_type(pk=self.object_id)`.\n4. The `shortcut()` view in `django/contrib/contenttypes/views.py` calls `content_type.get_object_for_this_type(pk=object_id)`.\n\nThe `ContentTypeManager._add_to_cache` method stores each ContentType in the cache dict under two keys: the `(app_label, model)` tuple AND the `ct.id` integer, both within the same `using` database namespace.",
    "rubric": [
      "Explains that model_class() calls apps.get_model(self.app_label, self.model), catching LookupError and returning None if not found",
      "Explains get_object_for_this_type chains through model_class()._base_manager.using(using).get(**kwargs)",
      "Explains get_all_objects_for_this_type chains through model_class()._base_manager.filter(**kwargs) and does NOT accept a using parameter",
      "Mentions _base_manager property on the model class delegates to cls._meta.base_manager",
      "Identifies GenericForeignKeyDescriptor.fetch_one() as a caller of get_object_for_this_type (passing using=instance._state.db and pk=pk_val, catching ObjectDoesNotExist)",
      "Identifies GenericForeignKeyDescriptor.get_prefetch_querysets() as a caller of get_all_objects_for_this_type(pk__in=fkeys)",
      "Identifies LogEntry.get_edited_object() as a caller of get_object_for_this_type(pk=self.object_id)",
      "Identifies the shortcut() view as a caller of content_type.get_object_for_this_type(pk=object_id)",
      "Explains that ContentTypeManager._add_to_cache stores each ContentType under both an (app_label, model) tuple key and the ct.id integer key within the same using-database namespace"
    ],
    "key_files": [
      "django/contrib/contenttypes/models.py",
      "django/contrib/contenttypes/fields.py",
      "django/db/models/base.py",
      "django/db/models/options.py",
      "django/apps/registry.py"
    ],
    "source_doc": "[docstring: django/contrib/contenttypes/models.py] django.contrib.contenttypes.models.ContentType\ndjango.contrib.contenttypes.models.ContentType.model_class:\n  Return the model class for this type of content.\n\ndjango.contrib.contenttypes.models.ContentType.get_object_for_this_type:\n  Return an object of this type for the keyword arguments given.\n        Basically, this is a proxy around this object_type's get_object() model\n        method. The ObjectNotExist exception, if thrown, will not be caught,\n        so code that calls this method should catch it.\n\ndjango.contrib.contenttypes.models.ContentType.get_all_objects_for_this_type:\n  Return all objects of this type for the keyword arguments given.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states get_object_for_this_type accepts a `using=None` parameter, which is plausible but not explicitly documented in the provided docstring.",
      "The answer states get_all_objects_for_this_type does NOT accept a `using` parameter, which is plausible but not documented.",
      "The documentation says get_object_for_this_type is 'a proxy around this object_type's get_object() model method', but the answer describes it as chaining through `_base_manager.using(using).get(**kwargs)` \u2014 the docstring says 'get_object()' not 'get()', though this is likely just an older/informal name for the same thing.",
      "The documentation mentions 'ObjectNotExist exception' while the answer references 'ObjectDoesNotExist' \u2014 the docstring uses 'ObjectNotExist' which is slightly different naming (though Django's actual exception is `ObjectDoesNotExist`).",
      "Many specific claims about line numbers, caller locations, and internal implementation details (e.g., `_add_to_cache`, `_base_manager` property at line 454) are not verifiable from the provided documentation \u2014 they are plausible code-level details but could be hallucinated."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor adjustments to remove unverifiable line numbers and one inference about router behavior; core claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_50",
    "repo": "django",
    "question": "Why does the ContentTypeManager use a dual-key caching strategy and what design decisions shape its overall caching architecture?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The ContentTypeManager's caching design is driven by several key design rationales:\n\n**1. Dual-key cache for cross-method sharing (`_add_to_cache`)**\nThe `_add_to_cache` method stores each ContentType under TWO keys in the same dictionary: a `(app_label, model)` tuple AND the integer `ct.id`. This allows `get_for_model` (which looks up by app_label/model_name tuple via `_get_from_cache`) and `get_for_id` (which looks up by integer ID) and `get_by_natural_key` (which looks up by (app_label, model) tuple) to all share a single cache dict per database. Once any of these methods populates the cache, the others benefit without additional DB queries.\n\n**2. Database-aware cache keying for multi-database support**\nThe cache is structured as `self._cache[self.db][key]`, where `self.db` is the database alias. The `db` property is inherited from `BaseManager` and resolves to `self._db or router.db_for_read(self.model, **self._hints)`. This means in multi-database setups, each database gets its own cache partition, preventing cross-database cache contamination.\n\n**3. Two-phase lookup in `get_for_model`**\nThe method uses `self.get()` first before falling back to `self.get_or_create()`. Since `get_or_create()` is designed for write operations (setting `_for_write = True` on the QuerySet), starting with `get()` ensures the initial lookup uses the read database path. Only if the ContentType doesn't exist does it fall back to `get_or_create()`.\n\n**4. Race condition protection via `get_or_create`**\nWhen `get_for_model` fails to find a ContentType in the DB, it uses `get_or_create()` (not bare `create()`) for the second attempt. This handles concurrent insert race conditions, as `get_or_create` is designed to handle IntegrityError from concurrent inserts.\n\n**5. Stale ContentType safety in `_add_to_cache`**\n`_add_to_cache` uses `ct.app_label` and `ct.model` (database field values) rather than `model._meta.app_label`. This is because ContentType objects can be 'stale' \u2014 referring to models that no longer exist in the codebase \u2014 and `model_class()` would return `None` for these (as implemented in `ContentType.model_class()`, which catches `LookupError` from `apps.get_model()`).\n\n**6. Batch query optimization in `get_for_models`**\nUnlike `get_for_model` which does individual lookups, `get_for_models` constructs a single batched `Q` filter using OR-connected conditions grouped by `app_label` with `model__in`. It uses `needed_opts.pop()` during result iteration so that after processing DB results, any remaining entries in `needed_opts` represent ContentTypes that need to be created. Notably, `get_for_models` uses bare `create()` rather than `get_or_create()` for missing entries, unlike `get_for_model`.\n\n**7. Per-instance cache, not shared across managers**\nThe `_cache` dict is initialized in `__init__`, making it per-manager-instance. Each `ContentTypeManager()` instance has its own independent cache.\n\n**8. `use_in_migrations = True`**\nThis attribute ensures the ContentTypeManager is serialized into migration state, allowing migration operations like `RenameContentType` to use its methods (like `get_by_natural_key`).",
    "rubric": [
      "Explains that _add_to_cache stores each ContentType under two keys: a (app_label, model) tuple AND the integer ct.id, enabling get_for_model, get_for_id, and get_by_natural_key to share one cache",
      "Explains that the cache is keyed by database alias (self.db) to support multi-database setups and prevent cross-database contamination",
      "Explains that get_for_model uses get() before get_or_create() to ensure the initial lookup uses the read database (db_for_read) rather than the write path",
      "Explains that get_or_create (not bare create) is used in get_for_model to handle race conditions from concurrent inserts",
      "Explains that _add_to_cache uses ct.app_label and ct.model (field values) rather than model._meta because ContentTypes can be stale (model_class() may return None)",
      "Explains that get_for_models uses a batched Q filter with OR conditions for efficient bulk lookup, and uses pop() on needed_opts so remaining entries represent ContentTypes needing creation",
      "Notes that use_in_migrations = True ensures the manager is available during migrations"
    ],
    "key_files": [
      "django/contrib/contenttypes/models.py",
      "django/db/models/manager.py",
      "django/db/models/query.py",
      "django/contrib/contenttypes/fields.py",
      "django/contrib/contenttypes/management/__init__.py"
    ],
    "source_doc": "[docstring: django/contrib/contenttypes/models.py] django.contrib.contenttypes.models.ContentTypeManager\ndjango.contrib.contenttypes.models.ContentTypeManager.get_for_model:\n  Return the ContentType object for a given model, creating the\n        ContentType if necessary. Lookups are cached so that subsequent lookups\n        for the same model don't hit the database.\n\ndjango.contrib.contenttypes.models.ContentTypeManager.get_for_models:\n  Given *models, return a dictionary mapping {model: content_type}.\n\ndjango.contrib.contenttypes.models.ContentTypeManager.get_for_id:\n  Lookup a ContentType by ID. Use the same shared cache as get_for_model\n        (though ContentTypes are not created on-the-fly by get_by_id).\n\ndjango.contrib.contenttypes.models.ContentTypeManager.clear_cache:\n  Clear out the content-type cache.\n\ndjango.contrib.contenttypes.models.ContentTypeManager._add_to_cache:\n  Insert a ContentType into the cache.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rely on comments (ticket references, design rationale explanations) or test files not provided.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_56",
    "repo": "django",
    "question": "How does the Oracle Spatial backend implement spatial operators, geometry conversion, and NULL handling in its operations class?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `django/contrib/gis/db/backends/oracle/operations.py` module implements the Oracle Spatial backend for Django's GIS framework through the `OracleOperations` class, which uses multiple inheritance from both `BaseSpatialOperations` and `DatabaseOperations` (the non-GIS Oracle backend).\n\n**Spatial Operator Implementation:**\nThe module defines four custom `SpatialOperator` subclasses that generate Oracle-specific SQL:\n\n1. **`SDOOperator`** \u2014 Uses template `%(func)s(%(lhs)s, %(rhs)s) = 'TRUE'` for standard SDO functions like `SDO_CONTAINS`, `SDO_COVEREDBY`, `SDO_COVERS`, `SDO_EQUAL`, `SDO_OVERLAPS`, `SDO_TOUCH`, `SDO_INSIDE`, and `SDO_OVERLAPBDYINTERSECT` (used for `intersects`).\n\n2. **`SDODWithin`** \u2014 Uses `SDO_WITHIN_DISTANCE(%(lhs)s, %(rhs)s, %%s) = 'TRUE'` with a special string parameter format `\"distance=%s\"` (constructed in `get_distance()`).\n\n3. **`SDODisjoint`** \u2014 Uses `SDO_GEOM.RELATE` with the hardcoded `DEFAULT_TOLERANCE` of `\"0.05\"`, generating SQL like `SDO_GEOM.RELATE(lhs, 'DISJOINT', rhs, 0.05) = 'DISJOINT'`.\n\n4. **`SDORelate`** \u2014 Validates the mask argument against a regex of allowed Oracle spatial masks (`TOUCH|OVERLAPBDYDISJOINT|OVERLAPBDYINTERSECT|EQUAL|INSIDE|COVEREDBY|CONTAINS|COVERS|ANYINTERACT|ON`), supports `+` combinations (e.g., `'inside+touch'`), and generates SQL like `SDO_RELATE(lhs, rhs, 'mask=...')`. The `as_sql` method extracts the mask from `sql_params[-1]`, puts it into `template_params[\"mask\"]`, and passes the remaining params to the parent.\n\n**Geometry Conversion Pipeline:**\nThe `select` attribute is `\"SDO_UTIL.TO_WKBGEOMETRY(%s)\"`, meaning all geometry columns are converted to WKB on the database side. The `get_geometry_converter` method creates a closure that:\n1. Obtains a `WKBReader` instance via `wkb_r()` (from `django.contrib.gis.geos.prototypes.io`)\n2. Reads the Oracle CLOB value as `memoryview(value.read())`\n3. Constructs a `GEOSGeometryBase` object from the WKB data with the appropriate `geom_class`\n4. Sets the SRID (unless it's -1, which maps to None)\n\n**Extent Conversion:**\n`convert_extent` handles two cases: Oracle may return either a `Polygon` (where it extracts corners `shell[0][:2]` and `shell[2][:2]`) or a single `Point` (where lower-left equals upper-right). For any other geometry type, it raises an Exception.\n\n**NULL Handling:**\n`get_geom_placeholder_sql` returns `(\"NULL\", ())` for None values, and `modify_insert_params` drops insert parameters when the placeholder is `\"NULL\"`.\n\n**Adapter:**\nThe `OracleSpatialAdapter` (extending `WKTAdapter`) fixes polygon ring orientation: outer rings must be counter-clockwise, inner rings must be clockwise. It also handles `GeometryCollection` instances containing polygons. Its `input_size` is `oracledb.CLOB`.\n\n**Area attribute:**\n`get_area_att_for_field` always returns `\"sq_m\"`, unconditionally \u2014 unlike the base class which checks geodetic/non-geodetic and may raise `NotImplementedError`.\n\n**Unsupported functions:**\nThe `unsupported_functions` cached property conditionally adds `\"GeometryType\"` to the unsupported set only for Oracle versions below 23.",
    "rubric": [
      "Mentions the four SpatialOperator subclasses: SDOOperator, SDODWithin, SDODisjoint, and SDORelate",
      "Explains SDOOperator uses the template `%(func)s(%(lhs)s, %(rhs)s) = 'TRUE'` for standard SDO functions",
      "Explains SDODWithin uses `SDO_WITHIN_DISTANCE` with a `distance=%s` string parameter format",
      "Explains SDODisjoint uses SDO_GEOM.RELATE with DEFAULT_TOLERANCE of '0.05' and a hardcoded 'DISJOINT' check",
      "Explains SDORelate validates mask arguments against allowed Oracle spatial masks via regex and extracts the mask from sql_params[-1] in as_sql",
      "Describes the geometry conversion pipeline: select uses SDO_UTIL.TO_WKBGEOMETRY, get_geometry_converter creates a closure using WKBReader on CLOB data to construct GEOSGeometryBase objects",
      "Explains convert_extent handles Polygon (extracting shell corners) and Point (lower-left equals upper-right) cases",
      "Describes NULL handling: get_geom_placeholder_sql returns ('NULL', ()) for None values and modify_insert_params drops params when placeholder is 'NULL'",
      "Mentions OracleSpatialAdapter fixes polygon ring orientation (outer CCW, inner CW) and has input_size of oracledb.CLOB",
      "Notes that unsupported_functions conditionally adds 'GeometryType' only for Oracle versions below 23"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/oracle/operations.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/utils.py",
      "django/contrib/gis/db/backends/oracle/adapter.py",
      "django/contrib/gis/db/models/lookups.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/oracle/operations.py] django.contrib.gis.db.backends.oracle.operations\nThis module contains the spatial lookup types, and the `get_geo_where_clause`\nroutine for Oracle Spatial.\n\nPlease note that WKT support is broken on the XE version, and thus\nthis backend will not work on such platforms. Specifically, XE lacks\nsupport for an internal JVM, and Java libraries are required to use\nthe WKT constructors.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: ticket reference (#10888), 'thread-local' characterization of wkb_r(), and explanation of when Point case occurs in convert_extent. All other claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_54",
    "repo": "django",
    "question": "Why does the `SpatialRefSysMixin` exist, and how does its `srs` property handle differences between PostGIS, SpatiaLite, and Oracle backends?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `SpatialRefSysMixin` exists as a design pattern to reduce redundant code across three different database-specific `SpatialRefSys` model implementations: `PostGISSpatialRefSys` (PostGIS), `SpatialiteSpatialRefSys` (SpatiaLite), and `OracleSpatialRefSys` (Oracle). Each database stores spatial reference system metadata differently (different table names, column names, and available fields), but they all need the same computed properties. The mixin bridges these differences by delegating to a GDAL `SpatialReference` object.\n\nKey design decisions revealed by reading the code:\n\n1. **Fallback mechanism in `srs` property**: The `srs` property (a `cached_property`) first tries to create a GDAL `SpatialReference` from `self.wkt`, and if that fails, falls back to `self.proj4text`. This is important because Oracle's `OracleSpatialRefSys` model has no `proj4text` field (it only has `wktext` mapped via a `wkt` property), so if WKT parsing fails on Oracle, it will raise an `AttributeError` before reaching the combined error message. PostGIS and SpatiaLite both have both `srtext` (mapped to `wkt` property) and `proj4text` fields.\n\n2. **`cached_property` for expensive GDAL call**: The `srs` property uses `cached_property` (from `django.utils.functional`) rather than a plain `@property`. This ensures the GDAL `SpatialReference` object is only created once per model instance (the GDAL SpatialReference involves ctypes calls into C libraries, as visible in `django/contrib/gis/gdal/srs.py`). All other properties (`ellipsoid`, `name`, `spheroid`, `datum`, etc.) use regular `@property` and delegate to `self.srs`, so they benefit from this caching.\n\n3. **Database-table abstraction**: Each backend maps to a different database table: PostGIS uses `spatial_ref_sys`, SpatiaLite also uses `spatial_ref_sys`, and Oracle uses `CS_SRS`. The Oracle model uniquely includes a `cs_bounds` `PolygonField` (nullable). All models have `managed = False`.\n\n4. **The `wkt` property pattern**: Each concrete subclass implements a `wkt` property that maps to its specific column name \u2014 `srtext` for PostGIS and SpatiaLite, `wktext` for Oracle. This normalizes access for the mixin.\n\n5. **`get_spheroid` output format**: When `string=True` (the default), it formats the result as `SPHEROID[\"name\",radius,flattening]`. When `string=False`, it returns a tuple of `(sphere_name, sphere_params)`.\n\n6. **Integration with `get_srid_info`**: The `get_srid_info` function in `django/contrib/gis/db/models/fields.py` uses the `spatial_ref_sys()` model returned by connection ops, calls `.srs` on it to get the GDAL `SpatialReference`, and caches the results in a module-level `_srid_cache` dictionary (keyed by connection alias and SRID). Notably, this function does NOT use `get_spheroid` \u2014 it constructs the spheroid string directly from `srs[\"spheroid\"]`, `srs.semi_major`, and `srs.inverse_flattening`.",
    "rubric": [
      "Explains the mixin reduces redundant code across three database-specific SpatialRefSys models (PostGIS, SpatiaLite, Oracle) that map to different tables and column names",
      "Describes the `srs` property's fallback mechanism: tries WKT first, then falls back to proj4text, noting Oracle lacks a proj4text field",
      "Mentions that `srs` uses `cached_property` to avoid recreating the expensive GDAL SpatialReference object, while derived properties like `ellipsoid`, `name`, `datum`, etc. use regular `@property` and delegate to `self.srs`",
      "Explains the `wkt` property pattern: each subclass maps its specific column (srtext for PostGIS/SpatiaLite, wktext for Oracle) to a normalized `wkt` property used by the mixin",
      "Notes the different database tables: PostGIS and SpatiaLite use `spatial_ref_sys`, Oracle uses `CS_SRS`, all with `managed = False`",
      "Describes how `get_srid_info` uses the mixin's `.srs` property to obtain SRID metadata and caches results in a module-level `_srid_cache` dictionary"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/base/models.py",
      "django/contrib/gis/gdal/srs.py",
      "django/contrib/gis/db/backends/postgis/models.py",
      "django/contrib/gis/db/backends/oracle/models.py",
      "django/contrib/gis/db/backends/spatialite/models.py",
      "django/contrib/gis/db/models/fields.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/base/models.py] django.contrib.gis.db.backends.base.models.SpatialRefSysMixin\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin:\n  The SpatialRefSysMixin is a class used by the database-dependent\n    SpatialRefSys objects to reduce redundant code.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.srs:\n  Return a GDAL SpatialReference object.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.ellipsoid:\n  Return a tuple of the ellipsoid parameters:\n        (semimajor axis, semiminor axis, and inverse flattening).\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.name:\n  Return the projection name.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.spheroid:\n  Return the spheroid name for this spatial reference.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.datum:\n  Return the datum for this spatial reference.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.projected:\n  Is this Spatial Reference projected?\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.local:\n  Is this Spatial Reference local?\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.geographic:\n  Is this Spatial Reference geographic?\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.linear_name:\n  Return the linear units name.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.linear_units:\n  Return the linear units.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.angular_name:\n  Return the name of the angular units.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.angular_units:\n  Return the angular units.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.units:\n  Return a tuple of the units and the name.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.get_units:\n  Return a tuple of (unit_value, unit_name) for the given WKT without\n        using any of the database fields.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.get_spheroid:\n  Class method used by GeometryField on initialization to\n        retrieve the `SPHEROID[..]` parameters from the given WKT.\n\ndjango.contrib.gis.db.backends.base.models.SpatialRefSysMixin.__str__:\n  Return the string representation, a 'pretty' OGC WKT.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (PostGISSpatialRefSys, SpatialiteSpatialRefSys, OracleSpatialRefSys, cached_property usage, fallback mechanisms, database table names, get_srid_info function details, _srid_cache dictionary, etc.) that are not present in the provided documentation. These appear to be derived from reading the actual source code rather than the documentation chunk provided.",
      "The documentation only states that SpatialRefSysMixin 'is a class used by the database-dependent SpatialRefSys objects to reduce redundant code' \u2014 the specific databases (PostGIS, SpatiaLite, Oracle) and their implementation details are extrapolations/external knowledge not verifiable from this doc.",
      "The claim about get_spheroid producing 'PostGIS-compatible output' with a specific format when string=True is not supported by the documentation, which only says it retrieves SPHEROID parameters from WKT.",
      "No claims in the answer directly contradict the documentation \u2014 the basic descriptions of properties (srs returns GDAL SpatialReference, ellipsoid returns tuple of parameters, etc.) are consistent with the doc."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rationale statements and one explicit comment reference needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_57",
    "repo": "django",
    "question": "What are the inheritance dependencies and key method implementations of the Oracle spatial operations class?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `OracleOperations` class in `django/contrib/gis/db/backends/oracle/operations.py` uses multiple inheritance from `BaseSpatialOperations` and `DatabaseOperations` (the non-spatial Oracle operations class from `django.db.backends.oracle.operations`). Here's a detailed dependency trace:\n\n**Class hierarchy**: `OracleOperations(BaseSpatialOperations, DatabaseOperations)` where `BaseSpatialOperations` is from `django.contrib.gis.db.backends.base.operations` and `DatabaseOperations` is from `django.db.backends.oracle.operations`.\n\n**Key implementation details**:\n\n1. **`geo_db_type()`** always returns the string `\"MDSYS.SDO_GEOMETRY\"` regardless of the geometry field type passed in. The base class `BaseSpatialOperations.geo_db_type()` raises `NotImplementedError`.\n\n2. **`get_distance()`** handles the `dwithin` lookup specially by formatting the distance parameter as `\"distance=%s\" % dist_param`. For geodetic fields, it uses `value.m` (meters). For non-geodetic fields, it dynamically gets the appropriate unit attribute via `Distance.unit_attname(f.units_name(self.connection))`.\n\n3. **`modify_insert_params()`** overrides the base `BaseDatabaseOperations.modify_insert_params()`, which simply returns params unchanged. The Oracle spatial override returns an empty list `[]` when the placeholder is `\"NULL\"`.\n\n4. **`spatial_aggregate_name()`** maps `\"union\"` to attribute `\"unionagg\"` (which is `\"SDO_AGGR_UNION\"`), and for other aggregate names, it lowercases and looks up the attribute (e.g., `self.extent` = `\"SDO_AGGR_MBR\"`).\n\n5. **Adapter**: The `OracleSpatialAdapter` extends `WKTAdapter` from `django.contrib.gis.db.backends.base.adapter` and has `input_size = oracledb.CLOB`. It fixes polygon ring orientation: outer rings must be counter-clockwise, inner rings must be clockwise.\n\n6. **`get_geometry_converter()`** uses `wkb_r().read` (a WKB reader imported from `django.contrib.gis.geos.prototypes.io`) to convert Oracle's WKB output back to `GEOSGeometryBase` objects. When `srid == -1`, it is set to `None`.\n\n7. **`select` attribute** is `\"SDO_UTIL.TO_WKBGEOMETRY(%s)\"` \u2014 Oracle geometry is selected as WKB.\n\n8. **`convert_extent()`** takes a CLOB parameter (unlike the base class which takes `(box, srid)`) and handles both Polygon and Point returns from Oracle's `SDO_AGGR_MBR` aggregate.\n\n9. **`DEFAULT_TOLERANCE`** is `\"0.05\"` and is used in `SDODisjoint`'s SQL template.\n\n10. **`get_area_att_for_field()`** always returns `\"sq_m\"` regardless of the field, overriding the base class logic that checks for geodetic vs. projected coordinate systems.\n\n11. **`from_text`** is set to `\"SDO_GEOMETRY\"`, used in `get_geom_placeholder_sql()` from the base class.\n\n12. **`unsupported_functions`** is a `@cached_property` that conditionally adds `\"GeometryType\"` if the Oracle version is below 23.\n\n13. **`geo_quote_name()`** calls `super().geo_quote_name(name).upper()` \u2014 the base class wraps in single quotes, then Oracle uppercases it.\n\n14. The `DatabaseWrapper` in `django/contrib/gis/db/backends/oracle/base.py` sets `ops_class = OracleOperations`, `features_class = DatabaseFeatures`, and `SchemaEditorClass = OracleGISSchemaEditor`.\n\n15. **SDORelate** validates mask arguments against a regex of Oracle-specific masks (TOUCH, OVERLAPBDYDISJOINT, OVERLAPBDYINTERSECT, EQUAL, INSIDE, COVEREDBY, CONTAINS, COVERS, ANYINTERACT, ON) joined by `+`.",
    "rubric": [
      "Identifies OracleOperations inherits from both BaseSpatialOperations and DatabaseOperations (from django.db.backends.oracle.operations)",
      "geo_db_type() always returns 'MDSYS.SDO_GEOMETRY' regardless of field type, overriding base class NotImplementedError",
      "get_distance() formats dwithin lookups as 'distance=%s' and uses value.m for geodetic fields",
      "modify_insert_params() returns empty list when placeholder is 'NULL'",
      "spatial_aggregate_name() maps 'union' to 'unionagg' attribute (SDO_AGGR_UNION) and looks up other names as lowercase attributes",
      "OracleSpatialAdapter extends WKTAdapter with input_size=oracledb.CLOB and fixes polygon ring orientation (outer CCW, inner CW)",
      "get_geometry_converter() uses wkb_r().read to convert Oracle WKB output to GEOSGeometryBase, setting srid to None when -1",
      "select attribute is 'SDO_UTIL.TO_WKBGEOMETRY(%s)' for selecting geometry as WKB",
      "convert_extent() handles both Polygon and Point geometry types from Oracle's SDO_AGGR_MBR",
      "get_area_att_for_field() always returns 'sq_m' unconditionally, unlike the base class conditional logic",
      "unsupported_functions is a cached_property that conditionally adds 'GeometryType' for Oracle versions below 23",
      "geo_quote_name() calls super().geo_quote_name(name).upper() \u2014 base wraps in quotes, Oracle uppercases",
      "DatabaseWrapper in the Oracle GIS backend sets ops_class=OracleOperations, features_class=DatabaseFeatures, SchemaEditorClass=OracleGISSchemaEditor",
      "SDORelate validates mask arguments against Oracle-specific masks (TOUCH, OVERLAPBDYDISJOINT, etc.) joined by +",
      "from_text is set to 'SDO_GEOMETRY' and DEFAULT_TOLERANCE is '0.05'"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/oracle/operations.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/oracle/adapter.py",
      "django/contrib/gis/db/backends/oracle/base.py",
      "django/contrib/gis/db/backends/utils.py",
      "django/db/backends/base/operations.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/oracle/operations.py] django.contrib.gis.db.backends.oracle.operations.OracleOperations\ndjango.contrib.gis.db.backends.oracle.operations.OracleOperations.geo_db_type:\n  Return the geometry database type for Oracle. Unlike other spatial\n        backends, no stored procedure is necessary and it's the same for all\n        geometry types.\n\ndjango.contrib.gis.db.backends.oracle.operations.OracleOperations.get_distance:\n  Return the distance parameters given the value and the lookup type.\n        On Oracle, geometry columns with a geodetic coordinate system behave\n        implicitly like a geography column, and thus meters will be used as\n        the distance parameter on them.\n\ndjango.contrib.gis.db.backends.oracle.operations.OracleOperations.spatial_aggregate_name:\n  Return the spatial aggregate SQL name.\n\ndjango.contrib.gis.db.backends.oracle.operations.OracleOperations.modify_insert_params:\n  Drop out insert parameters for NULL placeholder. Needed for Oracle\n        Spatial backend due to #10888.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: a comment about Java performance cost and a bug number reference; everything else is verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_55",
    "repo": "django",
    "question": "Where is `BaseSpatialOperations` used across the GIS backends, and where are its key methods like `geo_db_type` and `get_geom_placeholder_sql` called from?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `BaseSpatialOperations` class (in `django/contrib/gis/db/backends/base/operations.py`) serves as the foundational spatial operations mixin for all GIS database backends. It is called from two primary locations:\n\n1. **`geo_db_type()`** is called by `BaseSpatialField.db_type(connection)` at line 119 of `django/contrib/gis/db/models/fields.py`, which simply delegates to `connection.ops.geo_db_type(self)`. Each backend implements it differently:\n   - PostGIS returns `\"geometry(%s,%d)\"` or `\"geography(%s,%d)\"` (or `\"raster\"`)\n   - Oracle always returns `\"MDSYS.SDO_GEOMETRY\"`\n   - SpatiaLite returns `None`\n   - MySQL returns `f.geom_type` directly (e.g., \"POINT\", \"POLYGON\")\n\n2. **`get_geom_placeholder_sql()`** is called from two places:\n   - `BaseSpatialField.get_placeholder_sql()` at line 143 of `fields.py`\n   - `GISLookup.process_rhs()` at line 68 of `django/contrib/gis/db/models/lookups.py` (used during spatial lookups)\n\n3. **`get_distance()`** is called from `DistanceLookupBase.process_distance()` at line 326 of `lookups.py`, which is used by `DWithinLookup` and `DistanceLookupFromFunction` (for `distance_gt`, `distance_gte`, `distance_lt`, `distance_lte` lookups).\n\nThe base class's `get_geom_placeholder_sql` uses a `_must_transform_value` static method to check if `value.srid != field.srid`, and if transformation is needed, it wraps the SQL with `Transform(from_text(%s, %s), %s)`. The PostGIS override completely replaces this logic \u2014 it does NOT use `from_text` at all, instead using a simple `%s` placeholder that relies on the `PostGISAdapter`. The base class has a code path where if `has_spatialrefsys_table` is False, it omits the SRID parameter.\n\nThe `BaseSpatialOperations` class uses multiple inheritance in all backend implementations: e.g., `PostGISOperations(BaseSpatialOperations, DatabaseOperations)`, where `DatabaseOperations` is the respective backend's non-spatial operations class.",
    "rubric": [
      "Identifies that BaseSpatialOperations is defined in the base operations module and serves as the foundational spatial mixin for all GIS backends",
      "Mentions that geo_db_type() is called from BaseSpatialField.db_type(connection) which delegates to connection.ops.geo_db_type(self)",
      "Describes how different backends implement geo_db_type differently (e.g., PostGIS returns geometry/geography types, Oracle returns MDSYS.SDO_GEOMETRY, SpatiaLite returns None, MySQL returns f.geom_type)",
      "Identifies that get_geom_placeholder_sql() is called from BaseSpatialField.get_placeholder_sql() and GISLookup.process_rhs()",
      "Explains that get_distance() is called from DistanceLookupBase.process_distance() for distance-based lookups like dwithin and distance_gt/gte/lt/lte",
      "Notes that PostGIS overrides get_geom_placeholder_sql with its own logic using simple %s placeholders instead of from_text, relying on PostGISAdapter",
      "Mentions the base class's _must_transform_value static method that checks if value.srid != field.srid and wraps SQL with Transform if needed",
      "Notes that all backend implementations use multiple inheritance: e.g., PostGISOperations(BaseSpatialOperations, DatabaseOperations)"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/postgis/operations.py",
      "django/contrib/gis/db/models/lookups.py",
      "django/contrib/gis/db/models/fields.py",
      "django/contrib/gis/db/backends/oracle/operations.py",
      "django/contrib/gis/db/backends/mysql/operations.py",
      "django/contrib/gis/db/backends/spatialite/operations.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/base/operations.py] django.contrib.gis.db.backends.base.operations.BaseSpatialOperations\ndjango.contrib.gis.db.backends.base.operations.BaseSpatialOperations.geo_db_type:\n  Return the database column type for the geometry field on\n        the spatial backend.\n\ndjango.contrib.gis.db.backends.base.operations.BaseSpatialOperations.get_distance:\n  Return the distance parameters for the given geometry field,\n        lookup value, and lookup type.\n\ndjango.contrib.gis.db.backends.base.operations.BaseSpatialOperations.get_geom_placeholder_sql:\n  Return the placeholder for the given geometry field with the given\n        value. Depending on the spatial backend, the placeholder may contain a\n        stored procedure call to the transformation function of the spatial\n        backend.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (specific line numbers, backend-specific return values, method call chains, internal implementation details like _must_transform_value, PostGISAdapter, ST_GeomFromEWKB, etc.) that go far beyond what the documentation provides. These are plausible code-level details but cannot be verified from the given documentation alone.",
      "The documentation only describes three methods (geo_db_type, get_distance, get_geom_placeholder_sql) with brief docstrings. The answer extrapolates heavily into calling contexts, backend implementations, and internal logic that is not documented here.",
      "Claims about specific line numbers (e.g., line 119, line 143, line 68, line 326) cannot be verified from the documentation and could be inaccurate.",
      "The claim about SpatiaLite returning None because 'geometry columns are added via the AddGeometryColumn stored procedure' is not verifiable from the documentation.",
      "The claim about MySQL backend and issue #27464 is not verifiable from the documentation.",
      "No claims directly contradict the documentation - the three method descriptions are consistent with what the docstrings say."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in comments about MySQL backwards compatibility (#27464), the AddGeometryColumn rationale for SpatiaLite, PostGISAdapter's getquoted() internals, and SQLInsertCompiler reference.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_59",
    "repo": "django",
    "question": "Where is the SpatiaLite operations class defined, what does it inherit from, and how does it integrate with the database backend?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `SpatiaLiteOperations` class is defined in `django/contrib/gis/db/backends/spatialite/operations.py` and uses multiple inheritance from `BaseSpatialOperations` (in `django/contrib/gis/db/backends/base/operations.py`) and `DatabaseOperations` (in `django/db/backends/sqlite3/operations.py`). Here are key details discovered from the code:\n\n**Class hierarchy and location:**\n- `SpatiaLiteOperations` inherits from `BaseSpatialOperations` (first) and SQLite's `DatabaseOperations` (second).\n- It is registered as the `ops_class` in the `DatabaseWrapper` class in `django/contrib/gis/db/backends/spatialite/base.py`.\n\n**Key implementation details:**\n\n1. **`SpatialiteNullCheckOperator`**: A custom subclass of `SpatialOperator` defined in the same file that appends `\"> 0\"` to the SQL output. This is used for binary predicates like `equals`, `disjoint`, `touches`, `crosses`, `within`, `overlaps`, `contains`, `intersects`, `relate`, `coveredby`, `covers`, `same_as`, and `exact`. The bounding-box operators (`contained`, `bbcontains`, `bboverlaps`) use the plain `SpatialOperator` instead, mapping to `MbrWithin`, `MbrContains`, and `MbrOverlaps` respectively.\n\n2. **`spatial_version`** enforces a minimum SpatiaLite version of 4.3.0 \u2014 if the version is less than `(4, 3, 0)`, it raises `ImproperlyConfigured`.\n\n3. **`unsupported_functions`**: Always includes `{\"GeometryDistance\", \"MemSize\", \"Rotate\"}`. If `geom_lib_version()` returns a falsy value, `{\"Azimuth\", \"GeoHash\", \"MakeValid\"}` are also added. If `spatial_version < (5, 1)`, `{\"BoundingCircle\"}` is added.\n\n4. **`geom_lib_version()`** is version-dependent: for SpatiaLite >= 5.0 it calls `rttopo_version()` (which runs SQL `rttopo_version()`), otherwise it calls `lwgeom_version()` (which runs SQL `lwgeom_version()`).\n\n5. **`_get_spatialite_func()`** uses `self.connection._cursor()` directly (not `self.connection.cursor()`). The method explicitly closes the cursor in a `finally` block.\n\n6. **`convert_extent()`** receives polygon box data and parses it using `GEOSGeometry(box).shell`, extracting coordinates from indices 0 and 2 of the shell ring to form `(xmin, ymin, xmax, ymax)`.\n\n7. **`get_geometry_converter()`** uses `wkb_r().read` from `django/contrib/gis/geos/prototypes/io.py`. The converter wraps the result in `GEOSGeometryBase(read(value), geom_class)`.\n\n8. **`select`** attribute is set to `\"CAST (AsEWKB(%s) AS BLOB)\"` for geometry column selection, overriding the base class default of `\"%s\"`.\n\n9. **`function_names`** maps Django function names to SpatiaLite equivalents, including `\"ForcePolygonCW\": \"ST_ForceLHR\"`, `\"BoundingCircle\": \"GEOSMinimumBoundingCircle\"`, `\"Scale\": \"ScaleCoords\"`, and `\"LineLocatePoint\": \"ST_Line_Locate_Point\"`.\n\n10. **`disallowed_aggregates`** is set to `(models.Extent3D,)`, meaning 3D extent aggregation raises `NotSupportedError`.\n\n11. **`prepare_database()`** in the `DatabaseWrapper` uses `InitSpatialMetaData(1)` for SpatiaLite < 5.0 and `InitSpatialMetaDataFull(1)` for >= 5.0.\n\n12. **`DatabaseWrapper.__init__()`** tries multiple library paths: `settings.SPATIALITE_LIBRARY_PATH`, `\"mod_spatialite.so\"`, `\"mod_spatialite\"`, and `find_library(\"spatialite\")`, in that order.\n\n13. **`spatial_aggregate_name()`** replaces `\"union\"` with `\"unionagg\"` (which maps to `\"GUnion\"`) and lowercases the aggregate name before looking it up as an attribute.\n\n14. The `Adapter` is `SpatiaLiteAdapter` which inherits from `WKTAdapter` (in `django/contrib/gis/db/backends/base/adapter.py`) and implements `__conform__` for SQLite's `PrepareProtocol`, returning `str(self)`.\n\n15. **`BaseSpatialOperations.convert_extent()`** in the base class takes `(self, box, srid)` as parameters and raises `NotImplementedError`, while `SpatiaLiteOperations.convert_extent()` takes only `(self, box)` \u2014 a signature mismatch with the base class.",
    "rubric": [
      "Identifies that SpatiaLiteOperations is defined in django/contrib/gis/db/backends/spatialite/operations.py",
      "States that it inherits from BaseSpatialOperations and SQLite's DatabaseOperations (dual inheritance)",
      "Mentions it is registered as ops_class in the DatabaseWrapper class in django/contrib/gis/db/backends/spatialite/base.py",
      "Describes the SpatialiteNullCheckOperator that appends '> 0' to SQL for binary predicates",
      "Mentions the spatial_version property enforces a minimum SpatiaLite version of 4.3.0",
      "Notes the select attribute is set to 'CAST (AsEWKB(%s) AS BLOB)'",
      "Explains that geom_lib_version() calls rttopo_version() for SpatiaLite >= 5.0 or lwgeom_version() otherwise",
      "Mentions _get_spatialite_func uses connection._cursor() directly and explicitly closes it in a finally block",
      "Notes that convert_extent parses polygon box data using GEOSGeometry shell coordinates at indices 0 and 2",
      "Describes how DatabaseWrapper.__init__ tries multiple library paths including settings.SPATIALITE_LIBRARY_PATH, mod_spatialite.so, mod_spatialite, and find_library('spatialite')",
      "Mentions prepare_database uses InitSpatialMetaData(1) for < 5.0 and InitSpatialMetaDataFull(1) for >= 5.0"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/spatialite/operations.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/spatialite/base.py",
      "django/contrib/gis/db/backends/utils.py",
      "django/contrib/gis/db/backends/spatialite/adapter.py",
      "django/contrib/gis/db/backends/spatialite/features.py",
      "django/contrib/gis/db/backends/spatialite/models.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/spatialite/operations.py] django.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.spatial_version:\n  Determine the version of the SpatiaLite library.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.convert_extent:\n  Convert the polygon data received from SpatiaLite to min/max values.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.geo_db_type:\n  Return None because geometry columns are added via the\n        `AddGeometryColumn` stored procedure on SpatiaLite.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.get_distance:\n  Return the distance parameters for the given geometry field,\n        lookup value, and lookup type.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations._get_spatialite_func:\n  Helper routine for calling SpatiaLite functions and returning\n        their result.\n        Any error occurring in this method should be handled by the caller.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.geos_version:\n  Return the version of GEOS used by SpatiaLite as a string.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.proj_version:\n  Return the version of the PROJ library used by SpatiaLite.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.lwgeom_version:\n  Return the version of LWGEOM library used by SpatiaLite.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.rttopo_version:\n  Return the version of RTTOPO library used by SpatiaLite.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.geom_lib_version:\n  Return the version of the version-dependant geom library used by\n        SpatiaLite.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.spatialite_version:\n  Return the SpatiaLite library version as a string.\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.spatialite_version_tuple:\n  Return the SpatiaLite version as a tuple (version string, major,\n        minor, subminor).\n\ndjango.contrib.gis.db.backends.spatialite.operations.SpatiaLiteOperations.spatial_aggregate_name:\n  Return the spatial aggregate SQL template and function for the\n        given Aggregate instance.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many detailed implementation claims (line numbers, specific code patterns, class hierarchies, etc.) that go far beyond what the documentation provides. These cannot be verified from the documentation alone, but they are plausible code-level details.",
      "The claim about `geo_db_type` returning None is consistent with the documentation which states it returns None because geometry columns are added via AddGeometryColumn stored procedure.",
      "The claim about `convert_extent` converting polygon data to min/max values is consistent with the documentation, though the specific implementation detail about GEOSGeometry(box).shell and indices 0 and 2 cannot be verified from the doc.",
      "The claim about `spatial_version` enforcing minimum version 4.3.0 cannot be verified from the documentation, which only says it determines the version of the SpatiaLite library.",
      "The claim about `_get_spatialite_func` explicitly closing cursor in a finally block is consistent with the doc statement that 'Any error occurring in this method should be handled by the caller'.",
      "The claim about `spatial_aggregate_name` replacing 'union' with 'unionagg' is a specific implementation detail that goes beyond the doc's description of 'Return the spatial aggregate SQL template and function for the given Aggregate instance'.",
      "The documentation says `spatial_aggregate_name` returns 'the spatial aggregate SQL template and function for the given Aggregate instance' but the answer describes it as replacing strings and looking up attributes - these are implementation details that don't contradict but can't be verified."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; a few references to unseen files (BaseDatabaseWrapper internals, PostGIS implementation, aggregates.py) and one doc-dependent detail (WKT from WKTAdapter) were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_60",
    "repo": "django",
    "question": "How does BaseSpatialField handle value preparation, SRID resolution, and database adaptation for spatial lookups in Django's GIS framework?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `BaseSpatialField` class in `django/contrib/gis/db/models/fields.py` extends `django.db.models.Field` and serves as the base for both `GeometryField` and `RasterField`. Here's how it works in detail:\n\n**Initialization and SRID Handling:**\n- `__init__` accepts `verbose_name`, `srid` (default 4326), and `spatial_index` (default True). It stores `spatial_index` and `srid` as instance attributes, then passes `verbose_name` as a kwarg to `Field.__init__`.\n- `deconstruct()` always includes `srid` in kwargs, but only includes `spatial_index` if it's not `True` (the default).\n\n**SRID Caching System:**\n- A module-level `_srid_cache = defaultdict(dict)` stores SRID information per database alias. The `SRIDCacheEntry` is a namedtuple with fields `(units, units_name, spheroid, geodetic)`.\n- `get_srid_info(srid, connection)` first tries to get a `SpatialRefSys` model from `connection.ops.spatial_ref_sys()`. If `NotImplementedError` is raised, it falls back to using `gdal.SpatialReference` directly. The cache key is `connection.alias` when using the database, or `None` when using GDAL directly.\n- The `geodetic` value is determined by `srs.geographic`.\n- The `spheroid` is formatted as `'SPHEROID[\"name\",semi_major,inverse_flattening]'`.\n\n**`get_srid(obj)` Logic:**\n- Returns `self.srid` if: the object's SRID is `None`, the field's SRID is `-1`, or the object has SRID `-1` and the field doesn't. Otherwise returns the object's own SRID.\n\n**`get_prep_value` Pipeline:**\n1. If the value is already a `GEOSGeometry`, it passes through.\n2. Otherwise, it checks if the value is a \"candidate\" \u2014 meaning it's `bytes`, `str`, or has a `__geo_interface__` attribute.\n3. It first tries `get_raster_prep_value()` to convert to `GDALRaster`. For candidates, `GDALException` is silently caught. For `dict` values, `GDALException` raises a `ValueError`.\n4. If raster conversion fails and it's a candidate, it tries `GEOSGeometry(obj)`. Both `GEOSException` and `GDALException` are caught.\n5. Finally, `obj.srid` is set using `self.get_srid(obj)`.\n\n**`get_db_prep_value` and the Adapter Pattern:**\n- Calls `connection.ops.Adapter()` to wrap the value. Passes `geography=True` only if both `self.geography` is truthy AND `connection.features.supports_geography` is True.\n- Note: `geography` is NOT defined on `BaseSpatialField` itself \u2014 `GeometryField` sets it in `__init__` (default `False`), while `RasterField` hardcodes `geography = False` as a class attribute.\n- `PostGISAdapter` stores EWKB and uses `ST_GeomFromEWKB` or `ST_GeogFromWKB` depending on the `geography` flag.\n\n**`get_placeholder_sql` and Transformation:**\n- Delegates to `connection.ops.get_geom_placeholder_sql(self, value, compiler)`. The base implementation in `BaseSpatialOperations` inserts an `ST_Transform()` call when `value.srid != field.srid`.\n\n**Subclass Specifics:**\n- `GeometryField` adds `dim` (default 2), `geography` (default False), `_extent` (default `(-180.0, -90.0, 180.0, 90.0)`), `_tolerance` (default 0.05). It sets up a `SpatialProxy(geom_class or GEOSGeometry, self, load_func=GEOSGeometry)` descriptor in `contribute_to_class`. There are 7 concrete geometry subclasses (Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection), each setting a `geom_type`, `geom_class`, and `form_class`.\n- `RasterField` checks `connection.features.supports_raster` in `_check_connection()`, uses `SpatialProxy(gdal.GDALRaster, self)` without a `load_func`, and supports band index transforms via `get_transform()` which dynamically creates `SpecificRasterBandTransform` subclasses.\n\n**Lookup Registration:**\n- Over 20 spatial lookups are registered on `BaseSpatialField` using `@BaseSpatialField.register_lookup` in `lookups.py`. `SameAsLookup` is additionally registered with the `exact` lookup name. All spatial lookups extend `GISLookup`, which extends `Lookup`. `GISLookup.process_rhs()` wraps the RHS in `connection.ops.Adapter()` and calls `get_geom_placeholder_sql`.",
    "rubric": [
      "Explains that __init__ accepts srid (default 4326) and spatial_index (default True), storing them as instance attributes",
      "Describes the SRID caching system: module-level _srid_cache (defaultdict(dict)) with SRIDCacheEntry namedtuple storing units, units_name, spheroid, and geodetic",
      "Explains get_srid_info falls back from SpatialRefSys database model to gdal.SpatialReference when NotImplementedError is raised",
      "Describes get_srid logic: returns self.srid when obj.srid is None, field srid is -1, or obj has -1 and field doesn't; otherwise returns obj.srid",
      "Explains get_prep_value pipeline: passes GEOSGeometry through, checks candidates (bytes/str/__geo_interface__), tries raster conversion first, then GEOSGeometry construction, and sets srid via get_srid",
      "Describes get_raster_prep_value behavior: silently catches GDALException for candidates but raises ValueError for dict values",
      "Explains get_db_prep_value wraps value via connection.ops.Adapter, passing geography=True only when self.geography is truthy AND connection supports geography",
      "Notes that geography attribute is defined on GeometryField (default False) and RasterField (class attribute False), not on BaseSpatialField itself",
      "Describes how GeometryField and RasterField use SpatialProxy in contribute_to_class, with GeometryField passing load_func=GEOSGeometry",
      "Mentions that spatial lookups are registered on BaseSpatialField via register_lookup decorator, with GISLookup extending Lookup and using connection.ops.Adapter in process_rhs"
    ],
    "key_files": [
      "django/contrib/gis/db/models/fields.py",
      "django/contrib/gis/db/models/proxy.py",
      "django/contrib/gis/db/models/lookups.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/postgis/adapter.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/models/fields.py] django.contrib.gis.db.models.fields.BaseSpatialField\ndjango.contrib.gis.db.models.fields.BaseSpatialField:\n  The Base GIS Field.\n\n    It's used as a base class for GeometryField and RasterField. Defines\n    properties that are common to all GIS fields such as the characteristics\n    of the spatial reference system of the field.\n\ndjango.contrib.gis.db.models.fields.BaseSpatialField.__init__:\n  The initialization function for base spatial fields. Takes the\n        following as keyword arguments:\n\n        srid:\n         The spatial reference system identifier, an OGC standard.\n         Defaults to 4326 (WGS84).\n\n        spatial_index:\n         Indicates whether to create a spatial index. Defaults to True.\n         Set this instead of 'db_index' for geographic fields since index\n         creation is different for geometry columns.\n\ndjango.contrib.gis.db.models.fields.BaseSpatialField.geodetic:\n  Return true if this field's SRID corresponds with a coordinate\n        system that uses non-projected units (e.g., latitude/longitude).\n\ndjango.contrib.gis.db.models.fields.BaseSpatialField.get_placeholder_sql:\n  Return the placeholder for the spatial column for the\n        given value.\n\ndjango.contrib.gis.db.models.fields.BaseSpatialField.get_srid:\n  Return the default SRID for the given geometry or raster, taking into\n        account the SRID set for the field. For example, if the input geometry\n        or raster doesn't have an SRID, then the SRID of the field will be\n        returned.\n\ndjango.contrib.gis.db.models.fields.BaseSpatialField.get_raster_prep_value:\n  Return a GDALRaster if conversion is successful, otherwise return None.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many claims go far beyond what the documentation explicitly states (e.g., details about _srid_cache, SRIDCacheEntry namedtuple, Adapter patterns, specific backend adapters, lookup registration details, geometry subclass counts, etc.), but these appear to be plausible code-level details derived from reading the actual source code rather than contradicting the documentation.",
      "The claim that BaseSpatialField 'extends django.db.models.Field' is not stated in the documentation but is a reasonable inference from it being described as a base class for GeometryField and RasterField.",
      "The documentation says srid defaults to 4326 (WGS84) and spatial_index defaults to True \u2014 the answer correctly reflects both of these.",
      "The documentation says get_srid returns the field's SRID if the input doesn't have one \u2014 the answer's more detailed logic is consistent with this description.",
      "The documentation says get_raster_prep_value returns a GDALRaster if conversion is successful, otherwise None \u2014 the answer's pipeline description is consistent with this.",
      "The documentation says geodetic returns true for non-projected coordinate systems \u2014 the answer's description of how this works via GDAL is a plausible elaboration."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in rationale phrases ('for serialization stability') and claims about external code details (GDAL C API internals, backends not shown); most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_61",
    "repo": "django",
    "question": "What is the architecture of GeometryField, including its class hierarchy, proxy mechanism, SRID caching, and how subclasses specialize it?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `GeometryField` class in `django/contrib/gis/db/models/fields.py` is the base geometry field that maps to the OpenGIS Specification Geometry type. Here is a detailed architecture exploration:\n\n**Class Hierarchy:**\n- `GeometryField` extends `BaseSpatialField`, which extends `django.db.models.Field`.\n- `BaseSpatialField` also serves as the base for `RasterField`.\n- Seven specific geometry type subclasses extend `GeometryField`: `PointField`, `LineStringField`, `PolygonField`, `MultiPointField`, `MultiLineStringField`, `MultiPolygonField`, and `GeometryCollectionField`. Each sets its own `geom_type`, `geom_class`, and `form_class`.\n\n**Key Attributes on GeometryField:**\n- `geom_type = \"GEOMETRY\"`\n- `geom_class = None` (subclasses set this to specific GEOS classes like `Point`, `Polygon`, etc.)\n- `form_class = forms.GeometryField`\n- `geography` parameter (boolean, defaults to `False`)\n- `_extent` and `_tolerance` are stored as private attributes\n\n**Lazy Geometry via SpatialProxy (contribute_to_class):**\n`GeometryField.contribute_to_class()` sets up a `SpatialProxy` descriptor on the model class. The `SpatialProxy` class (in `proxy.py`) extends `DeferredAttribute` from `django.db.models.query_utils`. When `geom_class` is `None` (as in the base `GeometryField`), the proxy is initialized with `GEOSGeometry` as both the `_klass` (for isinstance checks) and `load_func`. For subclasses like `PointField`, `_klass` is set to the specific type (e.g., `Point`), but `load_func` is always `GEOSGeometry`.\n\n**select_format Method:**\nThe `select_format` method conditionally applies the backend's `select` string format. It only applies the format when the query is NOT a subquery (`not compiler.query.subquery`). The backend-specific `select` values are:\n- Base: `\"%s\"` (identity)\n- MySQL: `self.geom_func_prefix + \"AsBinary(%s)\"` which resolves to `\"ST_AsBinary(%s)\"`\n\n**SRID Caching:**\nA module-level `_srid_cache` (a `defaultdict(dict)`) caches SRID info keyed by `(connection alias, srid)`. Each entry is a `SRIDCacheEntry` namedtuple with fields `units`, `units_name`, `spheroid`, and `geodetic`. When `SpatialRefSys` is not available (e.g., `NotImplementedError`), it falls back to using `gdal.SpatialReference` directly, with `None` as the cache alias.\n\n**get_db_prep_value Geography Handling:**\n`BaseSpatialField.get_db_prep_value` wraps the value in the backend's `Adapter` class. If `self.geography` is `True` AND the backend supports geography (`connection.features.supports_geography`), it passes `geography=True` to the adapter. The `PostGISAdapter`, for example, uses this flag to call `ST_GeogFromWKB` instead of `ST_GeomFromEWKB` in its `getquoted()` method.\n\n**formfield Method:**\nWhen `dim > 2` and the form field's widget doesn't have `supports_3d = True`, it defaults the widget to `forms.Textarea`.\n\n**GeometryField does NOT define `from_db_value`** \u2014 database value conversion for geometry fields is handled lazily through the `SpatialProxy` descriptor, which calls `GEOSGeometry(geo_value)` on access.",
    "rubric": [
      "GeometryField extends BaseSpatialField, which extends django.db.models.Field",
      "Seven geometry subclasses (PointField, LineStringField, PolygonField, MultiPointField, MultiLineStringField, MultiPolygonField, GeometryCollectionField) each set geom_type, geom_class, and form_class",
      "GeometryField has geom_type='GEOMETRY', geom_class=None, and form_class=forms.GeometryField",
      "contribute_to_class sets up a SpatialProxy descriptor; when geom_class is None, GEOSGeometry is used as both _klass and load_func; for subclasses, _klass is the specific type but load_func is always GEOSGeometry",
      "SpatialProxy extends DeferredAttribute and lazily instantiates geometry objects on attribute access (no from_db_value defined on GeometryField)",
      "Module-level _srid_cache is a defaultdict(dict) keyed by (connection alias, srid), storing SRIDCacheEntry namedtuples with units, units_name, spheroid, and geodetic fields",
      "When SpatialRefSys is unavailable (NotImplementedError), the SRID cache falls back to using gdal.SpatialReference with None as the alias",
      "select_format only applies the backend's select format when the query is not a subquery",
      "get_db_prep_value passes geography=True to the backend Adapter when self.geography is True and the backend supports geography",
      "formfield defaults widget to forms.Textarea when dim > 2 and the widget doesn't have supports_3d=True"
    ],
    "key_files": [
      "django/contrib/gis/db/models/fields.py",
      "django/contrib/gis/db/models/proxy.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/db/backends/postgis/adapter.py",
      "django/contrib/gis/db/backends/mysql/operations.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/models/fields.py] django.contrib.gis.db.models.fields.GeometryField\ndjango.contrib.gis.db.models.fields.GeometryField:\n  The base Geometry field -- maps to the OpenGIS Specification Geometry type.\n\ndjango.contrib.gis.db.models.fields.GeometryField.__init__:\n  The initialization function for geometry fields. In addition to the\n        parameters from BaseSpatialField, it takes the following as keyword\n        arguments:\n\n        dim:\n         The number of dimensions for this geometry. Defaults to 2.\n\n        extent:\n         Customize the extent, in a 4-tuple of WGS 84 coordinates, for the\n         geometry field entry in the `USER_SDO_GEOM_METADATA` table. Defaults\n         to (-180.0, -90.0, 180.0, 90.0).\n\n        tolerance:\n         Define the tolerance, in meters, to use for the geometry field\n         entry in the `USER_SDO_GEOM_METADATA` table. Defaults to 0.05.\n\ndjango.contrib.gis.db.models.fields.GeometryField.select_format:\n  Return the selection format string, depending on the requirements\n        of the spatial backend. For example, Oracle and MySQL require custom\n        selection formats in order to retrieve geometries in OGC WKB.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc-leakage: Oracle/SpatiaLite/psycopg2 select values not in provided code, OpenLayersWidget default mentioned without code evidence, and some characterizations from docstrings.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_58",
    "repo": "django",
    "question": "Why does PostGISOperations use `cached_property` for `spatial_version` and what other performance-oriented design choices exist in that class?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `PostGISOperations.spatial_version` property uses `@cached_property` (from `django.utils.functional`) as a performance optimization to avoid repeated database queries for version detection. The `cached_property` decorator stores the result in `instance.__dict__` after the first access, so subsequent attribute accesses bypass the property and return the cached value directly. This is critical because `spatial_version` triggers `_get_postgis_func('version')` followed by `postgis_version_tuple()` \u2014 each of which executes SQL against the database via `temporary_connection()`.\n\nAdditionally, there are several other performance-motivated design decisions:\n\n1. **`POSTGIS_VERSION` settings shortcut**: The code explicitly checks `hasattr(settings, 'POSTGIS_VERSION')` before making any database query. This allows deployments to completely eliminate the version-detection query by providing the version via settings.\n\n2. **`select = \"%s\" if is_psycopg3 else \"%s::bytea\"`**: For psycopg2, geometry values use a `::bytea` cast, while psycopg3 does not require it.\n\n3. **`select_extent = None`**: PostGIS sets this to `None` (overriding the base class `@cached_property` which returns `self.select`). This means extent results are not wrapped with additional format casting.\n\n4. **`get_area_att_for_field` always returns `'sq_m'`**: The PostGIS override unconditionally returns `'sq_m'` instead of performing the base class's conditional checks (geodetic vs projected, feature support checks).\n\n5. **Thread-local WKB reader in `get_geometry_converter`**: The converter calls `wkb_r().read` which reuses a thread-local `_WKBReader` instance (from `ThreadLocalIO`) rather than creating a new reader for every geometry conversion. The reader is stored in `threading.local()` so a single instance is shared across all rows in a queryset within the same thread.\n\n6. **`@cached_property` on `function_names`**: The function name mapping dictionary is computed once and cached, avoiding repeated dictionary construction.",
    "rubric": [
      "Explains that `cached_property` avoids repeated database queries by caching the result after the first access (stored in instance.__dict__)",
      "Mentions that `spatial_version` triggers SQL queries via `_get_postgis_func` and `postgis_version_tuple()` using `temporary_connection()`",
      "Describes the `POSTGIS_VERSION` settings shortcut that lets deployments skip the version-detection database query entirely",
      "Notes that `select` uses `%s::bytea` cast for psycopg2 but plain `%s` for psycopg3",
      "Explains that `select_extent = None` overrides the base class's `@cached_property` that returns `self.select`",
      "Mentions that `get_area_att_for_field` unconditionally returns 'sq_m' instead of performing the base class's conditional geodetic/projected checks",
      "Explains that `wkb_r()` in `get_geometry_converter` returns a thread-local `_WKBReader` instance (via `ThreadLocalIO`/`threading.local`) reused across rows",
      "Notes that `function_names` also uses `@cached_property` to avoid repeated dictionary construction"
    ],
    "key_files": [
      "django/contrib/gis/db/backends/postgis/operations.py",
      "django/contrib/gis/db/backends/base/operations.py",
      "django/contrib/gis/geos/prototypes/io.py",
      "django/utils/functional.py",
      "django/db/backends/base/base.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/backends/postgis/operations.py] django.contrib.gis.db.backends.postgis.operations.PostGISOperations\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.spatial_version:\n  Determine the version of the PostGIS library.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.convert_extent:\n  Return a 4-tuple extent for the `Extent` aggregate by converting\n        the bounding box text returned by PostGIS (`box` argument), for\n        example: \"BOX(-90.0 30.0, -85.0 40.0)\".\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.convert_extent3d:\n  Return a 6-tuple extent for the `Extent3D` aggregate by converting\n        the 3d bounding-box text returned by PostGIS (`box3d` argument), for\n        example: \"BOX3D(-90.0 30.0 1, -85.0 40.0 2)\".\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.geo_db_type:\n  Return the database field type for the given spatial field.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.get_distance:\n  Retrieve the distance parameters for the given geometry field,\n        distance lookup value, and the distance lookup type.\n\n        This is the most complex implementation of the spatial backends due to\n        what is supported on geodetic geometry columns vs. what's available on\n        projected geometry columns. In addition, it has to take into account\n        the geography column type.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.get_geom_placeholder_sql:\n  Provide a proper substitution value for Geometries or rasters that are\n        not in the SRID of the field. Specifically, this routine will\n        substitute in the ST_Transform() function call.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations._get_postgis_func:\n  Helper routine for calling PostGIS functions and returning their\n        result.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_geos_version:\n  Return the version of the GEOS library used with PostGIS.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_lib_version:\n  Return the version number of the PostGIS library used with PostgreSQL.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_proj_version:\n  Return the version of the PROJ library used with PostGIS.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_version:\n  Return PostGIS version number and compile-time options.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_full_version:\n  Return PostGIS version number and compile-time options.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.postgis_version_tuple:\n  Return the PostGIS version as a tuple (version string, major,\n        minor, subminor).\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.proj_version_tuple:\n  Return the version of PROJ used by PostGIS as a tuple of the\n        major, minor, and subminor release numbers.\n\ndjango.contrib.gis.db.backends.postgis.operations.PostGISOperations.parse_raster:\n  Convert a PostGIS HEX String into a dict readable by GDALRaster.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific code patterns, threading behavior, psycopg3 handling) that go far beyond what the documentation provides. These cannot be verified from the documentation alone.",
      "The documentation describes `spatial_version` as 'Determine the version of the PostGIS library' but does not mention `@cached_property`, `_get_postgis_func('version')`, or `postgis_version_tuple()` being called internally.",
      "The documentation does not mention `POSTGIS_VERSION` settings, `temporary_connection()`, `select_extent`, `get_area_att_for_field`, thread-local WKB readers, or psycopg3 binary format handling.",
      "The claim about `_get_postgis_func` using `temporary_connection()` is not contradicted by the doc (which says it's a 'Helper routine for calling PostGIS functions and returning their result'), but the specific implementation details are unverifiable from the doc.",
      "No claims directly contradict the documentation - the answer's descriptions of `convert_extent` (4-tuple), `convert_extent3d` (6-tuple), `get_geom_placeholder_sql` (ST_Transform substitution), and `get_distance` (complex implementation for geodetic vs projected) align with the docstrings."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims relied on comment text or domain knowledge not present in stripped code; most claims about code structure and behavior were verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_65",
    "repo": "django",
    "question": "What does the GeoFeedMixin class do, and how do its subclasses and the Feed view integrate to emit geographic data in syndication feeds?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `GeoFeedMixin` class in `django/contrib/gis/feeds.py` is a mixin that provides methods to emit GeoRSS or W3C Geo XML elements. It is mixed into `SyndicationFeed` subclasses from `django.utils.feedgenerator` (specifically `Rss201rev2Feed` and `Atom1Feed`).\n\n**Key implementation details:**\n\n1. **`georss_coords(coords)`** swaps coordinate order using the format `\"%f %f\" % (coord[1], coord[0])` joined by spaces.\n\n2. **`add_georss_point(handler, coords, w3c_geo=False)`** has two code paths: when `w3c_geo=True`, it extracts `lon, lat = coords[:2]` and emits separate `geo:lat` and `geo:lon` elements. When `w3c_geo=False`, it wraps coords in a tuple and calls `georss_coords` to emit a single `georss:point` element.\n\n3. **`add_georss_element(handler, item, w3c_geo=False)`** retrieves the geometry from `item.get('geometry')` and supports multiple input forms:\n   - A 2-element flat tuple `(X, Y)` is treated as a point\n   - A 4-element flat tuple `(X0, Y0, X1, Y1)` is treated as a bounding box, reshaped into `(geom[:2], geom[2:])`\n   - A 2-element tuple of tuples `((X0, Y0), (X1, Y1))` is also treated as a box\n   - Geometry objects are dispatched by `geom.geom_type.lower()`: 'point', 'linestring'/'linearring' (emitting `georss:line`), or 'polygon' (emitting `georss:polygon` using only the exterior ring via `geom[0].coords`)\n   - W3C Geo mode raises `ValueError` for boxes and any non-point geometry types\n\n**Three concrete subclasses** use this mixin:\n- `GeoRSSFeed(Rss201rev2Feed, GeoFeedMixin)` \u2014 adds `xmlns:georss=\"http://www.georss.org/georss\"` namespace, calls `add_georss_element` without w3c_geo\n- `GeoAtom1Feed(Atom1Feed, GeoFeedMixin)` \u2014 same GeoRSS namespace but over Atom feeds, overrides `root_attributes()` instead of `rss_attributes()`\n- `W3CGeoFeed(Rss201rev2Feed, GeoFeedMixin)` \u2014 adds `xmlns:geo=\"http://www.w3.org/2003/01/geo/wgs84_pos#\"` namespace, passes `w3c_geo=True` to `add_georss_element`\n\n**Integration with the Feed view:** The `Feed` class in `django/contrib/gis/feeds.py` subclasses `django.contrib.syndication.views.Feed` (the `BaseFeed`), sets `feed_type = GeoRSSFeed` as default, and overrides `feed_extra_kwargs` and `item_extra_kwargs` to pass a `'geometry'` key. These use `self._get_dynamic_attr('geometry', obj)` and `self._get_dynamic_attr('item_geometry', item)` respectively, which is the mechanism from the base `Feed` view that checks `co_argcount` to determine whether to pass `obj` to the callable.\n\nThe `SyndicationFeed.add_item()` in `feedgenerator.py` accepts `**kwargs`, so the `geometry` key is stored directly in the item dictionary, making it accessible via `item.get('geometry')` in `add_georss_element`. Similarly, `SyndicationFeed.__init__` accepts `**kwargs` that get merged into `self.feed`, so feed-level geometry (from `feed_extra_kwargs`) also ends up in `self.feed` dict.\n\nBoth `add_item_elements` and `add_root_elements` are overridden in all three geo feed classes \u2014 the former adds geo elements to each item, and the latter adds geo elements to the feed/channel level (using `self.feed` as the item dict). The handler used is `SimplerXMLGenerator` from `django.utils.xmlutils`, which extends Python's `xml.sax.saxutils.XMLGenerator` with an `addQuickElement` convenience method.",
    "rubric": [
      "Explains that GeoFeedMixin provides methods for emitting GeoRSS or W3C Geo XML elements and is mixed into SyndicationFeed subclasses",
      "Describes georss_coords swapping coordinate order to lat/lon format using \"%f %f\" % (coord[1], coord[0])",
      "Explains add_georss_point has two paths: w3c_geo=True emits separate geo:lat and geo:lon elements, while False emits a single georss:point element",
      "Describes add_georss_element supporting multiple geometry input forms: 2-element tuple as point, 4-element tuple as box, tuple of tuples as box, and geometry objects dispatched by geom_type",
      "Notes that W3C Geo mode raises ValueError for boxes and non-point geometry types",
      "Identifies the three concrete subclasses: GeoRSSFeed (Rss201rev2Feed), GeoAtom1Feed (Atom1Feed), and W3CGeoFeed (Rss201rev2Feed with w3c_geo=True)",
      "Explains that the Feed view subclasses BaseFeed (django.contrib.syndication.views.Feed), sets feed_type=GeoRSSFeed, and overrides feed_extra_kwargs/item_extra_kwargs to pass geometry via _get_dynamic_attr",
      "Explains how geometry data flows from Feed kwargs into the item dictionary (via SyndicationFeed's add_item or __init__) making it accessible via item.get('geometry')"
    ],
    "key_files": [
      "django/contrib/gis/feeds.py",
      "django/utils/feedgenerator.py",
      "django/contrib/syndication/views.py",
      "django/utils/xmlutils.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/feeds.py] django.contrib.gis.feeds.GeoFeedMixin\ndjango.contrib.gis.feeds.GeoFeedMixin:\n  This mixin provides the necessary routines for SyndicationFeed subclasses\n    to produce simple GeoRSS or W3C Geo elements.\n\ndjango.contrib.gis.feeds.GeoFeedMixin.georss_coords:\n  In GeoRSS coordinate pairs are ordered by lat/lon and separated by\n        a single white space. Given a tuple of coordinates, return a string\n        GeoRSS representation.\n\ndjango.contrib.gis.feeds.GeoFeedMixin.add_georss_point:\n  Adds a GeoRSS point with the given coords using the given handler.\n        Handles the differences between simple GeoRSS and the more popular\n        W3C Geo specification.\n\ndjango.contrib.gis.feeds.GeoFeedMixin.add_georss_element:\n  Add a GeoRSS XML element using the given item and handler.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only minor phrasing about design purpose and GeoRSS standard motivation needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_64",
    "repo": "django",
    "question": "How does the SpatialProxy descriptor handle lazy loading and type validation for geometry and raster fields?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `SpatialProxy` class is a Python descriptor that inherits from `DeferredAttribute` (from `django.db.models.query_utils`) to handle spatial objects (geometries and rasters). Here's how it works in detail:\n\n**Initialization (`__init__`)**:\n- Takes `klass`, `field`, and an optional `load_func`.\n- `_klass` is used for `isinstance` checks in `__get__` and `__set__`, while `_load_func` is used to construct objects.\n- If `load_func` is not provided, it defaults to `klass` itself. This distinction is critical: for `GeometryField.contribute_to_class`, the proxy is created with `SpatialProxy(self.geom_class or GEOSGeometry, self, load_func=GEOSGeometry)`. This means for a `PointField`, `_klass` is `Point` (used for `isinstance` checks) but `_load_func` is always `GEOSGeometry` (used for actual object construction). For `RasterField`, both `_klass` and `_load_func` are `gdal.GDALRaster` since no `load_func` is passed.\n\n**`__get__` method**:\n1. Returns `self` if accessed on a class (not an instance).\n2. Tries to get the value from `instance.__dict__[self.field.attname]`.\n3. If the key doesn't exist (deferred field), falls back to `super().__get__()` which triggers `DeferredAttribute`'s deferred loading mechanism \u2014 this checks the parent chain and may trigger `instance._state.fetch_mode.fetch(self, instance)` to load from the database.\n4. If the value is already an instance of `_klass`, returns it directly.\n5. If the value is `None` or empty string, returns `None`.\n6. Otherwise, calls `self._load_func(geo_value)` to construct the geometry/raster object and caches it back into `instance.__dict__` via `setattr`.\n\n**`__set__` method**:\n1. Gets the field's `geom_type` to determine if it's a RASTER or geometry field.\n2. For RASTER fields: accepts `None`, `str`, `dict`, or `GDALRaster` instances.\n3. For geometry fields: if the value is an instance of `_klass` and has `srid=None`, assigns the field's SRID to the geometry. Also accepts `None`, `str`, or `memoryview`.\n4. Raises `TypeError` for any unrecognized value type.\n5. Stores the value directly in `instance.__dict__[self.field.attname]` and returns the value.\n\n**Key design decisions**:\n- The separation of `_klass` and `_load_func` allows `isinstance` checks to use specific geometry subclasses (e.g., `Point`) while always using the general `GEOSGeometry` constructor for loading.\n- The `__set__` method accepts `memoryview` for geometry fields.\n- The `__set__` method returns the value, which is unusual for Python descriptors.\n- The base `GeometryField` has `geom_class = None`, so the generic field uses `GEOSGeometry` as both `_klass` and `_load_func`.",
    "rubric": [
      "Explains that SpatialProxy inherits from DeferredAttribute and acts as a Python descriptor",
      "Describes the separation of _klass (for isinstance checks) and _load_func (for object construction), noting that load_func defaults to klass if not provided",
      "Explains that for GeometryField, _klass is the specific geometry class (e.g., Point) while _load_func is always GEOSGeometry, and for RasterField both are GDALRaster",
      "Describes __get__ behavior: returns self for class access, checks instance.__dict__ first, falls back to DeferredAttribute's deferred loading on KeyError, returns value directly if already _klass instance, returns None for None/empty string, otherwise calls _load_func and caches result via setattr",
      "Describes __set__ behavior: for RASTER fields accepts None/str/dict/_klass instances; for geometry fields accepts _klass (auto-assigning field SRID if geometry SRID is None), None, str, or memoryview; raises TypeError otherwise",
      "Notes that __set__ stores the value directly in instance.__dict__ and returns the value"
    ],
    "key_files": [
      "django/contrib/gis/db/models/proxy.py",
      "django/contrib/gis/db/models/fields.py",
      "django/db/models/query_utils.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/models/proxy.py] django.contrib.gis.db.models.proxy.SpatialProxy\ndjango.contrib.gis.db.models.proxy.SpatialProxy.__init__:\n  Initialize on the given Geometry or Raster class (not an instance)\n        and the corresponding field.\n\ndjango.contrib.gis.db.models.proxy.SpatialProxy.__get__:\n  Retrieve the geometry or raster, initializing it using the\n        corresponding class specified during initialization and the value of\n        the field. Currently, GEOS or OGR geometries as well as GDALRasters are\n        supported.\n\ndjango.contrib.gis.db.models.proxy.SpatialProxy.__set__:\n  Retrieve the proxied geometry or raster with the corresponding class\n        specified during initialization.\n\n        To set geometries, use values of None, HEXEWKB, or WKT.\n        To set rasters, use JSON or dict values.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states that __set__ accepts 'None, HEXEWKB, or WKT' for geometries, while the answer claims it also accepts 'memoryview' for WKB binary data. The answer acknowledges this discrepancy but presents the memoryview claim as factual without documentation support.",
      "The answer provides extensive implementation details (line numbers, specific method calls like 'instance._state.fetch_mode.fetch', inheritance from DeferredAttribute, etc.) that go well beyond what the documentation states. These appear to be derived from reading source code rather than the provided documentation.",
      "The documentation for __set__ says 'Retrieve the proxied geometry or raster' but the answer describes it as setting/storing values. This is consistent with what __set__ actually does (the docstring wording is slightly misleading), so this is not a contradiction per se.",
      "The claim that __set__ 'returns the value (line 84)' is a code-level detail not supported by the documentation but is plausible implementation knowledge."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Very minor doc leakage: a few purpose descriptions like 'WKB binary data' and 'can handle any format' came from documentation, but the vast majority of claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_62",
    "repo": "django",
    "question": "Why does `SQLiteDecimalToFloatMixin` convert Decimals to floats instead of relying on the existing `SQLiteNumericMixin` CAST approach?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `SQLiteDecimalToFloatMixin` exists because in Django's SQLite backend (`django/db/backends/sqlite3/base.py`), the adapter `Database.register_adapter(decimal.Decimal, str)` is registered, which converts all Python `Decimal` values to strings when binding parameters to SQLite queries. This breaks GIS/SpatiaLite functions like `Scale`, `SnapToGrid`, and `Translate` that expect numeric parameters.\n\nThe mixin's `as_sqlite` method works by creating a copy of the expression, iterating over all source expressions, and replacing any `Value` object whose `.value` is a `Decimal` with a new `Value(float(expr.value))`. This converts the Decimal to a native Python float before the sqlite3 adapter processes it, so the float passes directly to SQLite as a numeric value rather than being adapted to a string.\n\nNote that `SQLiteNumericMixin` in `django/db/models/expressions.py` adds `CAST(%s AS NUMERIC)` SQL wrapping for DecimalField values, but the `SQLiteDecimalToFloatMixin` takes a more direct approach by changing the Python value itself. The new `Value(float(...))` would auto-resolve its `output_field` to `FloatField` (via `Value._resolve_output_field`), completely avoiding the Decimal\u2192str adapter path. This is more reliable for SpatiaLite functions that may not correctly handle CAST-wrapped string parameters.\n\nThe mixin is used by three GIS functions: `Scale` and `SnapToGrid` directly inherit it, and `Translate` inherits it indirectly through `Scale`. All three accept `NUMERIC_TYPES = (int, float, Decimal)` parameters via `_handle_param`, which validates the type but returns the raw value. That raw value gets wrapped into a `Value` object by `BaseExpression._parse_expressions`, which wraps non-expression arguments with `Value(arg)` \u2014 this is where Decimal values enter the expression tree and would normally be string-adapted by SQLite's registered adapter.",
    "rubric": [
      "Explains that SQLite's backend registers `Database.register_adapter(decimal.Decimal, str)`, which converts Decimal values to strings at the parameter-binding level",
      "Explains that `SQLiteNumericMixin` wraps SQL output with `CAST(%s AS NUMERIC)` but this operates on the SQL/output side, not on the input parameter values, so it doesn't help GIS/SpatiaLite functions that receive Decimal parameters",
      "Explains that `SQLiteDecimalToFloatMixin.as_sqlite` replaces `Value` objects containing Decimals with `Value(float(...))`, converting at the Python level so the float bypasses the Decimal\u2192str adapter entirely",
      "Identifies the GIS functions that use the mixin: `Scale`, `SnapToGrid` (directly), and `Translate` (indirectly through `Scale`)",
      "Notes that these functions accept `Decimal` as a valid input type (via `NUMERIC_TYPES` and `_handle_param`), and raw values get wrapped into `Value` objects by the expression parsing machinery"
    ],
    "key_files": [
      "django/contrib/gis/db/models/functions.py",
      "django/db/backends/sqlite3/base.py",
      "django/db/models/expressions.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/models/functions.py] django.contrib.gis.db.models.functions.SQLiteDecimalToFloatMixin\ndjango.contrib.gis.db.models.functions.SQLiteDecimalToFloatMixin:\n  By default, Decimal values are converted to str by the SQLite backend,\n    which is not acceptable by the GIS functions expecting numeric values.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (specific line numbers, method behaviors, inheritance chains) that go far beyond what the documentation states. The documentation only provides a brief docstring explaining that Decimal values are converted to str by the SQLite backend, which is not acceptable by GIS functions expecting numeric values.",
      "Claims about specific line numbers (e.g., 'line 55 of django/db/backends/sqlite3/base.py', 'line 1193-1203', 'line 1222-1223', 'line 215-223') cannot be verified from the provided documentation and may be hallucinated or inaccurate.",
      "The claim that 'Translate inherits it indirectly through Scale' is a specific implementation detail that cannot be verified from the documentation provided.",
      "The detailed description of how as_sqlite method works (copying expression, iterating source expressions, replacing Value objects) cannot be verified from the provided documentation.",
      "The claim about Value._resolve_output_field auto-resolving to FloatField at specific lines cannot be verified."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with line numbers and some design rationale that is inferrable from code context rather than purely documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_63",
    "repo": "django",
    "question": "Where does the `SameAsLookup` get registered and how does it flow through to SQL generation across different spatial backends?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `SameAsLookup` class in `django/contrib/gis/db/models/lookups.py` inherits from `GISLookup` (which inherits from Django's `Lookup`) and is registered on `BaseSpatialField` under two lookup names: `\"same_as\"` (via the `@BaseSpatialField.register_lookup` decorator on line 178) and `\"exact\"` (via explicit call `BaseSpatialField.register_lookup(SameAsLookup, \"exact\")` on line 189). This double registration means that when users query a spatial field with `field=value` (which Django resolves to the `exact` lookup), it uses `SameAsLookup` instead of the default `Exact` lookup class registered on `Field`.\n\nThe data flow for SQL generation works as follows:\n1. `SameAsLookup.as_sql()` is inherited from `GISLookup.as_sql()` (line 78), which calls `process_lhs`, `process_rhs`, then `get_rhs_op(connection, rhs_sql)` (line 89).\n2. `get_rhs_op` (line 72-76) looks up the operator from `connection.ops.gis_operators[self.lookup_name]`, where `self.lookup_name` is always `\"same_as\"` regardless of whether the lookup was invoked via `__exact` or `__same_as`.\n3. The returned `SpatialOperator` object's `as_sql()` method generates the final SQL.\n\nThe operator differs per backend:\n- **PostGIS**: `PostGISOperator(op=\"~=\", raster=BILATERAL)` \u2014 generates SQL like `%(lhs)s ~= %(rhs)s`. PostGIS also has raster support with `BILATERAL` meaning mixed geometry-raster lookups trigger `ST_Polygon()` conversion.\n- **MySQL**: `SpatialOperator(func=\"ST_Equals\")` \u2014 generates `ST_Equals(%(lhs)s, %(rhs)s)`.\n- **Oracle**: `SDOOperator(func=\"SDO_EQUAL\")` \u2014 generates `SDO_EQUAL(%(lhs)s, %(rhs)s) = 'TRUE'` (via `SDOOperator.sql_template`).\n- **SpatiaLite**: `SpatialiteNullCheckOperator(func=\"Equals\")` \u2014 generates `Equals(%(lhs)s, %(rhs)s) > 0` (appends `> 0` to handle null checks).\n\nThe `SpatialOperator` base class (in `django/contrib/gis/db/backends/utils.py`) chooses between two default templates: `\"%(func)s(%(lhs)s, %(rhs)s)\"` when `func` is set, or `\"%(lhs)s %(op)s %(rhs)s\"` when only `op` is set. All backends define a separate `\"exact\"` key in their `gis_operators` dict with the same operator configuration, but this key is never actually accessed by `SameAsLookup` since its `lookup_name` attribute is `\"same_as\"`.",
    "rubric": [
      "Mentions SameAsLookup is registered under both 'same_as' (via decorator) and 'exact' (via explicit register_lookup call) on BaseSpatialField",
      "Explains that this double registration means the 'exact' lookup on spatial fields uses SameAsLookup instead of the default Exact lookup",
      "Describes that GISLookup.as_sql() calls get_rhs_op which looks up the operator from connection.ops.gis_operators using self.lookup_name ('same_as')",
      "Identifies that PostGIS uses PostGISOperator with op='~=' and raster=BILATERAL",
      "Identifies that MySQL uses SpatialOperator with func='ST_Equals'",
      "Identifies that Oracle uses SDOOperator with func='SDO_EQUAL' which appends = 'TRUE' via its sql_template",
      "Identifies that SpatiaLite uses SpatialiteNullCheckOperator with func='Equals' which appends '> 0'",
      "Explains SpatialOperator's default_template choosing between func-based and op-based templates"
    ],
    "key_files": [
      "django/contrib/gis/db/models/lookups.py",
      "django/contrib/gis/db/backends/utils.py",
      "django/contrib/gis/db/backends/postgis/operations.py",
      "django/contrib/gis/db/backends/spatialite/operations.py",
      "django/contrib/gis/db/backends/oracle/operations.py",
      "django/contrib/gis/db/backends/mysql/operations.py",
      "django/db/models/query_utils.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/db/models/lookups.py] django.contrib.gis.db.models.lookups.SameAsLookup\ndjango.contrib.gis.db.models.lookups.SameAsLookup:\n  The \"~=\" operator is the \"same as\" operator. It tests actual geometric\n    equality of two features. So if A and B are the same feature,\n    vertex-by-vertex, the operator returns true.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone; no documentation-dependent claims found.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_68",
    "repo": "django",
    "question": "How does the GDAL Envelope class handle initialization, coordinate validation, and the expand_to_include algorithm?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `Envelope` class in `django/contrib/gis/gdal/envelope.py` implements a bounding box wrapper around a C-level `OGREnvelope` ctypes `Structure`. Here's how the implementation works in detail:\n\n**OGREnvelope C Structure:**\nThe underlying `OGREnvelope` class (defined in the same file) is a `ctypes.Structure` with four `c_double` fields: `MinX`, `MaxX`, `MinY`, `MaxY`. Note that the field order is MinX, MaxX, MinY, MaxY (not min_x, min_y, max_x, max_y).\n\n**Initialization (`__init__`):**\nAccepts three forms of input:\n1. A single `OGREnvelope` instance \u2014 stored directly as `self._envelope`\n2. A single 4-element tuple/list \u2014 passed to `_from_sequence()` which creates a new `OGREnvelope()` and assigns elements at indices 0,1,2,3 to `MinX`, `MinY`, `MaxX`, `MaxY` respectively\n3. Four individual arguments \u2014 converted to floats via `[float(a) for a in args]` then passed to `_from_sequence()`\n\nAfter initialization, the constructor validates that `min_x <= max_x` and `min_y <= max_y`, raising `GDALException` if not. Equal values (e.g., `Envelope(0,0,0,0)`) are allowed.\n\n**Sequence mapping:** The `_from_sequence` method maps index 0\u2192MinX, 1\u2192MinY, 2\u2192MaxX, 3\u2192MaxY. This means the tuple order is `(min_x, min_y, max_x, max_y)` while the C structure field order is `(MinX, MaxX, MinY, MaxY)` \u2014 an important distinction.\n\n**`expand_to_include` algorithm:**\nThis method uses recursive delegation to normalize various input formats into a single 4-tuple case:\n- An `Envelope` argument \u2192 recurses with `args[0].tuple`\n- An object with `.x` and `.y` attributes \u2192 recurses with `(x, y, x, y)` (4 individual args)\n- A 2-tuple \u2192 recurses with `(x, y, x, y)` as a 4-tuple\n- Two individual args \u2192 recurses with `(args[0], args[1], args[0], args[1])` as a tuple\n- Four individual args \u2192 recurses with `args` as a tuple\n- A 4-tuple is the terminal case: directly compares and updates each of the four `_envelope` fields using `<` for min values and `>` for max values\n\n**WKT generation:**\nThe `wkt` property generates a `POLYGON` WKT string with 5 points tracing the bounding box counter-clockwise starting and ending at the lower-left corner: `(min_x,min_y) \u2192 (min_x,max_y) \u2192 (max_x,max_y) \u2192 (max_x,min_y) \u2192 (min_x,min_y)`.\n\n**`__eq__` comparison:**\nCompares against another `Envelope` or a 4-tuple (ordered as min_x, min_y, max_x, max_y). For any other type, it raises a `GDALException` rather than returning `NotImplemented`.\n\n**Integration with GDAL layer:**\nIn `geometries.py`, `OGRGeometry.envelope` creates an `OGREnvelope()`, passes it by reference to `capi.get_envelope` (which calls `OGR_G_GetEnvelope` via ctypes), and wraps the result in an `Envelope`. The `check_envelope` error-checking function in `prototypes/errcheck.py` simply extracts and returns the by-reference pointer argument (`ptr_byref(cargs, offset)`) without any error code checking. Similarly, `Layer.extent` creates an `OGREnvelope`, calls `capi.get_extent` with `force=1`, and wraps the result.\n\nThe `env_func` in `prototypes/geom.py` configures the ctypes function prototype with `restype=None` and `errcheck=check_envelope`, meaning the return value is discarded and the envelope is extracted from the by-reference argument.",
    "rubric": [
      "Explains that OGREnvelope is a ctypes Structure with four c_double fields in the order MinX, MaxX, MinY, MaxY",
      "Describes three initialization forms: OGREnvelope instance, 4-element tuple/list, or 4 individual arguments",
      "Explains that _from_sequence maps index 0\u2192MinX, 1\u2192MinY, 2\u2192MaxX, 3\u2192MaxY (tuple order is min_x, min_y, max_x, max_y, different from C struct field order)",
      "Notes that __init__ validates min_x <= max_x and min_y <= max_y, raising GDALException if violated",
      "Describes expand_to_include's recursive delegation: Envelope\u2192tuple, point/2-tuple/2-args\u2192(x,y,x,y), 4-args\u2192tuple, with 4-tuple as terminal case that compares and updates fields",
      "Explains that the terminal 4-tuple case uses < for minimums and > for maximums to expand the envelope",
      "Mentions that wkt property generates a POLYGON with 5 counter-clockwise vertices starting/ending at lower-left"
    ],
    "key_files": [
      "django/contrib/gis/gdal/envelope.py",
      "django/contrib/gis/gdal/prototypes/geom.py",
      "django/contrib/gis/gdal/prototypes/errcheck.py",
      "django/contrib/gis/gdal/geometries.py",
      "django/contrib/gis/gdal/layer.py",
      "tests/gis_tests/gdal_tests/test_envelope.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/envelope.py] django.contrib.gis.gdal.envelope.Envelope\ndjango.contrib.gis.gdal.envelope.Envelope:\n  The Envelope object is a C structure that contains the minimum and\n    maximum X, Y coordinates for a rectangle bounding box. The naming\n    of the variables is compatible with the OGR Envelope structure.\n\ndjango.contrib.gis.gdal.envelope.Envelope.__init__:\n  The initialization function may take an OGREnvelope structure,\n        4-element tuple or list, or 4 individual arguments.\n\ndjango.contrib.gis.gdal.envelope.Envelope.__eq__:\n  Return True if the envelopes are equivalent; can compare against\n        other Envelopes and 4-tuples.\n\ndjango.contrib.gis.gdal.envelope.Envelope.__str__:\n  Return a string representation of the tuple.\n\ndjango.contrib.gis.gdal.envelope.Envelope._from_sequence:\n  Initialize the C OGR Envelope structure from the given sequence.\n\ndjango.contrib.gis.gdal.envelope.Envelope.expand_to_include:\n  Modify the envelope to expand to include the boundaries of\n        the passed-in 2-tuple (a point), 4-tuple (an extent) or\n        envelope.\n\ndjango.contrib.gis.gdal.envelope.Envelope.min_x:\n  Return the value of the minimum X coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.min_y:\n  Return the value of the minimum Y coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.max_x:\n  Return the value of the maximum X coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.max_y:\n  Return the value of the maximum Y coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.ur:\n  Return the upper-right coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.ll:\n  Return the lower-left coordinate.\n\ndjango.contrib.gis.gdal.envelope.Envelope.tuple:\n  Return a tuple representing the envelope.\n\ndjango.contrib.gis.gdal.envelope.Envelope.wkt:\n  Return WKT representing a Polygon for this envelope.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.045,
    "strip_verify_summary": "Only one claim (a TODO comment reference) was removed as it relies on stripped documentation; all other claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_66",
    "repo": "django",
    "question": "Why does GeometryField override so many behaviors from the base Field class \u2014 like deserialization, SRID handling, type checking, and change detection?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `GeometryField` in `django/contrib/gis/forms/fields.py` makes several deliberate design decisions:\n\n**1. Widget-first deserialization pattern (`to_python`):** The `to_python` method first checks if the widget has a `deserialize` method (`if hasattr(self.widget, \"deserialize\")`), and only falls back to direct `GEOSGeometry` construction if it doesn't. This is a design decision that lets widgets control how input is parsed. The `OpenLayersWidget.deserialize()` method (in `widgets.py`) overrides the base `BaseGeometryWidget.deserialize()` to handle GeoJSON specially: when input matches `json_regex` and `map_srid != 4326`, it overrides the SRID to the map's SRID (3857 for OpenLayersWidget).\n\n**2. Defensive SRID fallback chain (`to_python`):** After geometry creation, if the geometry has no SRID, the code tries `self.widget.map_srid` first (catching `AttributeError` in case the widget doesn't have that attribute), then falls back to `self.srid`. This means widget-level SRID configuration takes precedence over field-level SRID configuration as a default.\n\n**3. Geometry type validation uses OGC string labels (`clean`):** The `clean` method compares `str(geom.geom_type).upper()` against `self.geom_type`. The special case `self.geom_type != \"GEOMETRY\"` acts as a wildcard \u2014 when `geom_type` is `\"GEOMETRY\"` (the default), any geometry type is accepted. This is why subclasses like `PointField`, `PolygonField`, etc. only need to override the `geom_type` class attribute.\n\n**4. SRID=-1 bypass in transform (`clean`):** The condition `self.srid and self.srid != -1 and self.srid != geom.srid` explicitly skips transformation when SRID is -1, treating -1 as a sentinel value meaning unknown/unset SRID.\n\n**5. Tolerance-based geographic comparison (`has_changed`):** The `has_changed` method uses `initial.equals_exact(data, tolerance=0.000001)` rather than simple equality. This tolerance-based approach accounts for precision loss during serialization/deserialization cycles. The parent `forms.Field.has_changed` simply uses `!=` comparison after coercion, but GeometryField completely overrides this to also transform `data` to `initial`'s SRID before comparison.\n\n**6. Both initial and data are converted in `has_changed`:** Unlike the parent class `forms.Field.has_changed` which only calls `to_python(data)`, the GIS `GeometryField.has_changed` calls `to_python()` on BOTH `initial` and `data`, ensuring both are proper GEOSGeometry objects before geographic comparison. It also transforms `data` to match `initial`'s SRID before comparison.\n\n**7. Seven geometry-specific subclasses:** The module defines `GeometryCollectionField`, `PointField`, `MultiPointField`, `LineStringField`, `MultiLineStringField`, `PolygonField`, and `MultiPolygonField` \u2014 each only overriding `geom_type`.\n\n**8. OpenLayersWidget uses SRID 3857 by default**, while `BaseGeometryWidget` uses 4326. This means the default form field widget operates in a different projection than the base widget.",
    "rubric": [
      "Explains the widget-first deserialization pattern in to_python: checks for widget.deserialize before falling back to direct GEOSGeometry construction, allowing widgets like OpenLayersWidget to control parsing (e.g., overriding SRID for GeoJSON with map_srid 3857)",
      "Describes the defensive SRID fallback chain: widget.map_srid is tried first (with AttributeError caught), then self.srid, giving widget-level SRID priority over field-level SRID",
      "Explains the GEOMETRY wildcard in clean: when geom_type is 'GEOMETRY' any geometry type is accepted, and subclasses (PointField, PolygonField, etc.) only need to override geom_type",
      "Explains the SRID=-1 sentinel bypass in clean's transform logic, treating -1 as unknown/unset SRID to skip transformation",
      "Explains why has_changed uses equals_exact with tolerance=0.000001 instead of simple equality, to account for floating-point precision loss during serialization/deserialization cycles",
      "Notes that has_changed calls to_python on both initial and data (unlike the parent Field which only calls to_python on data), and transforms data to initial's SRID before comparison",
      "Mentions the seven geometry-specific subclasses that each only override the geom_type attribute",
      "Notes that OpenLayersWidget defaults to SRID 3857 while BaseGeometryWidget uses 4326"
    ],
    "key_files": [
      "django/contrib/gis/forms/fields.py",
      "django/contrib/gis/forms/widgets.py",
      "django/contrib/gis/geos/geometry.py",
      "django/forms/fields.py",
      "django/contrib/gis/geometry.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/forms/fields.py] django.contrib.gis.forms.fields.GeometryField\ndjango.contrib.gis.forms.fields.GeometryField:\n  This is the basic form field for a Geometry. Any textual input that is\n    accepted by GEOSGeometry is accepted by this form. By default,\n    this includes WKT, HEXEWKB, WKB (in a buffer), and GeoJSON.\n\ndjango.contrib.gis.forms.fields.GeometryField.to_python:\n  Transform the value to a Geometry object.\n\ndjango.contrib.gis.forms.fields.GeometryField.clean:\n  Validate that the input value can be converted to a Geometry object\n        and return it. Raise a ValidationError if the value cannot be\n        instantiated as a Geometry.\n\ndjango.contrib.gis.forms.fields.GeometryField.has_changed:\n  Compare geographic value of data with its initial value.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in a comment reference and some domain knowledge parentheticals (PostGIS convention, projection names).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_67",
    "repo": "django",
    "question": "Where is the GDAL C library loaded and how does the class hierarchy for GDAL-wrapping objects connect back to C pointer management?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `django/contrib/gis/gdal/__init__.py` module imports and re-exports the key components of Django's GDAL/OGR ctypes interface. Here is where the key components are located and how they connect:\n\n**Class hierarchy:** GDAL-wrapping classes such as `SpatialReference`, `CoordTransform`, and `Driver` inherit from `GDALBase` (defined in `django/contrib/gis/gdal/base.py`), which itself inherits from `CPointerBase` (defined in `django/contrib/gis/ptr.py`). `CPointerBase` manages the C pointer lifecycle via a `ptr` property that raises `null_ptr_exception_class` (set to `GDALException` in `GDALBase`) when accessing a NULL pointer, and a `__del__` method that calls the subclass's `destructor` to free C memory.\n\n**GDAL library loading** happens in `django/contrib/gis/gdal/libgdal.py`. The library is loaded via `ctypes.CDLL`. On Windows (`os.name == 'nt'`), the library is *also* loaded as `WinDLL` into a `lwingdal` variable. The `std_call()` function returns the appropriate library handle. The module supports GDAL versions 3.3 through 3.12. On POSIX, lib names tried include `\"gdal\"`, `\"GDAL\"`, and versioned names like `\"gdal3.12.0\"` down to `\"gdal3.3.0\"`. On Windows, names range from `\"gdal312\"` to `\"gdal303\"`. A custom `CPLErrorHandler` is set up using `CFUNCTYPE(None, c_int, c_int, c_char_p)` that logs GDAL errors to the `\"django.contrib.gis\"` logger.\n\n**The `Envelope` class** (in `envelope.py`) does NOT inherit from `GDALBase`; it wraps a ctypes `Structure` subclass called `OGREnvelope` with four `c_double` fields: `MinX`, `MaxX`, `MinY`, `MaxY`.\n\n**The `__all__` tuple** exports `AxisOrder` (an `IntEnum` with values `TRADITIONAL=0` and `AUTHORITY=1`), `GDALRaster`, `check_err`, `gdal_version`, and `gdal_full_version`, among other items.\n\n**Prototype generation** lives in `django/contrib/gis/gdal/prototypes/generation.py`, which defines factory functions like `srs_output`, `void_output`, `voidptr_output`, `string_output`, etc. These set `argtypes`, `restype`, and `errcheck` on ctypes function objects. The `errcheck` callbacks are defined in `prototypes/errcheck.py` \u2014 for example, `check_string` calls `lgdal.VSIFree` to free GDAL-allocated string pointers, while `check_const_string` does NOT free the pointer.\n\n**The `Driver` class** maintains a `_alias` dictionary mapping shorthand names (e.g., `\"shp\"` \u2192 `\"ESRI Shapefile\"`, `\"tif\"` \u2192 `\"GTiff\"`), with TIGER aliases conditionally added only for GDAL versions \u2264 3.10.\n\n**CoordTransform** (in `srs.py`) stores the names of the source and target SRS as `_srs1_name` and `_srs2_name`, and its `__str__` returns `'Transform from \"<source>\" to \"<target>\"'`. Its destructor is `capi.destroy_ct`.",
    "rubric": [
      "Identifies that the GDAL library is loaded in libgdal.py using ctypes.CDLL (and WinDLL on Windows into lwingdal)",
      "Explains that GDALBase (in base.py) inherits from CPointerBase (in ptr.py)",
      "Describes that CPointerBase manages the C pointer via a ptr property that raises null_ptr_exception_class on NULL access",
      "Mentions that CPointerBase.__del__ calls the subclass's destructor to free C memory",
      "Notes that GDALBase sets null_ptr_exception_class to GDALException",
      "Mentions std_call() returns lwingdal on Windows or lgdal on POSIX",
      "Identifies that POSIX library names tried include 'gdal', 'GDAL', and versioned names from gdal3.12.0 down to gdal3.3.0",
      "Notes the CPLErrorHandler setup using CFUNCTYPE(None, c_int, c_int, c_char_p) that logs to 'django.contrib.gis' logger"
    ],
    "key_files": [
      "django/contrib/gis/gdal/__init__.py",
      "django/contrib/gis/gdal/libgdal.py",
      "django/contrib/gis/gdal/base.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/srs.py",
      "django/contrib/gis/gdal/envelope.py",
      "django/contrib/gis/gdal/driver.py",
      "django/contrib/gis/gdal/error.py",
      "django/contrib/gis/gdal/prototypes/generation.py",
      "django/contrib/gis/gdal/prototypes/errcheck.py",
      "django/contrib/gis/gdal/geometries.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/__init__.py] django.contrib.gis.gdal\nThis module houses ctypes interfaces for GDAL objects. The following GDAL\nobjects are supported:\n\nCoordTransform: Used for coordinate transformations from one spatial\n reference system to another.\n\nDriver: Wraps an OGR data source driver.\n\nDataSource: Wrapper for the OGR data source object, supports\n OGR-supported data sources.\n\nEnvelope: A ctypes structure for bounding boxes (GDAL library\n not required).\n\nOGRGeometry: Object for accessing OGR Geometry functionality.\n\nOGRGeomType: A class for representing the different OGR Geometry\n types (GDAL library not required).\n\nSpatialReference: Represents OSR Spatial Reference objects.\n\nThe GDAL library will be imported from the system path using the default\nlibrary name for the current OS. The default library path may be overridden\nby setting `GDAL_LIBRARY_PATH` in your settings with the path to the GDAL C\nlibrary on your system.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rely on unseen files (geometries.py, datasource.py) or documentation-only rationale.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_69",
    "repo": "django",
    "question": "What are the full inheritance chain, destructor mechanism, and key class dependencies involved in the OGR Feature wrapper in Django's GDAL module?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `Feature` class in `django/contrib/gis/gdal/feature.py` wraps an OGR Feature and inherits from `GDALBase`, which itself inherits from `CPointerBase` (in `django/contrib/gis/ptr.py`). The class relies on a chain of dependencies involving the Layer, DataSource, and Field classes, all connected through C pointer management.\n\n**Inheritance chain:** `Feature` \u2192 `GDALBase` \u2192 `CPointerBase`. `GDALBase` sets `null_ptr_exception_class = GDALException`, overriding `CPointerBase`'s default of `AttributeError`.\n\n**Destructor:** The `Feature` class sets `destructor = capi.destroy_feature`, which wraps the GDAL C function `OGR_F_Destroy` using `void_output` with `errcheck=False`.\n\n**Key dependencies:**\n1. The `encoding` property on `Feature` traverses `self._layer._ds.encoding`, going from Feature \u2192 Layer \u2192 DataSource. The `DataSource.__init__` defaults encoding to `\"utf-8\"`.\n2. The `__getitem__` method creates a `Field` instance, which dynamically reassigns its own class based on the OGR field type via `self.__class__ = OGRFieldTypes[self.type]` in `Field.__init__`. The `OGRFieldTypes` dictionary maps integer type codes (0-13) to subclasses like `OFTInteger`, `OFTReal`, `OFTString`, etc.\n3. The `geom` property clones the geometry pointer using `geom_api.clone_geom(geom_ptr)` before wrapping it in `OGRGeometry`, meaning the returned geometry is independent of the feature's lifecycle.\n4. The `__eq__` method uses `capi.feature_equal` which wraps `OGR_F_Equal` and returns an integer (via `int_output`), which is then wrapped in `bool()`.\n5. `CPointerBase.__del__` handles cleanup, calling `self.destructor(self.ptr)` while catching `AttributeError`, `ImportError`, and `TypeError` exceptions.\n6. The Layer stores its own `_ldefn` (layer definition pointer) via `capi.get_layer_defn(self._ptr)` during `__init__`, and the Feature uses `self._layer._ldefn` for operations like `get_feat_name`, `get_field_defn`, and `get_fd_geom_type`.\n7. The `Field.__init__` uses `capi.get_feat_field_defn` (wrapping `OGR_F_GetFieldDefnRef`) rather than layer-level field definition, meaning it gets the field definition from the feature itself.\n8. `Feature.index()` uses `force_bytes` from `django.utils.encoding` to convert the field name before calling `capi.get_field_index` (wrapping `OGR_F_GetFieldIndex`), and raises `IndexError` if the result is negative.",
    "rubric": [
      "Identifies the inheritance chain: Feature \u2192 GDALBase \u2192 CPointerBase, with GDALBase overriding null_ptr_exception_class to GDALException",
      "Explains that the destructor is capi.destroy_feature wrapping OGR_F_Destroy via void_output with errcheck=False",
      "Describes CPointerBase.__del__ cleanup logic: calls self.destructor(self.ptr) and catches AttributeError, ImportError, TypeError",
      "Explains the encoding property traversal: Feature._layer._ds.encoding, with DataSource defaulting encoding to 'utf-8'",
      "Describes Field.__init__ dynamically reassigning its class via self.__class__ = OGRFieldTypes[self.type], mapping integer type codes to subclasses",
      "Notes that the geom property clones the geometry pointer (via geom_api.clone_geom) before creating OGRGeometry, making it independent of feature lifecycle",
      "Explains that __eq__ uses capi.feature_equal (OGR_F_Equal) returning int_output, wrapped in bool()",
      "Mentions that Layer stores _ldefn via capi.get_layer_defn and Feature uses self._layer._ldefn for operations like get_feat_name and get_fd_geom_type",
      "Notes that Field uses capi.get_feat_field_defn (OGR_F_GetFieldDefnRef) to get field definition from the feature itself, not the layer",
      "Describes Feature.index() using force_bytes on field name before calling capi.get_field_index, raising IndexError if result is negative"
    ],
    "key_files": [
      "django/contrib/gis/gdal/feature.py",
      "django/contrib/gis/gdal/field.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/prototypes/ds.py",
      "django/contrib/gis/gdal/base.py",
      "django/contrib/gis/gdal/layer.py",
      "django/contrib/gis/gdal/datasource.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/feature.py] django.contrib.gis.gdal.feature.Feature\ndjango.contrib.gis.gdal.feature.Feature:\n  This class that wraps an OGR Feature, needs to be instantiated\n    from a Layer object.\n\ndjango.contrib.gis.gdal.feature.Feature.__init__:\n  Initialize Feature from a pointer and its Layer object.\n\ndjango.contrib.gis.gdal.feature.Feature.__getitem__:\n  Get the Field object at the specified index, which may be either\n        an integer or the Field's string label. Note that the Field object\n        is not the field's _value_ -- use the `get` method instead to\n        retrieve the value (e.g. an integer) instead of a Field instance.\n\ndjango.contrib.gis.gdal.feature.Feature.__len__:\n  Return the count of fields in this feature.\n\ndjango.contrib.gis.gdal.feature.Feature.__str__:\n  The string name of the feature.\n\ndjango.contrib.gis.gdal.feature.Feature.__eq__:\n  Do equivalence testing on the features.\n\ndjango.contrib.gis.gdal.feature.Feature.fid:\n  Return the feature identifier.\n\ndjango.contrib.gis.gdal.feature.Feature.layer_name:\n  Return the name of the layer for the feature.\n\ndjango.contrib.gis.gdal.feature.Feature.num_fields:\n  Return the number of fields in the Feature.\n\ndjango.contrib.gis.gdal.feature.Feature.fields:\n  Return a list of fields in the Feature.\n\ndjango.contrib.gis.gdal.feature.Feature.geom:\n  Return the OGR Geometry for this Feature.\n\ndjango.contrib.gis.gdal.feature.Feature.geom_type:\n  Return the OGR Geometry Type for this Feature.\n\ndjango.contrib.gis.gdal.feature.Feature.get:\n  Return the value of the field, instead of an instance of the Field\n        object. May take a string of the field name or a Field object as\n        parameters.\n\ndjango.contrib.gis.gdal.feature.Feature.index:\n  Return the index of the given field name.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only one minor design rationale phrase about garbage collection was removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_70",
    "repo": "django",
    "question": "Why does the GDAL Field class use `__class__` mutation, early `is_set` checks, and no destructor \u2014 what performance considerations drive these design choices?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The Field class uses several performance-oriented design decisions:\n\n1. **Early null-checking via `is_set` to avoid unnecessary C API calls**: Every value retrieval method (`as_double`, `as_int`, `as_string`, `as_datetime`) checks `self.is_set` before making the actual GDAL C API call. This avoids processing return values when the field has no meaningful data. The `is_set` property maps to `OGR_F_IsFieldSetAndNotNull` (via `bool_output`), which is a simple boolean check performed before calling the full field-value retrieval functions.\n\n2. **`__class__` mutation in `__init__` avoids repeated type dispatch**: In `Field.__init__`, the code does `self.__class__ = OGRFieldTypes[self.type]`, which dynamically reassigns the instance's class to the appropriate subclass (e.g., `OFTInteger`, `OFTReal`, `OFTDate`, etc.) based on the OGR field type integer. This means the correct `value` property is resolved via normal Python method resolution order (MRO) after the one-time class reassignment, rather than requiring a type-dispatch dictionary lookup or if/elif chain every time `value` is accessed.\n\n3. **No destructor on Field \u2014 avoiding unnecessary C memory management**: The `Field` class inherits from `GDALBase` \u2192 `CPointerBase`, which has a `__del__` method that calls `self.destructor(self.ptr)` if a destructor is set. However, `Field` does not set a `destructor` attribute (it remains `None` from `CPointerBase`). The field definition pointer is obtained via `OGR_F_GetFieldDefnRef` (a reference, not an owned pointer). The `Feature` class, by contrast, sets `destructor = capi.destroy_feature`. This means no destructor call occurs on Field garbage collection.\n\n4. **`bool_output` for `is_field_set` returns native Python bool**: The `is_field_set` binding uses `bool_output` (which sets `restype = c_bool`) with no errcheck callback, making it a ctypes call with a single C boolean return and no error checking overhead. By contrast, `int_output` functions that don't need errcheck still set `restype = c_int`, which requires Python int conversion.\n\n5. **`const_string_output` avoids memory allocation/deallocation**: Field string retrieval functions like `get_field_as_string` and `get_field_name` use `const_string_output` rather than `string_output`. The key difference is that `const_string_output` uses `check_const_string` which does NOT call `lgdal.VSIFree()` on the returned pointer, while `string_output` uses `check_string` which does call `VSIFree`. This avoids the VSIFree call for const strings.\n\n6. **`OFTInteger64` reuses `OFTInteger` via `_bit64` flag**: Rather than duplicating the integer retrieval logic, `OFTInteger64` subclasses `OFTInteger` and only overrides `_bit64 = True`, which causes `as_int(is_64=True)` to call `OGR_F_GetFieldAsInteger64` (returning `c_int64`) instead of `OGR_F_GetFieldAsInteger` (returning `c_int`). This avoids code duplication while ensuring correct integer width handling.",
    "rubric": [
      "Explains that `__class__` mutation in `__init__` (to an OGRFieldTypes subclass) avoids repeated type dispatch on every `value` access, using normal MRO instead",
      "Explains that `is_set` (mapping to `OGR_F_IsFieldSetAndNotNull` via `bool_output`) is checked before calling C API retrieval functions like `as_double`, `as_int`, `as_string`, `as_datetime` to avoid unnecessary C API calls on unset fields",
      "Explains that Field sets no `destructor` (it stays `None` from CPointerBase) because its pointer comes from `OGR_F_GetFieldDefnRef` (a borrowed reference), unlike Feature which sets `destructor = capi.destroy_feature`",
      "Mentions that `OFTInteger64` reuses `OFTInteger` by only overriding `_bit64 = True`, avoiding code duplication while selecting the correct `get_field_as_integer64` vs `get_field_as_integer` call",
      "Notes that `const_string_output` is used for field string functions (like `get_field_as_string` and `get_field_name`) which avoids the `VSIFree` call that `string_output`/`check_string` would perform",
      "Notes that `bool_output` for `is_field_set` uses `c_bool` restype with no errcheck, making it a lightweight ctypes call"
    ],
    "key_files": [
      "django/contrib/gis/gdal/field.py",
      "django/contrib/gis/gdal/prototypes/ds.py",
      "django/contrib/gis/gdal/prototypes/generation.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/feature.py",
      "django/contrib/gis/gdal/prototypes/errcheck.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/field.py] django.contrib.gis.gdal.field.Field\ndjango.contrib.gis.gdal.field.Field:\n  Wrap an OGR Field. Needs to be instantiated from a Feature object.\n\ndjango.contrib.gis.gdal.field.Field.__init__:\n  Initialize on the feature object and the integer index of\n        the field within the feature.\n\ndjango.contrib.gis.gdal.field.Field.__str__:\n  Return the string representation of the Field.\n\ndjango.contrib.gis.gdal.field.Field.as_double:\n  Retrieve the Field's value as a double (float).\n\ndjango.contrib.gis.gdal.field.Field.as_int:\n  Retrieve the Field's value as an integer.\n\ndjango.contrib.gis.gdal.field.Field.as_string:\n  Retrieve the Field's value as a string.\n\ndjango.contrib.gis.gdal.field.Field.as_datetime:\n  Retrieve the Field's value as a tuple of date & time components.\n\ndjango.contrib.gis.gdal.field.Field.is_set:\n  Return True if the value of this field isn't null, False otherwise.\n\ndjango.contrib.gis.gdal.field.Field.name:\n  Return the name of this Field.\n\ndjango.contrib.gis.gdal.field.Field.precision:\n  Return the precision of this Field.\n\ndjango.contrib.gis.gdal.field.Field.type:\n  Return the OGR type of this Field.\n\ndjango.contrib.gis.gdal.field.Field.type_name:\n  Return the OGR field type name for this Field.\n\ndjango.contrib.gis.gdal.field.Field.value:\n  Return the value of this Field.\n\ndjango.contrib.gis.gdal.field.Field.width:\n  Return the width of this Field.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides high-level docstrings for the Field class and its methods. The generated answer makes very specific implementation claims (line numbers, specific C API functions, class mutation patterns, destructor behavior, ctypes details) that cannot be verified from the provided documentation alone.",
      "Claim about `is_set` checking `OGR_F_IsFieldSetAndNotNull` is plausible but not verifiable from the docs.",
      "Claim about `__class__` mutation in `__init__` at line 35 is plausible for Django's GIS implementation but not verifiable from the docs.",
      "Claim about `bool_output`, `const_string_output`, `_bit64` flag, and destructor behavior are all implementation details that go far beyond what the documentation states.",
      "None of the claims directly contradict the documentation - the docs simply don't provide enough detail to confirm or deny most claims."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals of GDAL ownership semantics and cross-file pattern references that rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_72",
    "repo": "django",
    "question": "How does Django's GDAL Layer class wrap OGR layer objects, handle feature access, manage memory, and interact with the C API via ctypes?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `django.contrib.gis.gdal.layer.Layer` class provides a Python wrapper around OGR Layer objects using ctypes for C API interop. Here's how it works in detail:\n\n**Inheritance and Memory Management:**\n- `Layer` extends `GDALBase`, which extends `CPointerBase` (from `django/contrib/gis/ptr.py`). `CPointerBase` manages a C pointer (`_ptr`), with a `ptr` property that raises `null_ptr_exception_class` when a NULL pointer is accessed. Importantly, `Layer` does NOT define a `destructor` \u2014 meaning it does not free the underlying OGR layer pointer when garbage collected.\n\n**Initialization:**\n- `__init__` stores the raw pointer, keeps a reference to the parent `DataSource` (in `self._ds`), fetches the layer definition pointer (`self._ldefn`) via `capi.get_layer_defn`, and tests whether random reading is supported (caching result in `self._random_read`).\n\n**Feature Access (`_make_feature`):**\n- When random read is supported, it calls `capi.get_feature` (wrapping `OGR_L_GetFeature` with a `c_long` feature ID argument). If that raises `GDALException`, it falls through to raise an `IndexError`.\n- When random read is NOT supported, it iterates through ALL features (using `__iter__` which calls `capi.reset_reading` then `capi.get_next_feature` in a loop) and compares `feat.fid` to the requested feature ID.\n\n**Iteration:**\n- `__iter__` calls `OGR_L_ResetReading` before starting, then yields `Feature` objects created from `OGR_L_GetNextFeature` pointers. Features are created with a reference back to the layer.\n\n**Spatial Filter:**\n- The `spatial_filter` property uses a getter/setter pattern. The setter accepts an `OGRGeometry` instance, a 4-element tuple/list (which gets mapped through `c_double`), or `None` to clear the filter. The getter clones the geometry pointer from `OGR_L_GetSpatialFilter`.\n\n**SRS Handling:**\n- The `srs` property calls `OGR_L_GetSpatialRef` then clones it with `srs_api.clone_srs`. If an `SRSException` is raised, it returns `None`.\n\n**get_geoms with GEOS conversion:**\n- `get_geoms` accepts a `geos=False` parameter. When `geos=True`, it imports `GEOSGeometry` from `django.contrib.gis.geos` and converts each feature's geometry via WKB format.\n\n**C API Bindings:**\n- All ctypes prototypes are defined in `django/contrib/gis/gdal/prototypes/ds.py`. For example, `get_feature` uses `lgdal.OGR_L_GetFeature` with args `[c_void_p, c_long]` and returns via `voidptr_output`. The `test_capability` function is declared as `int_output(lgdal.OGR_L_TestCapability, [c_void_p, c_char_p])` \u2014 it returns a C int that Layer wraps with `bool()`.\n\n**DataSource creates Layers:**\n- `DataSource.__getitem__` creates `Layer` objects by passing the pointer from either `GDALDatasetGetLayer` (by index) or `GDALDatasetGetLayerByName` (by string name), along with `self` as the owning DataSource.",
    "rubric": [
      "Explains that Layer extends GDALBase which extends CPointerBase, and that CPointerBase manages a C pointer (_ptr) with a ptr property that raises an exception on NULL access",
      "Notes that Layer does NOT define a destructor, so it does not free the underlying OGR layer pointer on garbage collection (the DataSource owns/destroys it)",
      "Describes __init__: stores pointer, keeps a reference to the parent DataSource in self._ds, fetches the layer definition via capi.get_layer_defn, and caches random read capability in self._random_read",
      "Explains _make_feature: uses capi.get_feature (OGR_L_GetFeature) for random-read-capable layers, falls back to iterating all features comparing feat.fid when random read is not supported, raises IndexError if not found",
      "Describes __iter__: calls capi.reset_reading (OGR_L_ResetReading) first, then yields Feature objects from capi.get_next_feature (OGR_L_GetNextFeature) in a loop",
      "Explains the spatial_filter property: setter accepts OGRGeometry, a 4-element tuple/list (mapped through c_double), or None to clear; getter clones geometry from OGR_L_GetSpatialFilter",
      "Describes srs property: calls OGR_L_GetSpatialRef then clones with clone_srs, returns None if SRSException is raised",
      "Explains get_geoms: when geos=True, imports GEOSGeometry and converts each feature geometry via WKB",
      "Mentions that ctypes prototypes are defined in the ds prototypes module, e.g., get_feature uses OGR_L_GetFeature with [c_void_p, c_long] args and test_capability is an int_output wrapped with bool()",
      "Describes how DataSource.__getitem__ creates Layer objects by passing OGR layer pointers (from GDALDatasetGetLayer or GDALDatasetGetLayerByName) along with self as the owning DataSource"
    ],
    "key_files": [
      "django/contrib/gis/gdal/layer.py",
      "django/contrib/gis/gdal/prototypes/ds.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/datasource.py",
      "django/contrib/gis/gdal/feature.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/layer.py] django.contrib.gis.gdal.layer.Layer\ndjango.contrib.gis.gdal.layer.Layer:\n  A class that wraps an OGR Layer, needs to be instantiated from a DataSource\n    object.\n\ndjango.contrib.gis.gdal.layer.Layer.__init__:\n  Initialize on an OGR C pointer to the Layer and the `DataSource` object\n        that owns this layer. The `DataSource` object is required so that a\n        reference to it is kept with this Layer. This prevents garbage\n        collection of the `DataSource` while this Layer is still active.\n\ndjango.contrib.gis.gdal.layer.Layer.__getitem__:\n  Get the Feature at the specified index.\n\ndjango.contrib.gis.gdal.layer.Layer.__iter__:\n  Iterate over each Feature in the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.__len__:\n  The length is the number of features.\n\ndjango.contrib.gis.gdal.layer.Layer.__str__:\n  The string name of the layer.\n\ndjango.contrib.gis.gdal.layer.Layer._make_feature:\n  Helper routine for __getitem__ that constructs a Feature from the given\n        Feature ID. If the OGR Layer does not support random-access reading,\n        then each feature of the layer will be incremented through until the\n        a Feature is found matching the given feature ID.\n\ndjango.contrib.gis.gdal.layer.Layer.extent:\n  Return the extent (an Envelope) of this layer.\n\ndjango.contrib.gis.gdal.layer.Layer.name:\n  Return the name of this layer in the Data Source.\n\ndjango.contrib.gis.gdal.layer.Layer.num_feat:\n  Return the number of features in the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.num_fields:\n  Return the number of fields in the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.geom_type:\n  Return the geometry type (OGRGeomType) of the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.srs:\n  Return the Spatial Reference used in this Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.fields:\n  Return a list of string names corresponding to each of the Fields\n        available in this Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.field_types:\n  Return a list of the types of fields in this Layer. For example,\n        return the list [OFTInteger, OFTReal, OFTString] for an OGR layer that\n        has an integer, a floating-point, and string fields.\n\ndjango.contrib.gis.gdal.layer.Layer.field_widths:\n  Return a list of the maximum field widths for the features.\n\ndjango.contrib.gis.gdal.layer.Layer.field_precisions:\n  Return the field precisions for the features.\n\ndjango.contrib.gis.gdal.layer.Layer.get_fields:\n  Return a list containing the given field name for every Feature\n        in the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.get_geoms:\n  Return a list containing the OGRGeometry for every Feature in\n        the Layer.\n\ndjango.contrib.gis.gdal.layer.Layer.test_capability:\n  Return a bool indicating whether the this Layer supports the given\n        capability (a string). Valid capability strings include:\n          'RandomRead', 'SequentialWrite', 'RandomWrite', 'FastSpatialFilter',\n          'FastFeatureCount', 'FastGetExtent', 'CreateField', 'Transactions',\n          'DeleteFeature', and 'FastSetNextByIndex'.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: design rationale about memory ownership and internal errcheck function details not verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "django_gen_71",
    "repo": "django",
    "question": "Where is OGRGeometry defined, what's its inheritance chain, and how does it dynamically determine its runtime class?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `OGRGeometry` class is defined in `django/contrib/gis/gdal/geometries.py` and its inheritance chain flows through `GDALBase` (in `django/contrib/gis/gdal/base.py`) which extends `CPointerBase` (in `django/contrib/gis/ptr.py`). The class dynamically reassigns `self.__class__` in `__init__` based on the `GEO_CLASSES` dictionary mapping at the bottom of the file.\n\nKey architectural details:\n\n1. **Base class chain**: `OGRGeometry` \u2192 `GDALBase` \u2192 `CPointerBase`. `GDALBase` is only 2 lines \u2014 it sets `null_ptr_exception_class = GDALException`. `CPointerBase` manages the actual C pointer lifecycle through `_ptr`, a `ptr` property that raises the exception class on NULL access, and a `__del__` method that calls `self.destructor`.\n\n2. **Destructor**: `OGRGeometry` sets `destructor = capi.destroy_geom`, which wraps `lgdal.OGR_G_DestroyGeometry` as a `void_output` with `errcheck=False` (meaning restype is None, no error checking).\n\n3. **Dynamic class assignment**: In `__init__`, after the pointer is set, the code does `self.__class__ = geo_class` based on `GEO_CLASSES.get(self.geom_type.num)`. This means an `OGRGeometry('POINT(0 0)')` will have its class changed to `Point` at runtime.\n\n4. **GEO_CLASSES dictionary**: Maps OGR geometry type integers to Python classes. Key 101 maps to `LinearRing`. The dictionary uses `OGRGeomType.wkb25bit` (value `-2147483648`) to compute keys for 25D geometry types (e.g., key `1 + (-2147483648)` maps to `Point`).\n\n5. **Subclasses with `geos_support = False`**: `CircularString`, `CurvePolygon`, and `CompoundCurve` all set `geos_support = False` (and the pattern continues for `MultiSurface` and `MultiCurve` in the truncated portion). When `geos` property is accessed on these, a `GEOSException` is raised.\n\n6. **The `_geos_ptr` method**: The default implementation in `OGRGeometry` calls `GEOSGeometry._from_wkb(self.wkb)`. The `Point` subclass overrides `_geos_ptr` to handle empty geometries \u2014 it checks `self.empty` and if so returns `geos.Point._create_empty()` instead.\n\n7. **Topology functions**: In `django/contrib/gis/gdal/prototypes/geom.py`, `topology_func` sets argtypes to `[c_void_p, c_void_p]`, restype to `c_int`, and errcheck to `lambda result, func, cargs: bool(result)` \u2014 converting the C integer result to a Python boolean.\n\n8. **WKB export**: The `wkb` property uses `capi.to_iso_wkb` (wrapping `OGR_G_ExportToIsoWkb`) when `self.is_measured` is True, otherwise uses `capi.to_wkb` (wrapping `OGR_G_ExportToWkb`). Same logic applies to `wkt`/`to_iso_wkt`.\n\n9. **Pickle support**: `__getstate__` serializes as `(bytes(self.wkb), srs_wkt_or_None)`. `__setstate__` uses `capi.from_wkb` (wrapping `OGR_G_CreateFromWkbEx`) to reconstruct from the WKB bytes.\n\n10. **The `get_point` prototype** uses `lgdal.OGR_G_GetPointZM` to retrieve all four coordinates (X, Y, Z, M) at once, with `errcheck=False`.",
    "rubric": [
      "OGRGeometry is defined in django/contrib/gis/gdal/geometries.py",
      "Inheritance chain: OGRGeometry \u2192 GDALBase \u2192 CPointerBase",
      "GDALBase (in django/contrib/gis/gdal/base.py) sets null_ptr_exception_class = GDALException",
      "CPointerBase (in django/contrib/gis/ptr.py) manages the C pointer lifecycle via _ptr, a ptr property, and __del__ calling self.destructor",
      "OGRGeometry sets destructor = capi.destroy_geom which wraps OGR_G_DestroyGeometry",
      "In __init__, self.__class__ is reassigned based on GEO_CLASSES.get(self.geom_type.num), dynamically changing the instance's class at runtime",
      "GEO_CLASSES is a dictionary at the bottom of the file mapping OGR geometry type integers to Python subclasses (e.g., 1\u2192Point, 101\u2192LinearRing, and uses OGRGeomType.wkb25bit for 25D types)"
    ],
    "key_files": [
      "django/contrib/gis/gdal/geometries.py",
      "django/contrib/gis/gdal/prototypes/geom.py",
      "django/contrib/gis/ptr.py",
      "django/contrib/gis/gdal/base.py",
      "django/contrib/gis/gdal/geomtype.py",
      "django/contrib/gis/gdal/prototypes/generation.py"
    ],
    "source_doc": "[docstring: django/contrib/gis/gdal/geometries.py] django.contrib.gis.gdal.geometries.OGRGeometry\ndjango.contrib.gis.gdal.geometries.OGRGeometry:\n  Encapsulate an OGR geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__init__:\n  Initialize Geometry on either WKT or an OGR pointer as input.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.from_bbox:\n  Construct a Polygon from a bounding box (4-tuple).\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__or__:\n  Return the union of the two geometries.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__and__:\n  Return the intersection of this Geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__sub__:\n  Return the difference this Geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__xor__:\n  Return the symmetric difference of this Geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__eq__:\n  Is this Geometry equal to the other?\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.__str__:\n  WKT is used for the string representation.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.dimension:\n  Return 0 for points, 1 for lines, and 2 for surfaces.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.coord_dim:\n  Return the coordinate dimension of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.geom_count:\n  Return the number of elements in this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.point_count:\n  Return the number of Points in this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.num_points:\n  Alias for `point_count` (same name method in GEOS API.)\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.num_coords:\n  Alias for `point_count`.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.geom_type:\n  Return the Type for this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.geom_name:\n  Return the Name of this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.area:\n  Return the area for a LinearRing, Polygon, or MultiPolygon; 0\n        otherwise.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.envelope:\n  Return the envelope for this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.extent:\n  Return the envelope as a 4-tuple, instead of as an Envelope object.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.is_3d:\n  Return True if the geometry has Z coordinates.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.set_3d:\n  Set if this geometry has Z coordinates.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.is_measured:\n  Return True if the geometry has M coordinates.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.set_measured:\n  Set if this geometry has M coordinates.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.has_curve:\n  Return True if the geometry is or has curve geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.get_linear_geometry:\n  Return a linear version of this geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.get_curve_geometry:\n  Return a curve version of this geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry._get_srs:\n  Return the Spatial Reference for this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry._set_srs:\n  Set the SpatialReference for this geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.geos:\n  Return a GEOSGeometry object from this OGRGeometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.gml:\n  Return the GML representation of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.hex:\n  Return the hexadecimal representation of the WKB (a string).\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.json:\n  Return the GeoJSON representation of this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.kml:\n  Return the KML representation of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.wkb_size:\n  Return the size of the WKB buffer.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.wkb:\n  Return the WKB representation of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.wkt:\n  Return the WKT representation of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.ewkt:\n  Return the EWKT representation of the Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.clone:\n  Clone this OGR Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.close_rings:\n  If there are any rings within this geometry that have not been\n        closed, this routine will do so by adding the starting point at the\n        end.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.transform:\n  Transform this geometry to a different spatial reference system.\n        May take a CoordTransform object, a SpatialReference object, string\n        WKT or PROJ, and/or an integer SRID. By default, return nothing\n        and transform the geometry in-place. However, if the `clone` keyword is\n        set, return a transformed clone of this geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry._topology:\n  A generalized function for topology operations, takes a GDAL\n        function and the other geometry to perform the operation on.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.intersects:\n  Return True if this geometry intersects with the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.equals:\n  Return True if this geometry is equivalent to the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.disjoint:\n  Return True if this geometry and the other are spatially disjoint.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.touches:\n  Return True if this geometry touches the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.crosses:\n  Return True if this geometry crosses the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.within:\n  Return True if this geometry is within the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.contains:\n  Return True if this geometry contains the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.overlaps:\n  Return True if this geometry overlaps the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry._geomgen:\n  A helper routine for the OGR routines that generate geometries.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.boundary:\n  Return the boundary of this geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.convex_hull:\n  Return the smallest convex Polygon that contains all the points in\n        this Geometry.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.difference:\n  Return a new geometry consisting of the region which is the difference\n        of this geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.intersection:\n  Return a new geometry consisting of the region of intersection of this\n        geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.sym_difference:\n  Return a new geometry which is the symmetric difference of this\n        geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.union:\n  Return a new geometry consisting of the region which is the union of\n        this geometry and the other.\n\ndjango.contrib.gis.gdal.geometries.OGRGeometry.centroid:\n  Return the centroid (a Point) of this Polygon.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only one rationale about backward compatibility was documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_03",
    "repo": "scikit_learn",
    "question": "Where does bicluster data flow through when computing a consensus score, from input to final result?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `consensus_score` function in `sklearn/metrics/cluster/_bicluster.py` receives bicluster data as tuples and flows through several stages:\n\n1. **Entry point**: `consensus_score(a, b, *, similarity='jaccard')`. If `similarity == 'jaccard'`, it's replaced with the internal `_jaccard` function.\n\n2. **Pairwise similarity computation**: Data flows to `_pairwise_similarity(a, b, similarity)`, which first calls `_check_rows_and_columns(a, b)` to unpack and validate inputs. `_check_rows_and_columns` calls `check_consistent_length(*a)` and `check_consistent_length(*b)` from `sklearn.utils.validation`. It then applies `check_array(x, ensure_2d=False)` to each component.\n\n3. **Similarity matrix**: `_pairwise_similarity` builds an n_a \u00d7 n_b numpy array where element [i,j] is the similarity between bicluster i from set a and bicluster j from set b.\n\n4. **Assignment**: The cost matrix `1.0 - matrix` is passed to `scipy.optimize.linear_sum_assignment`, which returns `row_indices` and `col_indices`.\n\n5. **Final score**: The score is `matrix[row_indices, col_indices].sum() / max(n_a, n_b)` cast to float. The denominator is the size of the larger set, computed as `max(len(a[0]), len(b[0]))`.\n\n6. **Data source for models**: `BiclusterMixin.biclusters_` property (in `sklearn/base.py`) returns `(self.rows_, self.columns_)` \u2014 the same tuple format expected by `consensus_score`.\n\n7. **Parameter validation**: The function is decorated with `@validate_params`, constraining `a` and `b` to be tuples and `similarity` to be either callable or the string 'jaccard' (via `StrOptions({'jaccard'})`).\n\n8. **Jaccard computation**: The `_jaccard` function computes intersection as `(a_rows * b_rows).sum() * (a_cols * b_cols).sum()` and divides by `(a_size + b_size - intersection)` where sizes are computed as products of row and column sums.",
    "rubric": [
      "Entry point is consensus_score(a, b, *, similarity='jaccard'), where 'jaccard' string is replaced with the internal _jaccard function",
      "Data flows to _pairwise_similarity which calls _check_rows_and_columns to unpack/validate inputs using check_consistent_length and check_array",
      "_pairwise_similarity builds an n_a \u00d7 n_b similarity matrix by computing pairwise similarities between biclusters",
      "The cost matrix (1.0 - matrix) is passed to scipy.optimize.linear_sum_assignment to find optimal assignment",
      "Final score is matrix[row_indices, col_indices].sum() / max(n_a, n_b)",
      "BiclusterMixin.biclusters_ property in base.py provides the (rows_, columns_) tuple format expected by consensus_score",
      "The function uses @validate_params to constrain a and b to tuples and similarity to callable or StrOptions({'jaccard'})",
      "_jaccard computes intersection as (a_rows * b_rows).sum() * (a_cols * b_cols).sum() divided by (a_size + b_size - intersection)"
    ],
    "key_files": [
      "sklearn/metrics/cluster/_bicluster.py",
      "sklearn/base.py",
      "sklearn/metrics/cluster/tests/test_bicluster.py"
    ],
    "source_doc": "[docstring: sklearn/metrics/cluster/_bicluster.py] sklearn.metrics.cluster._bicluster.consensus_score\nThe similarity of two sets of biclusters.\n\n    Similarity between individual biclusters is computed. Then the best\n    matching between sets is found by solving a linear sum assignment problem,\n    using a modified Jonker-Volgenant algorithm.\n    The final score is the sum of similarities divided by the size of\n    the larger set.\n\n    Read more in the :ref:`User Guide <biclustering>`.\n\n    Parameters\n    ----------\n    a : tuple (rows, columns)\n        Tuple of row and column indicators for a set of biclusters.\n\n    b : tuple (rows, columns)\n        Another set of biclusters like ``a``.\n\n    similarity : 'jaccard' or callable, default='jaccard'\n        May be the string \"jaccard\" to use the Jaccard coefficient, or\n        any function that takes four arguments, each of which is a 1d\n        indicator vector: (a_rows, a_columns, b_rows, b_columns).\n\n    Returns\n    -------\n    consensus_score : float\n       Consensus score, a non-negative value, sum of similarities\n       divided by size of larger set.\n\n    See Also\n    --------\n    scipy.optimize.linear_sum_assignment : Solve the linear sum assignment problem.\n\n    References\n    ----------\n    * Hochreiter, Bodenhofer, et. al., 2010. `FABIA: factor analysis\n      for bicluster acquisition\n      <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2881408/>`__.\n\n    Examples\n    --------\n    >>> from sklearn.metrics import consensus_score\n    >>> a = ([[True, False], [False, True]], [[False, True], [True, False]])\n    >>> b = ([[False, True], [True, False]], [[True, False], [False, True]])\n    >>> consensus_score(a, b, similarity='jaccard')\n    1.0",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in line number references and behavioral descriptions of external functions; core logic claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_04",
    "repo": "scikit_learn",
    "question": "How does `contingency_matrix` build and return the co-occurrence matrix from true and predicted labels?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `contingency_matrix` function builds a contingency matrix using a sparse COO (Coordinate) array approach. Here's how it works step by step:\n\n1. **Input validation**: The function uses the `@validate_params` decorator which validates `eps` as an `Interval(Real, 0, None, closed=\"left\")` (must be strictly positive) or `None`, and `dtype` is delegated to SciPy with `\"no_validation\"`. Notably, `labels_true` and `labels_pred` are allowed to be `None` in the parameter validation spec.\n\n2. **Error check**: If both `eps` is not None and `sparse=True`, it raises `ValueError(\"Cannot set 'eps' when sparse=True\")`.\n\n3. **Label encoding**: It calls `np.unique(labels_true, return_inverse=True)` and `np.unique(labels_pred, return_inverse=True)` to get unique classes/clusters and their integer indices, mapping labels to consecutive integer indices.\n\n4. **Sparse construction**: It constructs a `scipy.sparse.coo_array` with `np.ones(class_idx.shape[0])` as data and `(class_idx, cluster_idx)` as row/column coordinates. The shape is `(n_classes, n_clusters)`. Since COO format sums duplicate entries when converted, the same (i,j) pair appearing multiple times effectively counts co-occurrences.\n\n5. **Output formatting**:\n   - If `sparse=True`: calls `_align_api_if_sparse(contingency)`, then `.tocsr()`, then `.sum_duplicates()` to consolidate the duplicate entries.\n   - If `sparse=False`: calls `.toarray()` to get dense output; if `eps` is provided, uses `contingency = contingency + eps` (not `+=`).\n\n6. **Key design detail**: `contingency_matrix` does NOT call `check_clusterings()` internally. All downstream consumers (like `pair_confusion_matrix`, `adjusted_mutual_info_score`, `homogeneity_completeness_v_measure`) call `check_clusterings()` themselves before calling `contingency_matrix`.\n\n7. **Downstream usage**: The sparse contingency matrix is consumed by `mutual_info_score` (which extracts nonzero values via `sp.find()`), `expected_mutual_information` (a Cython function that computes row/column sums and iterates over possible nij values), `pair_confusion_matrix` (which uses `contingency.data**2` for sum of squares), and `fowlkes_mallows_score` (which uses `np.dot(c.data, c.data)`).\n\n8. **The `_align_api_if_sparse` helper** (in `sklearn/utils/_sparse.py`) checks the global config's `sparse_interface` setting and converts between `sparray` and `spmatrix` formats as needed.",
    "rubric": [
      "Mentions label encoding via np.unique with return_inverse=True to map labels to integer indices",
      "Explains construction of a scipy sparse COO array with ones as data and (class_idx, cluster_idx) as coordinates",
      "Describes how duplicate COO entries are summed to count co-occurrences",
      "Explains the sparse=True path: converts to CSR format and calls sum_duplicates()",
      "Explains the sparse=False path: converts to dense array and optionally adds eps",
      "Notes the ValueError raised when both eps is not None and sparse=True",
      "Mentions that check_clusterings is NOT called inside contingency_matrix but by its callers"
    ],
    "key_files": [
      "sklearn/metrics/cluster/_supervised.py",
      "sklearn/utils/_sparse.py",
      "sklearn/metrics/cluster/_expected_mutual_info_fast.pyx"
    ],
    "source_doc": "[docstring: sklearn/metrics/cluster/_supervised.py] sklearn.metrics.cluster._supervised.contingency_matrix\nBuild a contingency matrix describing the relationship between labels.\n\n    Read more in the :ref:`User Guide <contingency_matrix>`.\n\n    Parameters\n    ----------\n    labels_true : array-like of shape (n_samples,)\n        Ground truth class labels to be used as a reference.\n\n    labels_pred : array-like of shape (n_samples,)\n        Cluster labels to evaluate.\n\n    eps : float, default=None\n        If a float, that value is added to all values in the contingency\n        matrix. This helps to stop NaN propagation.\n        If ``None``, nothing is adjusted.\n\n    sparse : bool, default=False\n        If `True`, return a sparse CSR contingency matrix. If `eps` is not\n        `None` and `sparse` is `True` will raise ValueError.\n\n        .. versionadded:: 0.18\n\n    dtype : numeric type, default=np.int64\n        Output dtype. Ignored if `eps` is not `None`.\n\n        .. versionadded:: 0.24\n\n    Returns\n    -------\n    contingency : {array-like, sparse}, shape=[n_classes_true, n_classes_pred]\n        Matrix :math:`C` such that :math:`C_{i, j}` is the number of samples in\n        true class :math:`i` and in predicted class :math:`j`.\n        If ``eps is None``, the dtype of this array will be integer unless set\n        otherwise with the ``dtype`` argument. If ``eps`` is given, the dtype\n        will be float.\n        If ``sparse=True`` will be a sparse CSR contingency.\n\n    Examples\n    --------\n    >>> from sklearn.metrics.cluster import contingency_matrix\n    >>> labels_true = [0, 0, 1, 1, 2, 2]\n    >>> labels_pred = [1, 0, 2, 1, 0, 2]\n    >>> contingency_matrix(labels_true, labels_pred)\n    array([[1, 1, 0],\n           [0, 1, 1],\n           [1, 0, 1]])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `eps` must be 'strictly positive' (Interval with closed='left' starting at 0) is not verifiable from the documentation alone - the doc only says 'If a float, that value is added to all values'. This is a code-level detail that could be correct but cannot be confirmed from the provided documentation.",
      "The claim that `labels_true` and `labels_pred` are allowed to be `None` in the parameter validation spec seems suspicious given the documentation describes them as 'array-like of shape (n_samples,)' - this could be a hallucination about internal implementation details.",
      "The claim about `_align_api_if_sparse` helper and `sparse_interface` global config setting is implementation detail that cannot be verified from the documentation.",
      "The claim that 'contingency_matrix does NOT call check_clusterings() internally' is an implementation detail not verifiable from the provided documentation.",
      "The documentation states 'If `True`, return a sparse CSR contingency matrix. If `eps` is not `None` and `sparse` is `True` will raise ValueError.' The answer's error message text 'Cannot set \\'eps\\' when sparse=True' may or may not be the exact error message - this is a minor detail.",
      "The documentation says 'dtype : numeric type, default=np.int64. Output dtype. Ignored if `eps` is not `None`.' The answer says 'dtype is delegated to SciPy with no_validation' which partially contradicts the documented default of np.int64.",
      "Most claims in the answer are about internal implementation details (COO array construction, sum_duplicates, downstream consumers, etc.) that go well beyond the provided documentation but are plausible code-level knowledge rather than contradictions."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in design rationale explanations (histogram trick, why not +=, why not histogram2d); all core code-level claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_02",
    "repo": "scikit_learn",
    "question": "Why does GammaRegressor use HalfGammaLoss directly instead of HalfTweedieLoss with power=2, and what constraints does this impose?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `GammaRegressor` uses `HalfGammaLoss` directly rather than `HalfTweedieLoss(power=2)`. In the Cython implementation, `CyHalfGammaLoss` and `CyHalfTweedieLoss` are separate classes (as seen in the class_list in `_loss.pyx.tp`), so using `HalfGammaLoss` directly avoids going through the Tweedie parameterization.\n\nThe `HalfGammaLoss` class hardcodes the `LogLink()` link function and sets `interval_y_true = Interval(0, np.inf, False, False)` (strictly positive, excluding zero). The Gamma loss formula is `raw_prediction + y_true * exp(-raw_prediction)` (visible in `closs_half_gamma`), and `constant_to_optimal_zero` computes `-np.log(y_true) - 1`, which requires `y_true > 0` since `log(y_true)` is undefined at zero.\n\nThe log link function serves as a natural constraint: since `predict()` applies `exp(X @ coef_ + intercept_)` (via `LogLink.inverse`), predictions are guaranteed to be strictly positive regardless of coefficient values. This ensures predictions remain in the valid domain of the Gamma distribution without needing explicit clipping.\n\nThe intercept initialization uses `link.link(_average(y, weights=sample_weight))` which for GammaRegressor computes `log(mean(y))`.\n\nNotably, `GammaRegressor` does NOT override `__sklearn_tags__` (unlike `PoissonRegressor`), so it inherits the parent's tag logic that sets `target_tags.positive_only = True` (since `HalfGammaLoss.in_y_true_range(-1.0)` returns False). Also, there is no `HalfGammaLossArrayAPI` class, so `GammaRegressor` does not support the Array API (unlike `PoissonRegressor` which has `array_api_support = self.solver == 'lbfgs'`).\n\nThe L-BFGS-B solver uses `maxls=50` and `ftol = 64 * np.finfo(float).eps`, which are shared across all GLM regressors.",
    "rubric": [
      "HalfGammaLoss is a separate Cython class (CyHalfGammaLoss) from CyHalfTweedieLoss, avoiding Tweedie parameterization overhead",
      "HalfGammaLoss hardcodes LogLink as its link function",
      "HalfGammaLoss sets interval_y_true to strictly positive (excluding zero) because the loss formula involves log(y_true)",
      "The LogLink ensures predictions are always strictly positive via exp() inverse, keeping them in the valid Gamma domain",
      "constant_to_optimal_zero computes -log(y_true) - 1, which requires y_true > 0",
      "GammaRegressor does not override __sklearn_tags__, inheriting positive_only=True from the parent class based on in_y_true_range",
      "No HalfGammaLossArrayAPI class exists, so GammaRegressor does not support Array API unlike PoissonRegressor"
    ],
    "key_files": [
      "sklearn/linear_model/_glm/glm.py",
      "sklearn/_loss/loss.py",
      "sklearn/_loss/_loss.pyx.tp",
      "sklearn/_loss/link.py"
    ],
    "source_doc": "[docstring: sklearn/linear_model/_glm/glm.py] sklearn.linear_model._glm.glm.GammaRegressor\nsklearn.linear_model._glm.glm.GammaRegressor:\n  Generalized Linear Model with a Gamma distribution.\n\n    This regressor uses the 'log' link function.\n\n    Read more in the :ref:`User Guide <Generalized_linear_models>`.\n\n    .. versionadded:: 0.23\n\n    Parameters\n    ----------\n    alpha : float, default=1\n        Constant that multiplies the L2 penalty term and determines the\n        regularization strength. ``alpha = 0`` is equivalent to unpenalized\n        GLMs. In this case, the design matrix `X` must have full column rank\n        (no collinearities).\n        Values of `alpha` must be in the range `[0.0, inf)`.\n\n    fit_intercept : bool, default=True\n        Specifies if a constant (a.k.a. bias or intercept) should be\n        added to the linear predictor `X @ coef_ + intercept_`.\n\n    solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs'\n        Algorithm to use in the optimization problem:\n\n        'lbfgs'\n            Calls scipy's L-BFGS-B optimizer.\n\n        'newton-cholesky'\n            Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to\n            iterated reweighted least squares) with an inner Cholesky based solver.\n            This solver is a good choice for `n_samples` >> `n_features`, especially\n            with one-hot encoded categorical features with rare categories. Be aware\n            that the memory usage of this solver has a quadratic dependency on\n            `n_features` because it explicitly computes the Hessian matrix.\n\n            .. versionadded:: 1.2\n\n    max_iter : int, default=100\n        The maximal number of iterations for the solver.\n        Values must be in the range `[1, inf)`.\n\n    tol : float, default=1e-4\n        Stopping criterion. For the lbfgs solver,\n        the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol``\n        where ``g_j`` is the j-th component of the gradient (derivative) of\n        the objective function.\n        Values must be in the range `(0.0, inf)`.\n\n    warm_start : bool, default=False\n        If set to ``True``, reuse the solution of the previous call to ``fit``\n        as initialization for `coef_` and `intercept_`.\n\n    verbose : int, default=0\n        For the lbfgs solver set verbose to any positive number for verbosity.\n        Values must be in the range `[0, inf)`.\n\n    Attributes\n    ----------\n    coef_ : array of shape (n_features,)\n        Estimated coefficients for the linear predictor (`X @ coef_ +\n        intercept_`) in the GLM.\n\n    intercept_ : float\n        Intercept (a.k.a. bias) added to linear predictor.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    n_iter_ : int\n        Actual number of iterations used in the solver.\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    PoissonRegressor : Generalized Linear Model with a Poisson distribution.\n    TweedieRegressor : Generalized Linear Model with a Tweedie distribution.\n\n    Examples\n    --------\n    >>> from sklearn import linear_model\n    >>> clf = linear_model.GammaRegressor()\n    >>> X = [[1, 2], [2, 3], [3, 4], [4, 3]]\n    >>> y = [19, 26, 33, 30]\n    >>> clf.fit(X, y)\n    GammaRegressor()\n    >>> clf.score(X, y)\n    np.float64(0.773)\n    >>> clf.coef_\n    array([0.073, 0.067])\n    >>> clf.intercept_\n    np.float64(2.896)\n    >>> clf.predict([[1, 0], [2, 8]])\n    array([19.483, 35.795])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (Cython code references, specific line numbers, HalfGammaLoss internals, L-BFGS-B parameters, Array API support details) that cannot be verified from the provided documentation alone.",
      "The claim about GammaRegressor NOT overriding __sklearn_tags__ and not supporting Array API cannot be verified from the documentation.",
      "The specific L-BFGS-B parameters (maxls=50, ftol = 64 * np.finfo(float).eps) cannot be verified from the documentation.",
      "The claim about HalfGammaLoss vs HalfTweedieLoss(power=2) implementation details cannot be verified from the documentation.",
      "However, the core claims that are verifiable from the documentation (log link function, Gamma distribution, alpha parameter behavior, solver options) are consistent with the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in design rationale ('null model', 'empirically determined', 'increased from default 20') and one docstring-sourced formula detail; most claims verified from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_05",
    "repo": "scikit_learn",
    "question": "What does `pair_confusion_matrix` compute, how is its 2\u00d72 output derived from the contingency matrix, and how do `rand_score` and `adjusted_rand_score` use it?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `pair_confusion_matrix` function computes a 2\u00d72 confusion matrix from two clusterings (true and predicted). It is defined in `sklearn/metrics/cluster/_supervised.py` and is decorated with `@validate_params` that validates both `labels_true` and `labels_pred` as \"array-like\" with `prefer_skip_nested_validation=True`.\n\n**Implementation details:**\n1. It first calls `check_clusterings(labels_true, labels_pred)` which validates both label arrays using `check_array` (with `ensure_2d=False`, `ensure_min_samples=0`, `dtype=None`), checks they are 1D, emits a `UserWarning` if either has \"continuous\" type (via `type_of_target`), and verifies consistent length.\n\n2. The number of samples `n_samples` is explicitly cast to `np.int64`.\n\n3. It builds a **sparse** contingency matrix by calling `contingency_matrix(labels_true, labels_pred, sparse=True, dtype=np.int64)`. Internally, `contingency_matrix` constructs a `scipy.sparse.coo_array` from ones at positions `(class_idx, cluster_idx)` (where indices come from `np.unique(..., return_inverse=True)`), then converts to CSR format via `_align_api_if_sparse` followed by `.tocsr()` and `.sum_duplicates()`.\n\n4. The 2\u00d72 matrix is computed from the sparse contingency matrix using:\n   - `n_c` = row sums, `n_k` = column sums (both via `np.ravel` of axis sums)\n   - `sum_squares` = sum of squared entries (`contingency.data**2`).sum()\n   - `C[1,1]` = `sum_squares - n_samples`\n   - `C[0,1]` = `contingency.dot(n_k).sum() - sum_squares`\n   - `C[1,0]` = `contingency.transpose().dot(n_c).sum() - sum_squares`\n   - `C[0,0]` = `n_samples**2 - C[0,1] - C[1,0] - sum_squares`\n   \n   (From `adjusted_rand_score`, these positions correspond to: `(tn, fp), (fn, tp) = pair_confusion_matrix(...)`)\n\n5. The result is a pre-allocated `np.empty((2, 2), dtype=np.int64)` array.\n\n**Callers:**\n- `rand_score` calls `pair_confusion_matrix` and computes the score as `diagonal().sum() / total_sum`, returning 1.0 when `numerator == denominator` or `denominator == 0`.\n- `adjusted_rand_score` destructures the result via `(tn, fp), (fn, tp) = pair_confusion_matrix(...)`, converts all four values to Python `int`, and computes `2.0 * (tp * tn - fn * fp) / ((tp + fn) * (fn + tn) + (tp + fp) * (fp + tn))`. It returns 1.0 when both `fn == 0` and `fp == 0`.\n\n**Test edge cases** in `test_supervised.py`:\n- Fully dispersed (each element its own cluster): expects `[[N*(N-1), 0], [0, 0]]`\n- Single cluster: expects `[[0, 0], [0, N*(N-1)]]`\n- Regular case uses a brute-force O(n\u00b2) pairwise comparison as reference.",
    "rubric": [
      "Explains that pair_confusion_matrix computes a 2\u00d72 confusion matrix by comparing all pairs of samples across two clusterings",
      "Mentions that it calls check_clusterings to validate inputs (1D check, consistent length, UserWarning for continuous types)",
      "Describes building a sparse contingency matrix via contingency_matrix with sparse=True and dtype=np.int64",
      "Explains the four matrix entries: C[1,1] = sum_squares - n_samples, C[0,1] = contingency.dot(n_k).sum() - sum_squares, C[1,0] = contingency.T.dot(n_c).sum() - sum_squares, C[0,0] = n_samples**2 - C[0,1] - C[1,0] - sum_squares",
      "Notes that n_c and n_k are row and column sums of the contingency matrix, and sum_squares is the sum of squared entries",
      "Describes that rand_score computes diagonal sum divided by total sum, returning 1.0 for special cases (numerator==denominator or denominator==0)",
      "Describes that adjusted_rand_score destructures as (tn, fp), (fn, tp) and computes 2*(tp*tn - fn*fp) / ((tp+fn)*(fn+tn) + (tp+fp)*(fp+tn)), returning 1.0 when fn==0 and fp==0",
      "Mentions the output is a np.int64 dtype 2\u00d72 array"
    ],
    "key_files": [
      "sklearn/metrics/cluster/_supervised.py",
      "sklearn/metrics/cluster/__init__.py",
      "sklearn/metrics/cluster/tests/test_supervised.py",
      "sklearn/utils/_sparse.py"
    ],
    "source_doc": "[docstring: sklearn/metrics/cluster/_supervised.py] sklearn.metrics.cluster._supervised.pair_confusion_matrix\nPair confusion matrix arising from two clusterings.\n\n    The pair confusion matrix :math:`C` computes a 2 by 2 similarity matrix\n    between two clusterings by considering all pairs of samples and counting\n    pairs that are assigned into the same or into different clusters under\n    the true and predicted clusterings [1]_.\n\n    Considering a pair of samples that is clustered together a positive pair,\n    then as in binary classification the count of true negatives is\n    :math:`C_{00}`, false negatives is :math:`C_{10}`, true positives is\n    :math:`C_{11}` and false positives is :math:`C_{01}`.\n\n    Read more in the :ref:`User Guide <pair_confusion_matrix>`.\n\n    Parameters\n    ----------\n    labels_true : array-like of shape (n_samples,), dtype=integral\n        Ground truth class labels to be used as a reference.\n\n    labels_pred : array-like of shape (n_samples,), dtype=integral\n        Cluster labels to evaluate.\n\n    Returns\n    -------\n    C : ndarray of shape (2, 2), dtype=np.int64\n        The contingency matrix.\n\n    See Also\n    --------\n    sklearn.metrics.rand_score : Rand Score.\n    sklearn.metrics.adjusted_rand_score : Adjusted Rand Score.\n    sklearn.metrics.adjusted_mutual_info_score : Adjusted Mutual Information.\n\n    References\n    ----------\n    .. [1] :doi:`Hubert, L., Arabie, P. \"Comparing partitions.\"\n           Journal of Classification 2, 193\u2013218 (1985).\n           <10.1007/BF01908075>`\n\n    Examples\n    --------\n    Perfectly matching labelings have all non-zero entries on the\n    diagonal regardless of actual label values:\n\n      >>> from sklearn.metrics.cluster import pair_confusion_matrix\n      >>> pair_confusion_matrix([0, 0, 1, 1], [1, 1, 0, 0])\n      array([[8, 0],\n             [0, 4]]...\n\n    Labelings that assign all classes members to the same clusters\n    are complete but may be not always pure, hence penalized, and\n    have some off-diagonal non-zero entries:\n\n      >>> pair_confusion_matrix([0, 0, 1, 2], [0, 0, 1, 1])\n      array([[8, 2],\n             [0, 2]]...\n\n    Note that the matrix is not symmetric.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc-leakage in semantic labels (true positives etc.), 'Rand Index' naming, and 'to avoid overflow' rationale; all computation details are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_01",
    "repo": "scikit_learn",
    "question": "What is the full class hierarchy and loss function architecture behind PoissonRegressor, including its solvers and convergence behavior?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `PoissonRegressor` class is a specialized Generalized Linear Model (GLM) that inherits from `_GeneralizedLinearRegressor`, which itself inherits from both `RegressorMixin` and `BaseEstimator`. Here is how the architecture is structured:\n\n**Class Hierarchy:**\n- `PoissonRegressor` \u2192 `_GeneralizedLinearRegressor` \u2192 `RegressorMixin, BaseEstimator`\n- `PoissonRegressor` overrides `_get_loss()` to return `HalfPoissonLoss()` (for NumPy) or `HalfPoissonLossArrayAPI(xp, device)` (for array API backends)\n- `PoissonRegressor` overrides `__sklearn_tags__()` to set `tags.array_api_support = (self.solver == 'lbfgs')`, meaning only the lbfgs solver supports the array API\n\n**Loss Function Chain:**\n- `HalfPoissonLoss` (in `sklearn/_loss/loss.py`) wraps `CyHalfPoissonLoss()` (Cython) with a `LogLink()` link function\n- `HalfPoissonLoss.__init__` sets `self.interval_y_true = Interval(0, np.inf, True, False)` \u2014 y_true must be in [0, \u221e), i.e., zero is included (left-closed) but infinity is not\n- The Cython loss function `closs_half_poisson` computes `exp(raw_prediction) - y_true * raw_prediction`\n- The Cython gradient `cgradient_half_poisson` computes `exp(raw_prediction) - y_true`\n- The Cython hessian in `cgrad_hess_half_poisson` is exactly `exp(raw_prediction)`\n- `HalfPoissonLoss.constant_to_optimal_zero` returns `xlogy(y_true, y_true) - y_true`, which is used in the score() method's D\u00b2 computation\n\n**Link Function:**\n- `LogLink` (in `sklearn/_loss/link.py`) implements `link(y_pred) = log(y_pred)` and `inverse(raw_prediction) = exp(raw_prediction)`\n- `LogLink.interval_y_pred = Interval(0, float('inf'), False, False)` \u2014 predictions are strictly positive\n\n**Fit Process:**\n- The `fit()` method in `_GeneralizedLinearRegressor` wraps the loss in a `LinearModelLoss` object, which handles `raw_prediction = X @ coef + intercept`\n- When not warm-starting and `fit_intercept=True`, the intercept is initialized to `link(weighted_mean(y))` \u2014 for Poisson, this is `log(mean(y))`\n- For the 'lbfgs' solver, scipy's L-BFGS-B is called with `maxls=50`, `ftol = 64 * np.finfo(float).eps`, and the `func` passed is `linear_loss.loss_gradient` (returns both loss and gradient jointly)\n- For the 'newton-cholesky' solver, a `NewtonCholeskySolver` is created and its `solve()` method runs: setup \u2192 loop of (update_gradient_hessian \u2192 inner_solve \u2192 line_search \u2192 check_convergence) \u2192 finalize\n- The Newton solver uses backtracking line search with parameters `beta=0.5, sigma=1/2048` (i.e., `0.00048828125` = `1/2**11`), with up to 21 line search iterations (down to step size `beta**20 ~ 1e-6`)\n- The Newton solver has two convergence criteria: (1) max |gradient| \u2264 tol, AND (2) Newton decrement: 1/2 * d\u00b2 \u2264 tol\n- If the Newton solver's inner Cholesky solver detects many negative hessian values, it falls back to lbfgs via `fallback_lbfgs_solve`\n\n**Score Method:**\n- The `score()` method overrides the default `RegressorMixin.score()` and computes `1 - (deviance + constant) / (deviance_null + constant)`, where `constant` comes from `constant_to_optimal_zero` and `deviance_null` uses predictions at `link(mean(y))`\n\n**Parameter Constraints:**\n- `PoissonRegressor._parameter_constraints` is a copy of `_GeneralizedLinearRegressor._parameter_constraints` (no additional constraints added, unlike `TweedieRegressor` which adds `power` and `link`)\n- The solver parameter also accepts `Hidden(type)`, allowing passing `NewtonSolver` subclasses directly, as seen in the `issubclass(self.solver, NewtonSolver)` branch in `fit()`\n\n**Array API Support:**\n- `HalfPoissonLossArrayAPI` is imported from `sklearn._loss.loss` and used in `PoissonRegressor._get_loss` for non-NumPy array API backends",
    "rubric": [
      "Identifies the class hierarchy: PoissonRegressor inherits from _GeneralizedLinearRegressor, which inherits from RegressorMixin and BaseEstimator",
      "Explains that PoissonRegressor overrides _get_loss() to return HalfPoissonLoss() (or HalfPoissonLossArrayAPI for array API backends)",
      "Describes HalfPoissonLoss wrapping CyHalfPoissonLoss (Cython) with a LogLink link function",
      "Notes LogLink implements link=log(y_pred) and inverse=exp(raw_prediction), with interval_y_pred being strictly positive (open interval)",
      "Mentions HalfPoissonLoss sets interval_y_true = Interval(0, inf, True, False), meaning y_true must be in [0, \u221e)",
      "Describes the fit() method creates a LinearModelLoss and initializes intercept to link(weighted_mean(y)) = log(mean(y)) when not warm-starting",
      "Explains the lbfgs solver path uses scipy L-BFGS-B with maxls=50 and ftol=64*eps",
      "Explains the newton-cholesky solver uses NewtonCholeskySolver with backtracking line search (beta=0.5, sigma=1/2048) and two convergence criteria (max|gradient|<=tol AND 1/2*d\u00b2<=tol)",
      "Mentions the Newton solver can fall back to lbfgs if it detects negative hessian values or a non-descent direction",
      "Describes the score() method computing D\u00b2 deviance using constant_to_optimal_zero (which returns xlogy(y_true, y_true) - y_true)",
      "Notes that __sklearn_tags__() sets array_api_support = (self.solver == 'lbfgs')"
    ],
    "key_files": [
      "sklearn/linear_model/_glm/glm.py",
      "sklearn/_loss/loss.py",
      "sklearn/_loss/link.py",
      "sklearn/_loss/_loss.pyx.tp",
      "sklearn/_loss/_loss.pxd",
      "sklearn/linear_model/_linear_loss.py",
      "sklearn/linear_model/_glm/_newton_solver.py"
    ],
    "source_doc": "[docstring: sklearn/linear_model/_glm/glm.py] sklearn.linear_model._glm.glm.PoissonRegressor\nsklearn.linear_model._glm.glm.PoissonRegressor:\n  Generalized Linear Model with a Poisson distribution.\n\n    This regressor uses the 'log' link function.\n\n    Read more in the :ref:`User Guide <Generalized_linear_models>`.\n\n    .. versionadded:: 0.23\n\n    Parameters\n    ----------\n    alpha : float, default=1\n        Constant that multiplies the L2 penalty term and determines the\n        regularization strength. ``alpha = 0`` is equivalent to unpenalized\n        GLMs. In this case, the design matrix `X` must have full column rank\n        (no collinearities).\n        Values of `alpha` must be in the range `[0.0, inf)`.\n\n    fit_intercept : bool, default=True\n        Specifies if a constant (a.k.a. bias or intercept) should be\n        added to the linear predictor (`X @ coef + intercept`).\n\n    solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs'\n        Algorithm to use in the optimization problem:\n\n        'lbfgs'\n            Calls scipy's L-BFGS-B optimizer.\n\n        'newton-cholesky'\n            Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to\n            iterated reweighted least squares) with an inner Cholesky based solver.\n            This solver is a good choice for `n_samples` >> `n_features`, especially\n            with one-hot encoded categorical features with rare categories. Be aware\n            that the memory usage of this solver has a quadratic dependency on\n            `n_features` because it explicitly computes the Hessian matrix.\n\n            .. versionadded:: 1.2\n\n    max_iter : int, default=100\n        The maximal number of iterations for the solver.\n        Values must be in the range `[1, inf)`.\n\n    tol : float, default=1e-4\n        Stopping criterion. For the lbfgs solver,\n        the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol``\n        where ``g_j`` is the j-th component of the gradient (derivative) of\n        the objective function.\n        Values must be in the range `(0.0, inf)`.\n\n    warm_start : bool, default=False\n        If set to ``True``, reuse the solution of the previous call to ``fit``\n        as initialization for ``coef_`` and ``intercept_`` .\n\n    verbose : int, default=0\n        For the lbfgs solver set verbose to any positive number for verbosity.\n        Values must be in the range `[0, inf)`.\n\n    Attributes\n    ----------\n    coef_ : array of shape (n_features,)\n        Estimated coefficients for the linear predictor (`X @ coef_ +\n        intercept_`) in the GLM.\n\n    intercept_ : float\n        Intercept (a.k.a. bias) added to linear predictor.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    n_iter_ : int\n        Actual number of iterations used in the solver.\n\n    See Also\n    --------\n    TweedieRegressor : Generalized Linear Model with a Tweedie distribution.\n\n    Examples\n    --------\n    >>> from sklearn import linear_model\n    >>> clf = linear_model.PoissonRegressor()\n    >>> X = [[1, 2], [2, 3], [3, 4], [4, 3]]\n    >>> y = [12, 17, 22, 21]\n    >>> clf.fit(X, y)\n    PoissonRegressor()\n    >>> clf.score(X, y)\n    np.float64(0.990)\n    >>> clf.coef_\n    array([0.121, 0.158])\n    >>> clf.intercept_\n    np.float64(2.088)\n    >>> clf.predict([[1, 1], [3, 4]])\n    array([10.676, 21.875])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in design rationale phrases like 'dropping constant terms' and 'increased from default 20', plus some claims about truncated ArrayAPI class.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_06",
    "repo": "scikit_learn",
    "question": "Why does the silhouette score implementation use `X[indices].T[indices].T` for precomputed metrics, divide intra-cluster distances by `label_freqs - 1`, and replace NaN values with zero?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `silhouette_score` function exists primarily as a convenience wrapper around `silhouette_samples` that adds two key capabilities: (1) random subsampling for scalability, and (2) returning the mean over all samples. Several important design decisions are embedded in the implementation:\n\n**Why sampling uses a double-transpose for precomputed metrics:**\nWhen `metric='precomputed'` and `sample_size` is set, the code uses `X[indices].T[indices].T` to correctly subset a symmetric distance matrix \u2014 selecting both rows and columns for the sampled indices. This ensures the resulting submatrix is still a valid pairwise distance matrix. For non-precomputed metrics, only row selection (`X[indices]`) is needed since distances are computed on-the-fly.\n\n**Why intra-cluster distances are divided by (label_freqs - 1) instead of label_freqs:**\nIn `_silhouette_reduce`, inter-cluster distances are divided by `label_freqs` (full cluster size). But in `silhouette_samples`, intra-cluster distances are divided by `(label_freqs - 1)` \u2014 because a sample's distance to its own cluster should not include the self-distance (which is 0), so the denominator is one less. The `(label_freqs - 1)` division is applied once after all chunks are processed, rather than per-chunk in `_silhouette_reduce`.\n\n**Why NaN values are replaced with 0:**\nFor singleton clusters (size 1), dividing by `(label_freqs - 1)` = 0 produces NaN. The code uses `xpx.nan_to_num(sil_samples)` with default `fill_value=0.0` to assign a silhouette coefficient of 0 to these samples.\n\n**Why the diagonal check uses a tolerance for floats but exact zero for ints:**\nFor float dtype precomputed matrices, diagonal entries are checked against `np.finfo(X.dtype).eps * 100` as tolerance, allowing small floating-point errors. For integer dtypes, exact zero is required.\n\n**Why `_silhouette_reduce` uses chunked processing:**\nThe function uses `pairwise_distances_chunked` with `_silhouette_reduce` as a `reduce_func`. The chunk size is determined by `get_chunk_n_rows` in `sklearn/utils/_chunking.py`, which calculates how many rows fit in `working_memory` (default from `sklearn.get_config()['working_memory']`).\n\n**Why `accept_sparse` differs between the two functions:**\n`silhouette_score` accepts both CSC and CSR sparse formats (`accept_sparse=['csc', 'csr']`). `silhouette_samples` accepts only CSR (`accept_sparse=['csr']`). `_silhouette_reduce` explicitly checks for CSR format and raises TypeError if non-CSR sparse is passed.\n\n**Why LabelEncoder is used:**\nLabels are re-encoded via `LabelEncoder` to ensure they are contiguous integers starting at 0. This allows `np.bincount` to work correctly in `_silhouette_reduce` and ensures the `label_freqs` array is properly sized. The test `test_non_encoded_labels` confirms that labels like `labels * 2 + 10` produce the same result as the original labels.",
    "rubric": [
      "Explains that the double-transpose pattern X[indices].T[indices].T correctly subsets both rows and columns of a symmetric pairwise distance matrix when subsampling with precomputed metrics",
      "Explains that intra-cluster distances use (label_freqs - 1) instead of label_freqs because the self-distance (always 0) should be excluded from the mean, reducing the denominator by one",
      "Explains that NaN values arise from singleton clusters where label_freqs - 1 equals 0, causing division by zero, and these are replaced with 0 as the silhouette coefficient for a single-sample cluster",
      "Notes that inter-cluster distances are divided by the full label_freqs inside _silhouette_reduce, while intra-cluster division by (label_freqs - 1) happens afterward in silhouette_samples",
      "Mentions that LabelEncoder is used to re-encode labels as contiguous integers starting from 0 so that np.bincount works correctly"
    ],
    "key_files": [
      "sklearn/metrics/cluster/_unsupervised.py",
      "sklearn/metrics/pairwise.py",
      "sklearn/utils/_chunking.py",
      "sklearn/metrics/cluster/tests/test_unsupervised.py"
    ],
    "source_doc": "[docstring: sklearn/metrics/cluster/_unsupervised.py] sklearn.metrics.cluster._unsupervised.silhouette_score\nCompute the mean Silhouette Coefficient of all samples.\n\n    The Silhouette Coefficient is calculated using the mean intra-cluster\n    distance (``a``) and the mean nearest-cluster distance (``b``) for each\n    sample.  The Silhouette Coefficient for a sample is ``(b - a) / max(a,\n    b)``.  To clarify, ``b`` is the distance between a sample and the nearest\n    cluster that the sample is not a part of.\n    Note that Silhouette Coefficient is only defined if number of labels\n    is ``2 <= n_labels <= n_samples - 1``.\n\n    This function returns the mean Silhouette Coefficient over all samples.\n    To obtain the values for each sample, use :func:`silhouette_samples`.\n\n    The best value is 1 and the worst value is -1. Values near 0 indicate\n    overlapping clusters. Negative values generally indicate that a sample has\n    been assigned to the wrong cluster, as a different cluster is more similar.\n\n    Read more in the :ref:`User Guide <silhouette_coefficient>`.\n\n    Parameters\n    ----------\n    X : {array-like, sparse matrix} of shape (n_samples_a, n_samples_a) if metric ==             \"precomputed\" or (n_samples_a, n_features) otherwise\n        An array of pairwise distances between samples, or a feature array.\n\n    labels : array-like of shape (n_samples,)\n        Predicted labels for each sample.\n\n    metric : str or callable, default='euclidean'\n        The metric to use when calculating distance between instances in a\n        feature array. If metric is a string, it must be one of the options\n        allowed by :func:`~sklearn.metrics.pairwise_distances`. If ``X`` is\n        the distance array itself, use ``metric=\"precomputed\"``.\n\n    sample_size : int, default=None\n        The size of the sample to use when computing the Silhouette Coefficient\n        on a random subset of the data.\n        If ``sample_size is None``, no sampling is used.\n\n    random_state : int, RandomState instance or None, default=None\n        Determines random number generation for selecting a subset of samples.\n        Used when ``sample_size is not None``.\n        Pass an int for reproducible results across multiple function calls.\n        See :term:`Glossary <random_state>`.\n\n    **kwds : optional keyword parameters\n        Any further parameters are passed directly to the distance function.\n        If using a scipy.spatial.distance metric, the parameters are still\n        metric dependent. See the scipy docs for usage examples.\n\n    Returns\n    -------\n    silhouette : float\n        Mean Silhouette Coefficient for all samples.\n\n    References\n    ----------\n\n    .. [1] `Peter J. Rousseeuw (1987). \"Silhouettes: a Graphical Aid to the\n       Interpretation and Validation of Cluster Analysis\". Computational\n       and Applied Mathematics 20: 53-65.\n       <https://www.sciencedirect.com/science/article/pii/0377042787901257>`_\n\n    .. [2] `Wikipedia entry on the Silhouette Coefficient\n           <https://en.wikipedia.org/wiki/Silhouette_(clustering)>`_\n\n    Examples\n    --------\n    >>> from sklearn.datasets import make_blobs\n    >>> from sklearn.cluster import KMeans\n    >>> from sklearn.metrics import silhouette_score\n    >>> X, y = make_blobs(random_state=42)\n    >>> kmeans = KMeans(n_clusters=2, random_state=42)\n    >>> silhouette_score(X, kmeans.fit_predict(X))\n    0.49...",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (specific line numbers, internal function names like _silhouette_reduce, chunking behavior, diagonal checks, LabelEncoder usage, sparse format handling) that are not present in the provided documentation. These appear to be plausible code-level details but cannot be verified from the given docstring alone.",
      "The claim about X[indices].T[indices].T for precomputed metrics is a specific implementation detail not verifiable from the documentation.",
      "The claim about NaN replacement with 0 for singleton clusters references Rousseeuw's convention, which is plausible but not explicitly stated in the provided documentation.",
      "Line number references (143, 198, 315, 323, 292-298, 171-172, 300-301) cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed Rousseeuw convention comment reference, issue number reference, and 'critical memory optimization' judgment; rewrote a few claims to remove doc-only rationale.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_07",
    "repo": "scikit_learn",
    "question": "Where is the silhouette score computation implemented and how does it handle chunked distance calculations and edge cases like single-element clusters?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `silhouette_samples` function is located in `sklearn/metrics/cluster/_unsupervised.py`. It is called by `silhouette_score` in the same file, which simply returns `float(np.mean(silhouette_samples(X, labels, metric=metric, **kwds)))`. The function's core computation is delegated to two key components:\n\n1. **`pairwise_distances_chunked`** (from `sklearn/metrics/pairwise.py`): This generator processes the distance matrix in chunks. Chunk sizes are determined by `get_chunk_n_rows` in `sklearn/utils/_chunking.py`, which computes `int(working_memory * (2**20) // row_bytes)`. If `working_memory` is None, it reads from `sklearn.get_config()['working_memory']`.\n\n2. **`_silhouette_reduce`** (in the same file): This is the reduce function passed via `functools.partial` to `pairwise_distances_chunked`. It accumulates per-cluster distance sums using `np.bincount` with distance weights. For sparse matrices, it requires CSR format and raises a `TypeError` if the format doesn't match. It returns a tuple of `(intra_cluster_distances, inter_cluster_distances)`. Notably, inter-cluster distances are normalized by `label_freqs` inside `_silhouette_reduce`, while intra-cluster distances are normalized *outside* the reduce function by dividing by `(label_freqs - 1).take(labels, mode='clip')`.\n\n3. **NaN handling**: For clusters of size 1, `denom` becomes 0, causing a division by zero that produces NaN. These NaN values are replaced with 0 using `xpx.nan_to_num(sil_samples)` where `xpx` is imported from `sklearn.externals.array_api_extra` (via `sklearn.utils._array_api`). The `nan_to_num` function (in `sklearn/externals/array_api_extra/_lib/_funcs.py`) replaces NaN with 0.0 by default and also replaces infinities with `finfo.max` and `finfo.min`.\n\n4. **Precomputed diagonal check**: When `metric='precomputed'`, the function checks diagonal entries. For float dtypes, the tolerance is `np.finfo(X.dtype).eps * 100` \u2014 values on the diagonal exceeding this absolute tolerance raise a `ValueError`. For integral dtypes, any non-zero diagonal raises the error.\n\n5. **Label encoding**: Labels are transformed via `sklearn.preprocessing.LabelEncoder` (`le = LabelEncoder()` and `labels = le.fit_transform(labels)`) so that labels are re-encoded to contiguous integers before computation.\n\n6. **Validation**: The `@validate_params` decorator validates that `metric` is either in `_VALID_METRICS` (a list of distance metrics defined in `sklearn/metrics/pairwise.py` including 'euclidean', 'l2', 'manhattan', 'cosine', 'nan_euclidean', 'haversine', etc., with some conditional entries based on scipy version) combined with 'precomputed', or a callable.",
    "rubric": [
      "Identifies that silhouette_samples is in sklearn/metrics/cluster/_unsupervised.py",
      "Explains that silhouette_score calls silhouette_samples and returns the mean of its results",
      "Mentions pairwise_distances_chunked is used to process distance matrix in chunks",
      "Explains that chunk size is determined by working_memory configuration via get_chunk_n_rows",
      "Identifies _silhouette_reduce as the reduce function passed to pairwise_distances_chunked",
      "Explains that _silhouette_reduce uses np.bincount to accumulate per-cluster distance sums",
      "Notes the asymmetry: inter-cluster distances are normalized inside _silhouette_reduce while intra-cluster distances are normalized outside by (label_freqs - 1)",
      "Explains that single-element clusters produce NaN (division by zero) which is replaced with 0 using nan_to_num",
      "Mentions that labels are re-encoded using LabelEncoder to contiguous integers",
      "Describes the precomputed diagonal check with dtype-dependent tolerance (eps*100 for floats, non-zero for integers)"
    ],
    "key_files": [
      "sklearn/metrics/cluster/_unsupervised.py",
      "sklearn/metrics/pairwise.py",
      "sklearn/utils/_chunking.py",
      "sklearn/externals/array_api_extra/_lib/_funcs.py"
    ],
    "source_doc": "[docstring: sklearn/metrics/cluster/_unsupervised.py] sklearn.metrics.cluster._unsupervised.silhouette_samples\nCompute the Silhouette Coefficient for each sample.\n\n    The Silhouette Coefficient is a measure of how well samples are clustered\n    with samples that are similar to themselves. Clustering models with a high\n    Silhouette Coefficient are said to be dense, where samples in the same\n    cluster are similar to each other, and well separated, where samples in\n    different clusters are not very similar to each other.\n\n    The Silhouette Coefficient is calculated using the mean intra-cluster\n    distance (``a``) and the mean nearest-cluster distance (``b``) for each\n    sample.  The Silhouette Coefficient for a sample is ``(b - a) / max(a,\n    b)``.\n    Note that Silhouette Coefficient is only defined if number of labels\n    is 2 ``<= n_labels <= n_samples - 1``.\n\n    This function returns the Silhouette Coefficient for each sample.\n\n    The best value is 1 and the worst value is -1. Values near 0 indicate\n    overlapping clusters.\n\n    Read more in the :ref:`User Guide <silhouette_coefficient>`.\n\n    Parameters\n    ----------\n    X : {array-like, sparse matrix} of shape (n_samples_a, n_samples_a) if metric ==             \"precomputed\" or (n_samples_a, n_features) otherwise\n        An array of pairwise distances between samples, or a feature array. If\n        a sparse matrix is provided, CSR format should be favoured avoiding\n        an additional copy.\n\n    labels : array-like of shape (n_samples,)\n        Label values for each sample.\n\n    metric : str or callable, default='euclidean'\n        The metric to use when calculating distance between instances in a\n        feature array. If metric is a string, it must be one of the options\n        allowed by :func:`~sklearn.metrics.pairwise_distances`.\n        If ``X`` is the distance array itself, use \"precomputed\" as the metric.\n        Precomputed distance matrices must have 0 along the diagonal.\n\n    **kwds : optional keyword parameters\n        Any further parameters are passed directly to the distance function.\n        If using a ``scipy.spatial.distance`` metric, the parameters are still\n        metric dependent. See the scipy docs for usage examples.\n\n    Returns\n    -------\n    silhouette : array-like of shape (n_samples,)\n        Silhouette Coefficients for each sample.\n\n    References\n    ----------\n\n    .. [1] `Peter J. Rousseeuw (1987). \"Silhouettes: a Graphical Aid to the\n       Interpretation and Validation of Cluster Analysis\". Computational\n       and Applied Mathematics 20: 53-65.\n       <https://www.sciencedirect.com/science/article/pii/0377042787901257>`_\n\n    .. [2] `Wikipedia entry on the Silhouette Coefficient\n       <https://en.wikipedia.org/wiki/Silhouette_(clustering)>`_\n\n    Examples\n    --------\n    >>> from sklearn.metrics import silhouette_samples\n    >>> from sklearn.datasets import make_blobs\n    >>> from sklearn.cluster import KMeans\n    >>> X, y = make_blobs(n_samples=50, random_state=42)\n    >>> kmeans = KMeans(n_clusters=3, random_state=42)\n    >>> labels = kmeans.fit_predict(X)\n    >>> silhouette_samples(X, labels)\n    array([...])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: one rationale explanation about why cluster_size-1 is used, and some explanatory examples/counts that aren't directly in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_08",
    "repo": "scikit_learn",
    "question": "How does the `_get_doc_link` method generate documentation URLs, and how does the `_doc_link_template` property handle versioning?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `_HTMLDocumentationLinkMixin._get_doc_link()` method works through the following algorithm:\n\n1. **Module check (early return)**: It first splits `self.__class__.__module__` by `.` and compares the first element (index 0) to `self._doc_link_module`. If they don't match, it immediately returns an empty string `\"\"`. This means an estimator from a different root package (e.g., `xgboost` vs `sklearn`) gets no documentation link.\n\n2. **Branch based on `_doc_link_url_param_generator`**:\n   - **If `None` (default path)**: It computes `estimator_name` as `self.__class__.__name__` and constructs `estimator_module` by using `itertools.takewhile` to collect module path parts that do NOT start with `_` (underscore). This strips private submodule names. For example, `sklearn.linear_model._logistic` becomes `sklearn.linear_model` because `_logistic` starts with `_`. It then formats the template with `{estimator_module}` and `{estimator_name}`.\n   - **If a custom callable is provided**: It calls `self._doc_link_url_param_generator()` (no arguments besides `self`) and unpacks the returned dictionary as keyword arguments to `self._doc_link_template.format(**...)`.\n\n3. **`_doc_link_template` is implemented as a property with getter/setter**:\n   - The getter uses `parse_version(__version__)`. It checks `sklearn_version.dev` \u2014 if `dev` is `None` (meaning a release version), the URL uses `\"{major}.{minor}\"` (e.g., `1.6`). If it's a dev version, it uses `\"dev\"` in the URL path.\n   - The getter uses `getattr(self, \"__doc_link_template\", <default>)` to check if a custom template was set, falling back to the default scikit-learn URL template.\n   - The setter stores the value via `setattr(self, \"__doc_link_template\", value)` \u2014 note the name-mangling trick using double-underscore prefix on the attribute name `__doc_link_template` to store the overridden value.\n\n4. **Integration with HTML rendering**: In `sklearn/utils/_repr_html/estimator.py`, the `_write_estimator_html` function checks `hasattr(estimator, \"_get_doc_link\")`. If true, it calls `estimator._get_doc_link()` to get the URL, which is then passed as `doc_link` to `_write_label_html`. There, if `doc_link` is non-empty, an `<a>` tag with CSS class `sk-estimator-doc-link` is created with `rel=\"noreferrer\"` and `target=\"_blank\"`, containing a `?` character and a `<span>` with label text like \"Documentation for {name}\".\n\n5. **Class hierarchy**: `BaseEstimator` in `sklearn/base.py` inherits from `ReprHTMLMixin`, `_HTMLDocumentationLinkMixin`, and `_MetadataRequester`. `BaseEstimator` sets `_html_repr = estimator_html_repr`, connecting the HTML repr system to the documentation link generation.\n\n6. **Version parsing detail**: The `Version.dev` property in `sklearn/externals/_packaging/version.py` returns `self._version.dev[1]` (the integer part) if `self._version.dev` is set, else `None`. The `major` and `minor` properties access `self.release[0]` and `self.release[1]` respectively from the parsed version tuple.",
    "rubric": [
      "Explains the early return: the method checks if the first part of `self.__class__.__module__` matches `_doc_link_module`, returning empty string if not",
      "Describes the default path (when `_doc_link_url_param_generator` is None): uses `itertools.takewhile` to strip private submodule parts (those starting with `_`) from the module path to build `estimator_module`",
      "Describes the custom callable path: calls `_doc_link_url_param_generator()` and unpacks the result as keyword arguments to `_doc_link_template.format(**...)`",
      "Explains that `_doc_link_template` is a property that uses `parse_version(__version__)` to check `sklearn_version.dev` \u2014 uses `dev` in the URL for dev versions, or `{major}.{minor}` for release versions",
      "Mentions the getter/setter pattern: getter checks `getattr(self, '__doc_link_template', <default>)` and setter uses `setattr(self, '__doc_link_template', value)` to allow overriding the template",
      "Mentions how the generated link integrates into HTML rendering: `_write_label_html` creates an `<a>` tag with class `sk-estimator-doc-link`, `rel='noreferrer'`, and `target='_blank'` when `doc_link` is non-empty"
    ],
    "key_files": [
      "sklearn/utils/_repr_html/base.py",
      "sklearn/utils/_repr_html/estimator.py",
      "sklearn/externals/_packaging/version.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/utils/_repr_html/base.py] sklearn.utils._repr_html.base._HTMLDocumentationLinkMixin\nsklearn.utils._repr_html.base._HTMLDocumentationLinkMixin:\n  Mixin class allowing to generate a link to the API documentation.\n\n    This mixin relies on three attributes:\n    - `_doc_link_module`: it corresponds to the root module (e.g. `sklearn`). Using this\n      mixin, the default value is `sklearn`.\n    - `_doc_link_template`: it corresponds to the template used to generate the\n      link to the API documentation. Using this mixin, the default value is\n      `\"https://scikit-learn.org/{version_url}/modules/generated/\n      {estimator_module}.{estimator_name}.html\"`.\n    - `_doc_link_url_param_generator`: it corresponds to a function that generates the\n      parameters to be used in the template when the estimator module and name are not\n      sufficient.\n\n    The method :meth:`_get_doc_link` generates the link to the API documentation for a\n    given estimator.\n\n    This mixin provides all the necessary states for\n    :func:`sklearn.utils.estimator_html_repr` to generate a link to the API\n    documentation for the estimator HTML diagram.\n\n    Examples\n    --------\n    If the default values for `_doc_link_module`, `_doc_link_template` are not suitable,\n    then you can override them and provide a method to generate the URL parameters:\n    >>> from sklearn.base import BaseEstimator\n    >>> doc_link_template = \"https://address.local/{single_param}.html\"\n    >>> def url_param_generator(estimator):\n    ...     return {\"single_param\": estimator.__class__.__name__}\n    >>> class MyEstimator(BaseEstimator):\n    ...     # use \"builtins\" since it is the associated module when declaring\n    ...     # the class in a docstring\n    ...     _doc_link_module = \"builtins\"\n    ...     _doc_link_template = doc_link_template\n    ...     _doc_link_url_param_generator = url_param_generator\n    >>> estimator = MyEstimator()\n    >>> estimator._get_doc_link()\n    'https://address.local/MyEstimator.html'\n\n    If instead of overriding the attributes inside the class definition, you want to\n    override a class instance, you can use `types.MethodType` to bind the method to the\n    instance:\n    >>> import types\n    >>> estimator = BaseEstimator()\n    >>> estimator._doc_link_template = doc_link_template\n    >>> estimator._doc_link_url_param_generator = types.MethodType(\n    ...     url_param_generator, estimator)\n    >>> estimator._get_doc_link()\n    'https://address.local/BaseEstimator.html'\n\nsklearn.utils._repr_html.base._HTMLDocumentationLinkMixin._get_doc_link:\n  Generates a link to the API documentation for a given estimator.\n\n        This method generates the link to the estimator's documentation page\n        by using the template defined by the attribute `_doc_link_template`.\n\n        Returns\n        -------\n        url : str\n            The URL to the API documentation for this estimator. If the estimator does\n            not belong to module `_doc_link_module`, the empty string (i.e. `\"\"`) is\n            returned.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 2 states that `_doc_link_url_param_generator` is called with no arguments besides `self`, but the documentation's example shows it defined as `def url_param_generator(estimator)` taking the estimator as an argument, and the class attribute version would receive `self` as the first argument. The `types.MethodType` binding example also suggests it takes the instance. This is technically consistent but the phrasing 'no arguments besides self' could be misleading since in the class attribute case it's not technically `self` but the estimator parameter.",
      "Points 3, 4, 5, and 6 contain extensive implementation details (property getter/setter mechanics, name-mangling tricks, specific line numbers, CSS classes, HTML rendering details, version parsing internals) that go far beyond what the documentation provides. These are plausible code-level extrapolations but cannot be verified from the provided documentation alone.",
      "The claim about `itertools.takewhile` stripping private submodule names (Point 2) is a plausible implementation detail but is not mentioned in the documentation.",
      "The documentation says `_doc_link_url_param_generator` 'corresponds to a function that generates the parameters to be used in the template when the estimator module and name are not sufficient' \u2014 the answer's description of calling it and unpacking as kwargs is plausible but unverified from the doc."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Very minor doc leakage: 'vendored copy of packaging.version' characterization and specific line numbers are not recoverable from stripped code; all algorithmic claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_09",
    "repo": "scikit_learn",
    "question": "What are all the components and mechanisms that make up the HTML diagram rendering system for scikit-learn estimators?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `estimator_html_repr` function in `sklearn/utils/_repr_html/estimator.py` is the central function that builds a complete HTML representation of a scikit-learn estimator. Here's how it fits into the broader system:\n\n**Architecture & Dependencies:**\n\n1. **BaseEstimator integration**: `BaseEstimator` (in `sklearn/base.py`, line 158) inherits from `ReprHTMLMixin` and `_HTMLDocumentationLinkMixin`. It sets `_html_repr = estimator_html_repr` (line 209), which is how the `ReprHTMLMixin._repr_html_` property and `_repr_mimebundle_` method delegate to this function for Jupyter rendering.\n\n2. **ReprHTMLMixin** (in `sklearn/utils/_repr_html/base.py`, line 118): The `_repr_html_` property checks if `get_config()[\"display\"] == \"diagram\"` and raises `AttributeError` if not. This makes `hasattr(estimator, \"_repr_html_\")` return `True` or `False` depending on the config. The `_repr_mimebundle_` method always includes `text/plain` and only adds `text/html` when display is `\"diagram\"`.\n\n3. **CSS loading**: The `_get_css_style()` function reads and concatenates two separate CSS files: `estimator.css` and `params.css`. The combined CSS is cached in the module-level `_CSS_STYLE` variable at import time. The `estimator.css` defines a color scheme with separate CSS variables for unfitted (orange tones: `#fff5e6`, `#f6e4d2`, `#ffe0b3`, `chocolate`) and fitted estimators (blue tones: `#f0f8ff`, `#d4ebff`, `#b3dbfd`, `cornflowerblue`).\n\n4. **ID generation**: Two module-level `_IDCounter` instances (`_CONTAINER_ID_COUNTER` with prefix `\"sk-container-id\"` and `_ESTIMATOR_ID_COUNTER` with prefix `\"sk-estimator-id\"`) generate sequential unique IDs for HTML elements.\n\n5. **Fitted status detection**: The function uses `check_is_fitted` from `sklearn.utils.validation` (imported lazily inside the function) to determine if the estimator is fitted. If the estimator doesn't have a `fit` method, it's treated as \"Not fitted\".\n\n6. **JavaScript injection**: The function reads `estimator.js` at each call (not cached like CSS) and injects it into the HTML. The JS includes a `detectTheme` function adapted from Skrub that checks VSCode themes (`data-vscode-theme-kind`/`data-vscode-theme-name`), Jupyter themes (`data-jp-theme-light`), computed CSS color luma, and system `prefers-color-scheme`. The `forceTheme(containerId)` function is called for each diagram to apply the detected theme as a CSS class.\n\n7. **Visual block system**: `_get_visual_block` creates `_VisualBlock` instances with `kind` of `'serial'`, `'parallel'`, or `'single'`. Estimators can define `_sk_visual_block_()` \u2014 Pipeline returns a `\"serial\"` block, ColumnTransformer, voting estimators, etc. have custom implementations. If `_sk_visual_block_` raises an exception, it falls back gracefully to a `'single'` block.\n\n8. **Parameter HTML**: `BaseEstimator._get_params_html()` (line 265) returns a `ParamsDict` which inherits from both `ReprHTMLMixin` and `UserDict`. It sorts parameters so non-default parameters appear first. Non-default detection compares against `inspect.signature(self.__init__).parameters` defaults, with special handling for `BaseEstimator` instances (type comparison), pandas NA values, scalar NaN, and `inspect._empty` (no default).\n\n9. **Fitted attributes HTML**: `BaseEstimator._get_fitted_attr_html()` returns an `AttrsDict`. It displays up to 100 fitted attributes (those ending with `_` and not starting with `_`). Array-like attributes (with `shape` and `dtype` but not `numbers.Number`) show shape and dtype info. The `_read_fitted_attr` function in `fitted_attributes.py` uses `reprlib.Repr` with all collection max limits set to 4, `maxstring=9`, and formats floats with `:.4g`.\n\n10. **Documentation links**: `_HTMLDocumentationLinkMixin._get_doc_link()` generates URLs using `itertools.takewhile` to construct the module path up to the first private submodule. The `_doc_link_template` property dynamically builds the URL based on the scikit-learn version (dev vs release). Parameter-level documentation links use Text Fragments (RFC) with `urllib.parse.quote`.\n\n11. **Docstring scraping**: `common.py` uses `sklearn.externals._numpydoc.docscrape.NumpyDocString` with `@lru_cache` to parse docstrings for parameter tooltips.\n\n12. **Fallback message**: The HTML includes a `sk-text-repr-fallback` div shown by default (with the plain `repr()` of the estimator), which CSS hides when loaded. The `sk-container` div is hidden by default and shown by CSS \u2014 this handles untrusted notebooks gracefully.\n\n13. **Recursive rendering**: `_write_estimator_html` is called recursively for nested estimators. For non-first calls, `is_fitted_icon` is reset to empty string, and `config_context(print_changed_only=True)` is used. For parallel blocks, each child is wrapped in a serial `_VisualBlock` with `dash_wrapped=False`. A `param_prefix` is built up and passed down for nested parameter names (e.g., `\"pipeline__stepname__\"`).",
    "rubric": [
      "Identifies that BaseEstimator inherits from ReprHTMLMixin and _HTMLDocumentationLinkMixin, and sets _html_repr = estimator_html_repr",
      "Explains that ReprHTMLMixin._repr_html_ checks get_config()['display'] == 'diagram' and raises AttributeError if not, controlling hasattr behavior for Jupyter",
      "Describes CSS loading: _get_css_style() concatenates estimator.css and params.css, cached in module-level _CSS_STYLE variable",
      "Mentions the two _IDCounter instances (_CONTAINER_ID_COUNTER with 'sk-container-id' and _ESTIMATOR_ID_COUNTER with 'sk-estimator-id') for unique HTML element IDs",
      "Explains fitted status detection using check_is_fitted (lazily imported) with fallback to 'Not fitted' when estimator lacks a fit method",
      "Describes the JavaScript theme detection system (estimator.js read at each call, not cached) including VSCode, Jupyter, CSS luma, and prefers-color-scheme checks",
      "Explains the _VisualBlock system with 'serial', 'parallel', and 'single' kinds, and the _get_visual_block fallback when _sk_visual_block_ raises an exception",
      "Describes ParamsDict (inheriting ReprHTMLMixin and UserDict) with non-default parameter sorting and detection logic including special handling for BaseEstimator instances, pandas NA, scalar NaN, and inspect._empty",
      "Explains AttrsDict and _get_fitted_attr_html: up to 100 fitted attributes (ending with _ but not starting with _), with array-like attributes showing shape/dtype",
      "Describes _read_fitted_attr using reprlib.Repr with collection max limits of 4, maxstring=9, and float formatting with :.4g",
      "Mentions _HTMLDocumentationLinkMixin._get_doc_link() using itertools.takewhile to construct module paths, with version-dependent URL templates",
      "Describes common.py docstring scraping with NumpyDocString and @lru_cache, plus Text Fragment URL generation using urllib.parse.quote",
      "Explains the fallback mechanism: sk-text-repr-fallback div shown by default with plain repr(), hidden by CSS when loaded; sk-container div hidden by default and shown by CSS",
      "Describes recursive rendering via _write_estimator_html: is_fitted_icon reset to empty for non-first calls, config_context(print_changed_only=True) used, parallel items wrapped in serial _VisualBlock(dash_wrapped=False), and param_prefix built up for nested names"
    ],
    "key_files": [
      "sklearn/utils/_repr_html/estimator.py",
      "sklearn/base.py",
      "sklearn/utils/_repr_html/base.py",
      "sklearn/utils/_repr_html/params.py",
      "sklearn/utils/_repr_html/fitted_attributes.py",
      "sklearn/utils/_repr_html/common.py",
      "sklearn/utils/_repr_html/estimator.js"
    ],
    "source_doc": "[docstring: sklearn/utils/_repr_html/estimator.py] sklearn.utils._repr_html.estimator.estimator_html_repr\nBuild an HTML representation of an estimator.\n\n    Read more in the :ref:`User Guide <visualizing_composite_estimators>`.\n\n    Parameters\n    ----------\n    estimator : estimator object\n        The estimator to visualize.\n\n    Returns\n    -------\n    html: str\n        HTML representation of estimator.\n\n    Examples\n    --------\n    >>> from sklearn.utils._repr_html.estimator import estimator_html_repr\n    >>> from sklearn.linear_model import LogisticRegression\n    >>> estimator_html_repr(LogisticRegression())\n    '<style>.sk-global...'",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_11",
    "repo": "scikit_learn",
    "question": "Where is `config_context` defined, how does it manage state, and where is it consumed across the codebase?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `config_context` function is defined in `sklearn/_config.py` and is exposed at the top level of sklearn (other modules import it via `from sklearn._config import config_context, get_config, set_config`). It is implemented as a generator-based context manager using Python's `@contextmanager` decorator (imported as `from contextlib import contextmanager as contextmanager`).\n\n**Implementation mechanics:**\n- `config_context` saves the current config via `old_config = get_config()`, calls `set_config(...)` with the new values, yields, and in the `finally` block restores by calling `set_config(**old_config)` \u2014 meaning the old config dict is unpacked as keyword arguments to `set_config()`.\n- Configuration is stored per-thread using `threading.local()`. The private function `_get_threadlocal_config()` checks `hasattr(_threadlocal, 'global_config')` and copies `_global_config` into it on first access.\n- `get_config()` returns a **copy** of the threadlocal config (`return _get_threadlocal_config().copy()`), so callers cannot mutate internal state directly.\n- `config_context` accepts only keyword arguments (the `*` in the signature enforces this); passing positional arguments raises `TypeError`.\n\n**Three environment variables** can override global defaults at import time:\n- `SKLEARN_ASSUME_FINITE` \u2192 `assume_finite` (cast with `bool()`)\n- `SKLEARN_WORKING_MEMORY` \u2192 `working_memory` (cast with `int()`, default 1024)\n- `SKLEARN_PAIRWISE_DIST_CHUNK_SIZE` \u2192 `pairwise_dist_chunk_size` (cast with `int()`, default 256)\n\n**Where config_context is consumed in the codebase:**\n\n1. **`sklearn/utils/parallel.py`**: The `_FuncWrapper.__call__` method uses `with config_context(**config)` to propagate the scikit-learn thread-local config to joblib worker processes. The custom `Parallel.__call__` captures the config via `get_config()` and attaches it to each delayed function.\n\n2. **`sklearn/utils/_param_validation.py`**: The `validate_params` decorator uses `config_context(skip_parameter_validation=...)` to disable validation in nested function calls after the outer function has already validated.\n\n3. **`sklearn/utils/_pprint.py`**: The `_EstimatorPrettyPrinter.__init__` reads `get_config()[\"print_changed_only\"]` to decide whether to show only changed parameters.\n\n4. **`sklearn/utils/_repr_html/base.py`**: The `ReprHTMLMixin._repr_html_` property checks `get_config()[\"display\"] != \"diagram\"` and raises `AttributeError` if display is not 'diagram', making `hasattr(estimator, \"_repr_html_\")` return False when display mode is 'text'.\n\n5. **`sklearn/utils/_sparse.py`**: The `_align_api_if_sparse` function reads `get_config()[\"sparse_interface\"]` and converts sparse objects between `sparray` and `spmatrix` formats using `getattr(sp.sparse, X.format + \"_array\")` or `getattr(sp.sparse, X.format + \"_matrix\")`.\n\n6. **`sklearn/utils/_chunking.py`**: `get_chunk_n_rows` reads `get_config()[\"working_memory\"]` when the `working_memory` parameter is None.\n\n**Validation of `array_api_dispatch`**: When `set_config(array_api_dispatch=True)` is called, it lazily imports and calls `_check_array_api_dispatch` from `sklearn/utils/_array_api.py`, which requires SciPy >= 1.14.0 and the environment variable `SCIPY_ARRAY_API=1` to be set, otherwise raising `ImportError` or `RuntimeError`.",
    "rubric": [
      "Identifies that config_context is defined in sklearn/_config.py and uses the @contextmanager decorator from contextlib",
      "Explains that it saves old config via get_config(), calls set_config() with new values, yields, and restores via set_config(**old_config) in a finally block",
      "Mentions that configuration is stored per-thread using threading.local() (the _threadlocal object)",
      "Notes that get_config() returns a copy of the threadlocal config to prevent direct mutation",
      "Mentions that config_context accepts only keyword arguments (enforced by the * in the signature)",
      "Identifies usage in sklearn/utils/parallel.py where _FuncWrapper.__call__ uses config_context(**config) to propagate config to joblib workers",
      "Identifies usage in sklearn/utils/_param_validation.py where validate_params uses config_context(skip_parameter_validation=...) to control nested validation",
      "Mentions environment variables (SKLEARN_ASSUME_FINITE, SKLEARN_WORKING_MEMORY, SKLEARN_PAIRWISE_DIST_CHUNK_SIZE) that override global defaults at import time"
    ],
    "key_files": [
      "sklearn/_config.py",
      "sklearn/utils/parallel.py",
      "sklearn/base.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/utils/_sparse.py",
      "sklearn/utils/_repr_html/base.py",
      "sklearn/utils/_pprint.py",
      "sklearn/utils/_chunking.py",
      "sklearn/utils/_array_api.py"
    ],
    "source_doc": "[docstring: sklearn/_config.py] sklearn._config.config_context\nContext manager to temporarily change the global scikit-learn configuration.\n\n    This context manager can be used to apply scikit-learn configuration changes within\n    the scope of the with statement. Once the context exits, the global configuration is\n    restored again.\n\n    The default global configurations (which take effect when scikit-learn is imported)\n    are defined below in the parameter list.\n\n    Parameters\n    ----------\n    assume_finite : bool, default=None\n        If True, validation for finiteness will be skipped,\n        saving time, but leading to potential crashes. If\n        False, validation for finiteness will be performed,\n        avoiding error. If None, the existing configuration won't change.\n        Global default: False.\n\n    working_memory : int, default=None\n        If set, scikit-learn will attempt to limit the size of temporary arrays\n        to this number of MiB (per job when parallelised), often saving both\n        computation time and memory on expensive operations that can be\n        performed in chunks. If None, the existing configuration won't change.\n        Global default: 1024.\n\n    print_changed_only : bool, default=None\n        If True, only the parameters that were set to non-default\n        values will be printed when printing an estimator. For example,\n        ``print(SVC())`` while True will only print 'SVC()', but would print\n        'SVC(C=1.0, cache_size=200, ...)' with all the non-changed parameters\n        when False. If None, the existing configuration won't change.\n        Global default: True.\n\n        .. versionchanged:: 0.23\n           Global default configuration changed from False to True.\n\n    display : {'text', 'diagram'}, default=None\n        If 'diagram', estimators will be displayed as a diagram in a Jupyter\n        lab or notebook context. If 'text', estimators will be displayed as\n        text. If None, the existing configuration won't change.\n        Global default: 'diagram'.\n\n        .. versionadded:: 0.23\n\n    pairwise_dist_chunk_size : int, default=None\n        The number of row vectors per chunk for the accelerated pairwise-\n        distances reduction backend. Global default: 256 (suitable for most of\n        modern laptops' caches and architectures).\n\n        Intended for easier benchmarking and testing of scikit-learn internals.\n        End users are not expected to benefit from customizing this configuration\n        setting.\n\n        .. versionadded:: 1.1\n\n    enable_cython_pairwise_dist : bool, default=None\n        Use the accelerated pairwise-distances reduction backend when\n        possible. Global default: True.\n\n        Intended for easier benchmarking and testing of scikit-learn internals.\n        End users are not expected to benefit from customizing this configuration\n        setting.\n\n        .. versionadded:: 1.1\n\n    array_api_dispatch : bool, default=None\n        Use Array API dispatching when inputs follow the Array API standard.\n        Global default: False.\n\n        See the :ref:`User Guide <array_api>` for more details.\n\n        .. versionadded:: 1.2\n\n    transform_output : str, default=None\n        Configure output of `transform` and `fit_transform`.\n\n        See :ref:`sphx_glr_auto_examples_miscellaneous_plot_set_output.py`\n        for an example on how to use the API.\n\n        - `\"default\"`: Default output format of a transformer\n        - `\"pandas\"`: DataFrame output\n        - `\"polars\"`: Polars output\n        - `None`: Transform configuration is unchanged\n\n        Global default: \"default\".\n\n        .. versionadded:: 1.2\n        .. versionadded:: 1.4\n            `\"polars\"` option was added.\n\n    enable_metadata_routing : bool, default=None\n        Enable metadata routing. By default this feature is disabled.\n\n        Refer to :ref:`metadata routing user guide <metadata_routing>` for more\n        details.\n\n        - `True`: Metadata routing is enabled\n        - `False`: Metadata routing is disabled, use the old syntax.\n        - `None`: Configuration is unchanged\n\n        Global default: False.\n\n        .. versionadded:: 1.3\n\n    skip_parameter_validation : bool, default=None\n        If `True`, disable the validation of the hyper-parameters' types and values in\n        the fit method of estimators and for arguments passed to public helper\n        functions. It can save time in some situations but can lead to low level\n        crashes and exceptions with confusing error messages.\n        Global default: False.\n\n        Note that for data parameters, such as `X` and `y`, only type validation is\n        skipped but validation with `check_array` will continue to run.\n\n        .. versionadded:: 1.3\n\n    sparse_interface : str, default=\"spmatrix\"\n\n        The sparse interface used for every sparse object that scikit-learn produces,\n        e.g., function returns, estimator attributes, estimator properties, etc.\n\n        - `\"sparray\"`: Return sparse as SciPy sparse array\n        - `\"spmatrix\"`: Return sparse as SciPy sparse matrix\n\n        .. versionadded:: 1.8\n\n    Yields\n    ------\n    None.\n\n    See Also\n    --------\n    set_config : Set global scikit-learn configuration.\n    get_config : Retrieve current values of the global configuration.\n\n    Notes\n    -----\n    All settings, not just those presently modified, will be returned to\n    their previous values when the context manager is exited.\n\n    Examples\n    --------\n    >>> import sklearn\n    >>> from sklearn.utils.validation import assert_all_finite\n    >>> with sklearn.config_context(assume_finite=True):\n    ...     assert_all_finite([float('nan')])\n    >>> with sklearn.config_context(assume_finite=True):\n    ...     with sklearn.config_context(assume_finite=False):\n    ...         assert_all_finite([float('nan')])\n    Traceback (most recent call last):\n    ...\n    ValueError: Input contains NaN...",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only specific line numbers and one reference to _fit_context (not visible in truncated code) needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_10",
    "repo": "scikit_learn",
    "question": "Why is scikit-learn's global configuration system designed the way it is, and how do the various config options short-circuit expensive operations for performance?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "**Thread-safety via `threading.local()`**: Configuration is stored in a thread-local variable (`_threadlocal = threading.local()`). The `_get_threadlocal_config()` function lazily copies the `_global_config` dict into per-thread storage if it doesn't exist yet. This design ensures thread-safety through the use of `threading.local()`.\n\n**`assume_finite` skips all finiteness validation**: In `sklearn/utils/validation.py`, `_assert_all_finite()` checks `_get_config()[\"assume_finite\"]` as its very first operation and immediately returns if True, completely bypassing the `xp.isfinite(xp.sum(X))` check plus the element-wise fallback.\n\n**`skip_parameter_validation` with nested propagation**: The `validate_params` decorator in `sklearn/utils/_param_validation.py` checks `get_config()[\"skip_parameter_validation\"]` and returns immediately if True, skipping validation. After validating parameters, it uses `config_context(skip_parameter_validation=(prefer_skip_nested_validation or global_skip_validation))` to automatically skip validation for nested function calls.\n\n**`working_memory` controls chunking**: In `sklearn/utils/_chunking.py`, `get_chunk_n_rows()` calculates how many rows fit in the configured MiB via `int(working_memory * (2**20) // row_bytes)`, with a minimum of 1 row and a warning if even one row exceeds the budget.\n\n**`enable_cython_pairwise_dist` gates the accelerated backend**: In `sklearn/metrics/_pairwise_distances_reduction/_dispatcher.py`, the dispatcher's `is_usable_for` method checks `get_config().get(\"enable_cython_pairwise_dist\", True)` as the first condition in a compound boolean. If disabled, the dispatcher reports the reduction as not usable.\n\n**Environment variable overrides for 3 settings**: Three config values can be overridden via environment variables at import time: `SKLEARN_ASSUME_FINITE`, `SKLEARN_WORKING_MEMORY`, and `SKLEARN_PAIRWISE_DIST_CHUNK_SIZE`. The other 8 settings cannot be set via environment variables.\n\n**`array_api_dispatch` has an active validation gate**: Unlike other settings that are simply stored, setting `array_api_dispatch=True` triggers `_check_array_api_dispatch()` from `sklearn/utils/_array_api.py`, which verifies SciPy >= 1.14.0 and that `SCIPY_ARRAY_API=1` environment variable is set, raising `ImportError` or `RuntimeError` otherwise.\n\n**`get_config()` returns a copy**: `get_config()` returns `_get_threadlocal_config().copy()`. In contrast, `set_config()` gets the mutable reference directly via `_get_threadlocal_config()` and mutates it in-place.\n\n**`config_context` restores via `set_config(**old_config)`**: The context manager saves old config before entering, then in its `finally` block unpacks the entire old config dict back into `set_config()`. This means ALL settings are restored, not just the ones that were changed.\n\n**Parallel worker config propagation**: `sklearn/utils/parallel.py` contains a custom `Parallel` subclass that captures `get_config()` at dispatch time in its `__call__` method and propagates it to workers via `_FuncWrapper`, which applies `config_context(**config)` around each function call in the worker.",
    "rubric": [
      "Explains that thread-safety is achieved via threading.local(), with _get_threadlocal_config() lazily copying _global_config into per-thread storage",
      "Explains that assume_finite causes _assert_all_finite() to return immediately before any finiteness checks (xp.isfinite, element-wise fallback)",
      "Explains that skip_parameter_validation causes the validate_params decorator to skip validation and propagates the skip to nested calls via config_context",
      "Explains that working_memory controls chunking via get_chunk_n_rows(), computing int(working_memory * (2**20) // row_bytes) with a minimum of 1 row",
      "Explains that enable_cython_pairwise_dist is checked as the first condition in is_usable_for(), gating the accelerated pairwise distances backend",
      "Identifies that three config values (assume_finite, working_memory, pairwise_dist_chunk_size) can be overridden via environment variables at import time",
      "Explains that array_api_dispatch triggers an active validation gate (_check_array_api_dispatch) checking SciPy version and SCIPY_ARRAY_API env var",
      "Explains that get_config() returns a copy of the threadlocal config while set_config() mutates it in place",
      "Explains that config_context restores all settings (not just changed ones) via set_config(**old_config) in a finally block",
      "Explains that parallel worker config propagation is handled by a custom Parallel subclass capturing config at dispatch time and _FuncWrapper applying config_context around each worker call"
    ],
    "key_files": [
      "sklearn/_config.py",
      "sklearn/utils/validation.py",
      "sklearn/base.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/utils/parallel.py",
      "sklearn/utils/_chunking.py",
      "sklearn/metrics/_pairwise_distances_reduction/_dispatcher.py",
      "sklearn/utils/_sparse.py",
      "sklearn/utils/_array_api.py"
    ],
    "source_doc": "[docstring: sklearn/_config.py] sklearn._config.set_config\nSet global scikit-learn configuration.\n\n    These settings control the behaviour of scikit-learn functions during a library\n    usage session. Global configuration defaults (as described in the parameter list\n    below) take effect when scikit-learn is imported.\n\n    This function can be used to modify the global scikit-learn configuration at\n    runtime. Passing `None` as an argument (the default) leaves the corresponding\n    setting unchanged. This allows users to selectively update the global configuration\n    values without affecting the others.\n\n    .. versionadded:: 0.19\n\n    Parameters\n    ----------\n    assume_finite : bool, default=None\n        If True, validation for finiteness will be skipped,\n        saving time, but leading to potential crashes. If\n        False, validation for finiteness will be performed,\n        avoiding error. Global default: False.\n\n        .. versionadded:: 0.19\n\n    working_memory : int, default=None\n        If set, scikit-learn will attempt to limit the size of temporary arrays\n        to this number of MiB (per job when parallelised), often saving both\n        computation time and memory on expensive operations that can be\n        performed in chunks. Global default: 1024.\n\n        .. versionadded:: 0.20\n\n    print_changed_only : bool, default=None\n        If True, only the parameters that were set to non-default\n        values will be printed when printing an estimator. For example,\n        ``print(SVC())`` while True will only print 'SVC()' while the default\n        behaviour would be to print 'SVC(C=1.0, cache_size=200, ...)' with\n        all the non-changed parameters. Global default: True.\n\n        .. versionadded:: 0.21\n        .. versionchanged:: 0.23\n           Global default configuration changed from False to True.\n\n    display : {'text', 'diagram'}, default=None\n        If 'diagram', estimators will be displayed as a diagram in a Jupyter\n        lab or notebook context. If 'text', estimators will be displayed as\n        text. Global default: 'diagram'.\n\n        .. versionadded:: 0.23\n\n    pairwise_dist_chunk_size : int, default=None\n        The number of row vectors per chunk for the accelerated pairwise-\n        distances reduction backend. Global default: 256 (suitable for most of\n        modern laptops' caches and architectures).\n\n        Intended for easier benchmarking and testing of scikit-learn internals.\n        End users are not expected to benefit from customizing this configuration\n        setting.\n\n        .. versionadded:: 1.1\n\n    enable_cython_pairwise_dist : bool, default=None\n        Use the accelerated pairwise-distances reduction backend when\n        possible. Global default: True.\n\n        Intended for easier benchmarking and testing of scikit-learn internals.\n        End users are not expected to benefit from customizing this configuration\n        setting.\n\n        .. versionadded:: 1.1\n\n    array_api_dispatch : bool, default=None\n        Use Array API dispatching when inputs follow the Array API standard.\n        Global default: False.\n\n        See the :ref:`User Guide <array_api>` for more details.\n\n        .. versionadded:: 1.2\n\n    transform_output : str, default=None\n        Configure output of `transform` and `fit_transform`.\n\n        See :ref:`sphx_glr_auto_examples_miscellaneous_plot_set_output.py`\n        for an example on how to use the API.\n\n        - `\"default\"`: Default output format of a transformer\n        - `\"pandas\"`: DataFrame output\n        - `\"polars\"`: Polars output\n        - `None`: Transform configuration is unchanged\n\n        Global default: \"default\".\n\n        .. versionadded:: 1.2\n        .. versionadded:: 1.4\n            `\"polars\"` option was added.\n\n    enable_metadata_routing : bool, default=None\n        Enable metadata routing. By default this feature is disabled.\n\n        Refer to :ref:`metadata routing user guide <metadata_routing>` for more\n        details.\n\n        - `True`: Metadata routing is enabled\n        - `False`: Metadata routing is disabled, use the old syntax.\n        - `None`: Configuration is unchanged\n\n        Global default: False.\n\n        .. versionadded:: 1.3\n\n    skip_parameter_validation : bool, default=None\n        If `True`, disable the validation of the hyper-parameters' types and values in\n        the fit method of estimators and for arguments passed to public helper\n        functions. It can save time in some situations but can lead to low level\n        crashes and exceptions with confusing error messages.\n        Global default: False.\n\n        Note that for data parameters, such as `X` and `y`, only type validation is\n        skipped but validation with `check_array` will continue to run.\n\n        .. versionadded:: 1.3\n\n    sparse_interface : str, default=\"spmatrix\"\n\n        The sparse interface used for every sparse object that scikit-learn produces,\n        e.g., function returns, estimator attributes, estimator properties, etc.\n\n        - `\"sparray\"`: Return sparse as SciPy sparse array\n        - `\"spmatrix\"`: Return sparse as SciPy sparse matrix\n\n        .. versionadded:: 1.9\n\n    See Also\n    --------\n    config_context : Context manager for global scikit-learn configuration.\n    get_config : Retrieve current values of the global configuration.\n\n    Examples\n    --------\n    >>> from sklearn import set_config\n    >>> set_config(display='diagram')  # doctest: +SKIP",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references many implementation details (threading.local, line numbers, specific file paths, environment variable overrides, array_api_dispatch validation gate, parallel worker propagation, etc.) that go far beyond what the provided documentation covers. These cannot be verified from the doc alone, but they are plausible code-level details.",
      "The claim about 3 environment variable overrides (SKLEARN_ASSUME_FINITE, SKLEARN_WORKING_MEMORY, SKLEARN_PAIRWISE_DIST_CHUNK_SIZE) cannot be verified from the documentation provided, but is plausible.",
      "The claim about array_api_dispatch requiring SciPy >= 1.14.0 and SCIPY_ARRAY_API=1 environment variable cannot be verified from the documentation and may be version-specific or inaccurate.",
      "The answer mentions '8 settings' that cannot be set via environment variables, implying 11 total settings. The documentation lists 12 parameters (assume_finite, working_memory, print_changed_only, display, pairwise_dist_chunk_size, enable_cython_pairwise_dist, array_api_dispatch, transform_output, enable_metadata_routing, skip_parameter_validation, sparse_interface), so 3 + 8 = 11 doesn't match 12.",
      "The answer does not mention the `sparse_interface` parameter at all, which is documented in the provided documentation (added in version 1.9).",
      "All claims about the documented parameters' defaults and behaviors (assume_finite default False, working_memory default 1024, print_changed_only default True, display default 'diagram', etc.) are consistent with the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed pure rationale statements and unverifiable references to truncated code/line numbers/test names.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_12",
    "repo": "scikit_learn",
    "question": "How does sklearn's `clone` function dispatch and handle cloning for estimators, containers, and non-estimator objects?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `sklearn.base.clone` function implements a two-step dispatch mechanism to create copies of estimators:\n\n**Step 1: Protocol check (`__sklearn_clone__`)**\n`clone` first checks if the estimator has a `__sklearn_clone__` method AND is not a class (using `inspect.isclass(estimator)`). If both conditions are met, it delegates entirely to `estimator.__sklearn_clone__()`, bypassing the parametrized clone logic. The `safe` parameter is ignored in this path.\n\n**Step 2: Fallback to `_clone_parametrized`**\nOtherwise, it calls `_clone_parametrized(estimator, safe=safe)`, which handles the actual cloning logic:\n\n1. **Container types**: Dicts are recursively cloned key-by-key. Lists, tuples, sets, and frozensets are cloned element-by-element, preserving the container type via `estimator_type([clone(e, safe=safe) for e in estimator])`.\n\n2. **Non-estimator check**: If the object lacks `get_params` or is a class type, it either performs `copy.deepcopy` (when `safe=False`) or raises `TypeError` with distinct messages for classes vs non-estimators.\n\n3. **Core estimator cloning**: For actual estimators, it:\n   - Calls `estimator.get_params(deep=False)` to get shallow parameters\n   - Recursively clones each parameter value with `clone(param, safe=False)` \u2014 note `safe=False` is used so non-estimator parameter values get deep-copied\n   - Instantiates the new object via `klass(**new_object_params)` using `estimator.__class__`\n   - Deep-copies `_metadata_request` attribute if it exists\n   - Performs an identity-based sanity check (`param1 is not param2`) comparing the cloned parameters against the newly constructed object's parameters to verify the constructor properly stored them\n   - Preserves `_sklearn_output_config` via deep copy\n\n**Key implementations of `__sklearn_clone__`**:\n- `BaseEstimator.__sklearn_clone__` simply delegates back to `_clone_parametrized(self)`\n- `FrozenEstimator.__sklearn_clone__` returns `self` (identity), making clone a no-op\n\n**`get_params` internals**: The `_get_param_names` classmethod introspects the `__init__` signature using `inspect.signature`, excludes `self`, rejects `VAR_POSITIONAL` (`*args`) parameters (raising RuntimeError), and returns sorted parameter names. Parameters with `VAR_KEYWORD` (`**kwargs`) kind are also excluded.\n\n**Usage in the ecosystem**: In `model_selection/_validation.py`, `clone(estimator)` is called before each cross-validation fold in `cross_validate`. Gaussian process kernels use it in `clone_with_theta` to create a kernel copy with modified hyperparameters.",
    "rubric": [
      "Explains the __sklearn_clone__ protocol check: if the estimator has __sklearn_clone__ and is not a class, it delegates to that method",
      "Explains that otherwise it falls back to _clone_parametrized(estimator, safe=safe)",
      "Describes container handling: dicts are cloned key-by-key, and lists/tuples/sets/frozensets are cloned element-by-element preserving the container type",
      "Describes non-estimator handling: objects without get_params or class types get deep-copied when safe=False, or raise TypeError when safe=True",
      "Describes core estimator cloning: calls get_params(deep=False), recursively clones each param with safe=False, then instantiates the class with the cloned params",
      "Mentions the sanity check that verifies each parameter in the new object is identity-equal to the one passed in",
      "Mentions that _metadata_request and _sklearn_output_config are deep-copied if present",
      "Describes BaseEstimator.__sklearn_clone__ delegating back to _clone_parametrized and FrozenEstimator.__sklearn_clone__ returning self"
    ],
    "key_files": [
      "sklearn/base.py",
      "sklearn/frozen/_frozen.py",
      "sklearn/model_selection/_validation.py",
      "sklearn/gaussian_process/kernels.py"
    ],
    "source_doc": "[docstring: sklearn/base.py] sklearn.base.clone\nConstruct a new unfitted estimator with the same parameters.\n\n    Clone does a deep copy of the model in an estimator\n    without actually copying attached data. It returns a new estimator\n    with the same parameters that has not been fitted on any data.\n\n    .. versionchanged:: 1.3\n        Delegates to `estimator.__sklearn_clone__` if the method exists.\n\n    Parameters\n    ----------\n    estimator : {list, tuple, set} of estimator instance or a single             estimator instance\n        The estimator or group of estimators to be cloned.\n    safe : bool, default=True\n        If safe is False, clone will fall back to a deep copy on objects\n        that are not estimators. Ignored if `estimator.__sklearn_clone__`\n        exists.\n\n    Returns\n    -------\n    estimator : object\n        The deep copy of the input, an estimator if input is an estimator.\n\n    Notes\n    -----\n    If the estimator's `random_state` parameter is an integer (or if the\n    estimator doesn't have a `random_state` parameter), an *exact clone* is\n    returned: the clone and the original estimator will give the exact same\n    results. Otherwise, *statistical clone* is returned: the clone might\n    return different results from the original estimator. More details can be\n    found in :ref:`randomness`.\n\n    Examples\n    --------\n    >>> from sklearn.base import clone\n    >>> from sklearn.linear_model import LogisticRegression\n    >>> X = [[-1, 0], [0, 1], [0, -1], [1, 0]]\n    >>> y = [0, 0, 1, 1]\n    >>> classifier = LogisticRegression().fit(X, y)\n    >>> cloned_classifier = clone(classifier)\n    >>> hasattr(classifier, \"classes_\")\n    True\n    >>> hasattr(cloned_classifier, \"classes_\")\n    False\n    >>> classifier is cloned_classifier\n    False",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in purpose/rationale descriptions and a reference to sklearn/__init__.py which isn't shown.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_15",
    "repo": "scikit_learn",
    "question": "Where does the similarity data flow through the affinity propagation implementation, from the public API function down to the actual message-passing algorithm?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The public `affinity_propagation` function in `sklearn/cluster/_affinity_propagation.py` does NOT directly implement the algorithm. Instead, it creates an `AffinityPropagation` estimator instance with `affinity='precomputed'` and calls `.fit(S)` on it. The `AffinityPropagation.fit()` method then delegates to the private `_affinity_propagation()` function, always passing `return_n_iter=True` to get `n_iter_` for the estimator attribute.\n\nThe data flow is:\n1. `affinity_propagation()` (public API) \u2192 creates `AffinityPropagation(affinity='precomputed').fit(S)`\n2. `AffinityPropagation.fit()` \u2192 validates data via `validate_data()`, computes preference as `np.median(self.affinity_matrix_)` if None, calls `check_random_state()`, then calls `_affinity_propagation()`\n3. `_affinity_propagation()` (private) \u2192 contains the actual message-passing algorithm\n\nKey implementation details in `_affinity_propagation()`:\n- Before the main loop, noise is added: `S += (np.finfo(S.dtype).eps * S + np.finfo(S.dtype).tiny * 100) * random_state.standard_normal(size=(n_samples, n_samples))`\n- The algorithm initializes three `n_samples \u00d7 n_samples` matrices: A, R, and tmp, all as zeros\n- Convergence is tracked via an array `e` of shape `(n_samples, convergence_iter)` used as a circular buffer (via `it % convergence_iter`) storing exemplar indicators (E = (np.diag(A) + np.diag(R)) > 0) per iteration\n- Convergence is checked only after `convergence_iter` iterations, by verifying that every sample's exemplar status has been consistently either True or False across the last `convergence_iter` iterations AND at least one exemplar exists (K > 0)\n- If n_samples == 1 or all similarities and preferences are equal (checked by `_equal_similarities_and_preferences()`), the algorithm short-circuits: returns all samples as cluster centers if preference > similarity, otherwise returns a single cluster center at index 0\n- When K > 0 after the loop but the algorithm never converged, a `ConvergenceWarning` is issued; when K == 0, a different `ConvergenceWarning` is issued and labels are set to all `-1`\n- After finding exemplars, labels are refined: for each cluster, the exemplar is re-selected as the sample maximizing the sum of similarities within the cluster\n- Labels are then reduced to a sorted gapless list using `np.unique` and `np.searchsorted`\n\nThe `AffinityPropagation` class inherits from `ClusterMixin` and `BaseEstimator`. `ClusterMixin` (in `sklearn/base.py`) provides `fit_predict()` which calls `self.fit(X)` then returns `self.labels_`. The `AffinityPropagation.fit_predict()` method explicitly delegates to `super().fit_predict(X, y)`.\n\nThe `predict()` method uses `pairwise_distances_argmin(X, self.cluster_centers_)` from `sklearn.metrics.pairwise` wrapped in `config_context(assume_finite=True)`, and raises `ValueError` if `cluster_centers_` doesn't exist (which occurs when `affinity='precomputed'`, since `fit()` only sets `cluster_centers_` when affinity is not 'precomputed').\n\nThe `__sklearn_tags__` method sets `input_tags.pairwise = True` when affinity is 'precomputed' and `input_tags.sparse = True` when affinity is NOT 'precomputed'.",
    "rubric": [
      "Explains that the public `affinity_propagation()` function creates an `AffinityPropagation` estimator with `affinity='precomputed'` and calls `.fit(S)` on it",
      "Explains that `AffinityPropagation.fit()` validates data, computes preference as `np.median(self.affinity_matrix_)` if None, and delegates to the private `_affinity_propagation()` function with `return_n_iter=True`",
      "Describes the private `_affinity_propagation()` function as containing the actual message-passing algorithm with A (availability), R (responsibility), and tmp matrices",
      "Mentions the noise addition step using `np.finfo(S.dtype).eps` and `random_state.standard_normal` to remove degeneracies",
      "Describes the convergence tracking using a circular buffer array `e` of shape `(n_samples, convergence_iter)` that stores exemplar indicators per iteration",
      "Mentions the short-circuit path when n_samples == 1 or all similarities and preferences are equal (via `_equal_similarities_and_preferences`)",
      "Describes the post-loop exemplar refinement step where each cluster's exemplar is re-selected as the sample maximizing the sum of intra-cluster similarities",
      "Mentions that labels are reduced to a sorted gapless list using `np.unique` and `np.searchsorted`"
    ],
    "key_files": [
      "sklearn/cluster/_affinity_propagation.py",
      "sklearn/base.py",
      "sklearn/cluster/__init__.py",
      "sklearn/exceptions.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_affinity_propagation.py] sklearn.cluster._affinity_propagation.affinity_propagation\nPerform Affinity Propagation Clustering of data.\n\n    Read more in the :ref:`User Guide <affinity_propagation>`.\n\n    Parameters\n    ----------\n    S : array-like of shape (n_samples, n_samples)\n        Matrix of similarities between points.\n\n    preference : array-like of shape (n_samples,) or float, default=None\n        Preferences for each point - points with larger values of\n        preferences are more likely to be chosen as exemplars. The number of\n        exemplars, i.e. of clusters, is influenced by the input preferences\n        value. If the preferences are not passed as arguments, they will be\n        set to the median of the input similarities (resulting in a moderate\n        number of clusters). For a smaller amount of clusters, this can be set\n        to the minimum value of the similarities.\n\n    convergence_iter : int, default=15\n        Number of iterations with no change in the number\n        of estimated clusters that stops the convergence.\n\n    max_iter : int, default=200\n        Maximum number of iterations.\n\n    damping : float, default=0.5\n        Damping factor between 0.5 and 1.\n\n    copy : bool, default=True\n        If copy is False, the affinity matrix is modified inplace by the\n        algorithm, for memory efficiency.\n\n    verbose : bool, default=False\n        The verbosity level.\n\n    return_n_iter : bool, default=False\n        Whether or not to return the number of iterations.\n\n    random_state : int, RandomState instance or None, default=None\n        Pseudo-random number generator to control the starting state.\n        Use an int for reproducible results across function calls.\n        See the :term:`Glossary <random_state>`.\n\n        .. versionadded:: 0.23\n            this parameter was previously hardcoded as 0.\n\n    Returns\n    -------\n    cluster_centers_indices : ndarray of shape (n_clusters,)\n        Index of clusters centers.\n\n    labels : ndarray of shape (n_samples,)\n        Cluster labels for each point.\n\n    n_iter : int\n        Number of iterations run. Returned only if `return_n_iter` is\n        set to True.\n\n    Notes\n    -----\n    For an example usage,\n    see :ref:`sphx_glr_auto_examples_cluster_plot_affinity_propagation.py`.\n    You may also check out,\n    :ref:`sphx_glr_auto_examples_applications_plot_stock_market.py`\n\n    When the algorithm does not converge, it will still return an array of\n    ``cluster_center_indices`` and labels if there are any exemplars/clusters,\n    however they may be degenerate and should be used with caution.\n\n    When all training samples have equal similarities and equal preferences,\n    the assignment of cluster centers and labels depends on the preference.\n    If the preference is smaller than the similarities, a single cluster center\n    and label ``0`` for every sample will be returned. Otherwise, every\n    training sample becomes its own cluster center and is assigned a unique\n    label.\n\n    References\n    ----------\n    Brendan J. Frey and Delbert Dueck, \"Clustering by Passing Messages\n    Between Data Points\", Science Feb. 2007\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import affinity_propagation\n    >>> from sklearn.metrics.pairwise import euclidean_distances\n    >>> X = np.array([[1, 2], [1, 4], [1, 0],\n    ...               [4, 2], [4, 4], [4, 0]])\n    >>> S = -euclidean_distances(X, squared=True)\n    >>> cluster_centers_indices, labels = affinity_propagation(S, random_state=0)\n    >>> cluster_centers_indices\n    array([0, 3])\n    >>> labels\n    array([0, 0, 0, 1, 1, 1])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only line number references and a few documentation-derived rationale phrases needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_13",
    "repo": "scikit_learn",
    "question": "What is the full inheritance hierarchy and architecture of BaseEstimator, including its parent classes, key methods, and how repr/cloning/metadata routing work?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `BaseEstimator` class in scikit-learn (defined in `sklearn/base.py`) inherits from three parent classes in this order: `ReprHTMLMixin`, `_HTMLDocumentationLinkMixin`, and `_MetadataRequester`. Here's how the architecture connects:\n\n**Inheritance Chain:**\n- `ReprHTMLMixin` (from `sklearn/utils/_repr_html/base.py`) provides `_repr_html_` property and `_repr_mimebundle_` method. It checks `get_config()['display'] == 'diagram'` before rendering HTML. The `_html_repr` attribute on BaseEstimator is set to the `estimator_html_repr` function.\n- `_HTMLDocumentationLinkMixin` (from `sklearn/utils/_repr_html/base.py`) provides the `_get_doc_link()` method. It has a `_doc_link_module` class attribute defaulting to `\"sklearn\"` and a `_doc_link_template` property that dynamically builds a URL based on the sklearn version (using \"dev\" for dev versions).\n- `_MetadataRequester` (from `sklearn/utils/_metadata_requests.py`) is imported and used as a parent class. SIMPLE_METHODS defines the list of methods for metadata routing.\n\n**Key Implementation Details:**\n1. `_get_param_names()` is a classmethod that uses `inspect.signature` on `__init__`, excludes 'self' and VAR_KEYWORD parameters, raises RuntimeError for VAR_POSITIONAL, and returns parameter names **sorted alphabetically**.\n2. `get_params(deep=True)` recursively gets sub-estimator params using double-underscore notation (key + \"__\" + sub_key) when `deep=True` and the value has a `get_params` method.\n3. `set_params` uses `defaultdict(dict)` for nested parameters, partitions keys on \"__\", and calls `set_params` recursively on sub-estimators.\n4. `__repr__` uses N_CHAR_MAX=700 as the default and N_MAX_ELEMENTS_TO_SHOW=30. It uses `_EstimatorPrettyPrinter` (extends `pprint.PrettyPrinter`) with `compact=True, indent=1, indent_at_name=True`. If output exceeds N_CHAR_MAX non-blank characters, it uses regex-based bruteforce ellipsis.\n5. `__getstate__` raises TypeError if `__slots__` is used. For sklearn-internal estimators (module starts with \"sklearn.\"), it injects `_sklearn_version` into the state dict.\n6. `__setstate__` defaults the version to `\"pre-0.18\"` if `_sklearn_version` is missing and raises `InconsistentVersionWarning` when versions don't match.\n7. `__sklearn_tags__` returns a `Tags` dataclass with `estimator_type=None` and `target_tags=TargetTags(required=False)`, with all optional tag fields (transformer_tags, regressor_tags, classifier_tags) set to None.\n8. `__dir__` filters attributes by calling `hasattr(self, attr)` on each entry from `super().__dir__()` while suppressing FutureWarning.\n9. `_validate_params()` delegates to `validate_parameter_constraints()` which iterates over `self._parameter_constraints` dict and calls `make_constraint()` on each constraint. Parameters not in `_parameter_constraints` are silently skipped.\n10. `_get_fitted_attr_html` displays up to 100 fitted attributes (those ending with '_' but not starting with '_') and shows array-like attributes with shape/dtype info.\n11. SIMPLE_METHODS in metadata routing are: fit, partial_fit, predict, predict_proba, predict_log_proba, decision_function, score, split, transform, inverse_transform.\n12. `__sklearn_clone__` simply delegates to `_clone_parametrized(self)`. The clone function also copies `_metadata_request` via deepcopy and `_sklearn_output_config` if present.\n13. `_EstimatorPrettyPrinter` respects the `print_changed_only` config option - when True, only non-default parameters are shown in `__repr__`.",
    "rubric": [
      "Identifies the three parent classes in order: ReprHTMLMixin, _HTMLDocumentationLinkMixin, and _MetadataRequester",
      "Explains that ReprHTMLMixin provides _repr_html_ property that checks get_config()['display'] == 'diagram' before rendering HTML",
      "Explains that _HTMLDocumentationLinkMixin provides _get_doc_link() with _doc_link_module defaulting to 'sklearn' and dynamic URL building based on version",
      "Explains _get_param_names() uses inspect.signature on __init__, excludes 'self' and VAR_KEYWORD, raises RuntimeError for VAR_POSITIONAL, returns sorted names",
      "Explains get_params(deep=True) recursively gets sub-estimator params using double-underscore notation",
      "Explains set_params uses defaultdict(dict) for nested parameters, splitting on '__'",
      "Explains __repr__ uses N_CHAR_MAX=700 and N_MAX_ELEMENTS_TO_SHOW=30 with _EstimatorPrettyPrinter and bruteforce ellipsis for long output",
      "Explains __getstate__ raises TypeError for __slots__ and injects _sklearn_version for sklearn-internal estimators",
      "Explains __setstate__ defaults version to 'pre-0.18' if missing and raises InconsistentVersionWarning on mismatch",
      "Explains __sklearn_tags__ returns a Tags dataclass with estimator_type=None and target_tags with required=False",
      "Explains __sklearn_clone__ delegates to _clone_parametrized",
      "Mentions that _EstimatorPrettyPrinter respects print_changed_only config option",
      "Identifies SIMPLE_METHODS list for metadata routing (fit, partial_fit, predict, predict_proba, predict_log_proba, decision_function, score, split, transform, inverse_transform)"
    ],
    "key_files": [
      "sklearn/base.py",
      "sklearn/utils/_repr_html/base.py",
      "sklearn/utils/_metadata_requests.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/utils/_tags.py",
      "sklearn/utils/_pprint.py"
    ],
    "source_doc": "[docstring: sklearn/base.py] sklearn.base.BaseEstimator\nsklearn.base.BaseEstimator:\n  Base class for all estimators in scikit-learn.\n\n    Inheriting from this class provides default implementations of:\n\n    - setting and getting parameters used by `GridSearchCV` and friends;\n    - textual and HTML representation displayed in terminals and IDEs;\n    - estimator serialization;\n    - parameters validation;\n    - data validation;\n    - feature names validation.\n\n    Read more in the :ref:`User Guide <rolling_your_own_estimator>`.\n\n\n    Notes\n    -----\n    All estimators should specify all the parameters that can be set\n    at the class level in their ``__init__`` as explicit keyword\n    arguments (no ``*args`` or ``**kwargs``).\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.base import BaseEstimator\n    >>> class MyEstimator(BaseEstimator):\n    ...     def __init__(self, *, param=1):\n    ...         self.param = param\n    ...     def fit(self, X, y=None):\n    ...         self.is_fitted_ = True\n    ...         return self\n    ...     def predict(self, X):\n    ...         return np.full(shape=X.shape[0], fill_value=self.param)\n    >>> estimator = MyEstimator(param=2)\n    >>> estimator.get_params()\n    {'param': 2}\n    >>> X = np.array([[1, 2], [2, 3], [3, 4]])\n    >>> y = np.array([1, 0, 1])\n    >>> estimator.fit(X, y).predict(X)\n    array([2, 2, 2])\n    >>> estimator.set_params(param=3).fit(X, y).predict(X)\n    array([3, 3, 3])\n\nsklearn.base.BaseEstimator._get_param_names:\n  Get parameter names for the estimator\n\nsklearn.base.BaseEstimator.get_params:\n  Get parameters for this estimator.\n\n        Parameters\n        ----------\n        deep : bool, default=True\n            If True, will return the parameters for this estimator and\n            contained subobjects that are estimators.\n\n        Returns\n        -------\n        params : dict\n            Parameter names mapped to their values.\n\nsklearn.base.BaseEstimator._get_params_html:\n  Get parameters for this estimator with a specific HTML representation.\n\n        Parameters\n        ----------\n        deep : bool, default=True\n            If True, will return the parameters for this estimator and\n            contained subobjects that are estimators.\n\n        doc_link : str\n            URL to the estimator documentation.\n            Used for linking to the estimator's parameters documentation\n            available in HTML displays.\n\n        Returns\n        -------\n        params : ParamsDict\n            Parameter names mapped to their values. We return a `ParamsDict`\n            dictionary, which renders a specific HTML representation in table\n            form.\n\nsklearn.base.BaseEstimator._get_fitted_attr_html:\n  Get fitted attributes of the estimator.\n\nsklearn.base.BaseEstimator.set_params:\n  Set the parameters of this estimator.\n\n        The method works on simple estimators as well as on nested objects\n        (such as :class:`~sklearn.pipeline.Pipeline`). The latter have\n        parameters of the form ``<component>__<parameter>`` so that it's\n        possible to update each component of a nested object.\n\n        Parameters\n        ----------\n        **params : dict\n            Estimator parameters.\n\n        Returns\n        -------\n        self : estimator instance\n            Estimator instance.\n\nsklearn.base.BaseEstimator._validate_params:\n  Validate types and values of constructor parameters\n\n        The expected type and values must be defined in the `_parameter_constraints`\n        class attribute, which is a dictionary `param_name: list of constraints`. See\n        the docstring of `validate_parameter_constraints` for a description of the\n        accepted constraints.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims BaseEstimator is 'defined at line 158 of sklearn/base.py' \u2014 this specific line number is not in the documentation and cannot be verified from it.",
      "The answer claims BaseEstimator inherits from 'ReprHTMLMixin', '_HTMLDocumentationLinkMixin', and '_MetadataRequester' \u2014 the documentation does not mention these parent classes. This is plausible code-level detail but not verifiable from the doc.",
      "The answer provides extensive implementation details (line numbers, specific constants like N_CHAR_MAX=700, N_MAX_ELEMENTS_TO_SHOW=30, regex-based bruteforce ellipsis, __slots__ TypeError, _sklearn_version injection, etc.) that are not present in the documentation. These are plausible code-level details but cannot be verified.",
      "The answer states '_validate_params() delegates to validate_parameter_constraints()' \u2014 the doc says it validates against '_parameter_constraints' and references 'validate_parameter_constraints' docstring, which is consistent.",
      "The answer states 'Parameters not in _parameter_constraints are silently skipped' in point 9 \u2014 the documentation says it validates parameters defined in _parameter_constraints but doesn't explicitly say others are skipped. This is a plausible but unverifiable extrapolation.",
      "The answer's claim about _fit_context decorator, SIMPLE_METHODS list, __sklearn_clone__, and many other implementation details go far beyond what the documentation covers. While these are likely accurate code-level details, they cannot be verified from the provided doc chunk.",
      "The documentation explicitly states parameters should be 'explicit keyword arguments (no *args or **kwargs)' in __init__. The answer's point 1 about _get_param_names raising RuntimeError for VAR_POSITIONAL is consistent with this spirit but adds unverifiable detail.",
      "The answer's description of get_params and set_params behavior (points 2-3) is broadly consistent with the documentation's description of deep parameter and nested object handling."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed available_if design rationale and unverifiable line numbers/PEP references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_14",
    "repo": "scikit_learn",
    "question": "Why does `calibration_curve` use `searchsorted` on `bins[1:-1]` instead of the full bins array, and why does it filter out empty bins afterward?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `calibration_curve` function uses `np.searchsorted(bins[1:-1], y_prob)` (line 1330) rather than `np.digitize` or `np.searchsorted` on the full bins array. By passing only the internal bin edges (`bins[1:-1]`), the function effectively creates an inclusive first and last bin, ensuring that values exactly at 0.0 or 1.0 are always assigned to a valid bin (bin 0 or bin n_bins-1 respectively). The default `side='left'` parameter means values exactly on an internal bin edge go into the right bin, while `KBinsDiscretizer` (in `sklearn/preprocessing/_discretization.py:466`) uses `side='right'` for a different boundary convention.\n\nThe function filters out empty bins (`nonzero = bin_total != 0`) to avoid division-by-zero errors in subsequent computation. This is why the returned arrays may have fewer than `n_bins` values, since with uniform binning over [0,1] many bins can be empty when predictions cluster in a narrow range.\n\nThe function converts `y_true` to a boolean array via `y_true = y_true == pos_label` (line 1317) after the pos_label is determined. This allows the function to use `np.bincount` with `y_true` as weights to sum the number of positive samples per bin, since True/False get treated as 1/0 in arithmetic operations.\n\nThe `_check_pos_label_consistency` helper (in `sklearn/utils/validation.py:2557`) defaults `pos_label=1` only when labels are in the set {0, 1}, {-1, 1}, {0}, {-1}, or {1}. For string-typed targets or labels with more than 2 classes or labels not in these recognized sets, it raises a ValueError requiring explicit specification of `pos_label`.\n\nThe function is decorated with `@validate_params` (line 1215) which validates that `n_bins` is an integer \u2265 1 (using `Interval(Integral, 1, None, closed='left')`) and `strategy` is one of {'uniform', 'quantile'} (using `StrOptions`). Despite this decorator validation, the function body still contains an `else` clause (line 1324-1328) that raises ValueError for invalid strategy, since the `@validate_params` decorator already prevents invalid strategy values from reaching the function body.\n\nThe 'quantile' strategy computes bin edges using `np.percentile(y_prob, quantiles * 100)` where quantiles are linearly spaced from 0 to 1, meaning the bin edges are data-dependent and adapt to the distribution of predicted probabilities.",
    "rubric": [
      "Explains that slicing to `bins[1:-1]` (internal edges only) makes the first and last bins inclusive, so values exactly at 0.0 or 1.0 always land in a valid bin (bin 0 or bin n_bins-1).",
      "Notes that the default `side='left'` means values on an internal edge go into the right/higher bin, contrasting with `KBinsDiscretizer` which uses `side='right'`.",
      "Explains that filtering with `nonzero = bin_total != 0` avoids division-by-zero when computing `prob_true` and `prob_pred`, which is why the returned arrays may have fewer than `n_bins` entries.",
      "Mentions that `y_true` is converted to boolean via `y_true == pos_label`, allowing `np.bincount` with `y_true` as weights to sum positives per bin (True=1, False=0).",
      "Describes how `_check_pos_label_consistency` defaults `pos_label=1` only for recognized label sets ({0,1}, {-1,1}, {0}, {-1}, {1}), raising ValueError otherwise.",
      "Notes that the 'quantile' strategy uses `np.percentile` with linearly spaced quantiles so bin edges adapt to the predicted probability distribution."
    ],
    "key_files": [
      "sklearn/calibration.py",
      "sklearn/utils/validation.py",
      "sklearn/utils/_plotting.py",
      "sklearn/preprocessing/_discretization.py"
    ],
    "source_doc": "[docstring: sklearn/calibration.py] sklearn.calibration.calibration_curve\nCompute true and predicted probabilities for a calibration curve.\n\n    The method assumes the inputs come from a binary classifier, and\n    discretize the [0, 1] interval into bins.\n\n    Calibration curves may also be referred to as reliability diagrams.\n\n    Read more in the :ref:`User Guide <calibration>`.\n\n    Parameters\n    ----------\n    y_true : array-like of shape (n_samples,)\n        True targets.\n\n    y_prob : array-like of shape (n_samples,)\n        Probabilities of the positive class.\n\n    pos_label : int, float, bool or str, default=None\n        The label of the positive class.\n\n        .. versionadded:: 1.1\n\n    n_bins : int, default=5\n        Number of bins to discretize the [0, 1] interval. A bigger number\n        requires more data. Bins with no samples (i.e. without\n        corresponding values in `y_prob`) will not be returned, thus the\n        returned arrays may have less than `n_bins` values.\n\n    strategy : {'uniform', 'quantile'}, default='uniform'\n        Strategy used to define the widths of the bins.\n\n        uniform\n            The bins have identical widths.\n        quantile\n            The bins have the same number of samples and depend on `y_prob`.\n\n    Returns\n    -------\n    prob_true : ndarray of shape (n_bins,) or smaller\n        The proportion of samples whose class is the positive class, in each\n        bin (fraction of positives).\n\n    prob_pred : ndarray of shape (n_bins,) or smaller\n        The mean predicted probability in each bin.\n\n    See Also\n    --------\n    CalibrationDisplay.from_predictions : Plot calibration curve using true\n        and predicted labels.\n    CalibrationDisplay.from_estimator : Plot calibration curve using an\n        estimator and data.\n\n    References\n    ----------\n    Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good\n    Probabilities With Supervised Learning, in Proceedings of the 22nd\n    International Conference on Machine Learning (ICML).\n    See section 4 (Qualitative Analysis of Predictions).\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.calibration import calibration_curve\n    >>> y_true = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1])\n    >>> y_pred = np.array([0.1, 0.2, 0.3, 0.4, 0.65, 0.7, 0.8, 0.9,  1.])\n    >>> prob_true, prob_pred = calibration_curve(y_true, y_pred, n_bins=3)\n    >>> prob_true\n    array([0. , 0.5, 1. ])\n    >>> prob_pred\n    array([0.2  , 0.525, 0.85 ])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., line 1330, 1336, 1317, 1215, 1324-1328) and implementation details like `np.searchsorted(bins[1:-1], y_prob)`, `np.bincount`, `np.percentile`, `@validate_params` decorator, `Interval(Integral, 1, None, closed='left')`, and `_check_pos_label_consistency` helper \u2014 none of these are mentioned in the provided documentation. These are plausible code-level details but cannot be verified from the docstring alone.",
      "The answer claims `_check_pos_label_consistency` defaults `pos_label=1` only for specific label sets like {0, 1}, {-1, 1}, etc. \u2014 this is not in the documentation and cannot be verified.",
      "The answer claims the function body has redundant defensive code for invalid strategy values alongside the `@validate_params` decorator \u2014 this is a code-level claim not verifiable from the docstring.",
      "The answer's claims about `side='left'` vs `side='right'` and comparison with `KBinsDiscretizer` are not in the documentation and cannot be verified.",
      "The core documented behavior is accurately reflected: n_bins default is 5, strategy options are 'uniform' and 'quantile', empty bins are filtered out, returned arrays may have fewer than n_bins values, and the function works with binary classification probabilities."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals of design rationale language and one pure documentation-only usage claim.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_16",
    "repo": "scikit_learn",
    "question": "How does the AffinityPropagation algorithm's message-passing loop work, and how does the class handle convergence and edge cases?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The AffinityPropagation clustering algorithm in scikit-learn is implemented through a class that inherits from `ClusterMixin` and `BaseEstimator` (in that MRO order), with the core iterative algorithm encapsulated in the private `_affinity_propagation` function.\n\n## How the Algorithm Works (Implementation Details)\n\n### Initialization and Degeneracy Removal\n1. When `n_samples == 1` or all similarities and preferences are equal (checked via `_equal_similarities_and_preferences`), the algorithm short-circuits: if preference > similarity, each sample becomes its own cluster (returns `n_iter=0`); otherwise, a single cluster centered at index 0 is returned.\n\n2. Preferences are placed on the diagonal of the similarity matrix S: `S.flat[:: (n_samples + 1)] = preference`.\n\n3. Random noise is added to S using the formula: `S += (np.finfo(S.dtype).eps * S + np.finfo(S.dtype).tiny * 100) * random_state.standard_normal(size=(n_samples, n_samples))`.\n\n### Message Passing Loop\nThe algorithm maintains three matrices:\n- **R**: initialized to zeros\n- **A**: initialized to zeros\n- **tmp**: a working buffer reused across computations\n\nEach iteration:\n1. **Compute responsibilities**: `tmp = A + S`, find the max and second-max per row, then `R_new[i,k] = S[i,k] - max(A[i,k'] + S[i,k'])` for k' != k. Damping is applied as: `R = damping * R + (1 - damping) * R_new`.\n\n2. **Compute availabilities**: First clip R at 0 (except diagonal), sum columns, then negate. Clip positive off-diagonal elements, preserve diagonal. Damping: `A = damping * A - (1 - damping) * (-A_new)`.\n\n3. **Check convergence**: Exemplars are identified where `diag(A) + diag(R) > 0`. A circular buffer `e` of shape `(n_samples, convergence_iter)` tracks exemplar assignments. Convergence occurs when each sample's exemplar status has been constant for `convergence_iter` iterations AND at least one exemplar exists.\n\n### Post-Processing\nAfter convergence (or max_iter reached):\n- Exemplars are refined: for each cluster k, the point with the maximum total similarity to all cluster members is selected as the true exemplar.\n- Labels are assigned via `argmax(S[:, I])` where I are exemplar indices.\n- Labels are reduced to a sorted, gapless list using `np.searchsorted(np.unique(labels), labels)`.\n\n### Class-Level Design\n- The `__sklearn_tags__` method sets `input_tags.pairwise = True` when `affinity=='precomputed'` and `input_tags.sparse = True` when `affinity != 'precomputed'` (i.e., sparse input is NOT supported for precomputed affinity).\n- The `fit` method uses `validate_data` with `force_writeable=True` for precomputed affinity, and `accept_sparse='csr'` for euclidean affinity.\n- The `predict` method uses `pairwise_distances_argmin` wrapped in `config_context(assume_finite=True)`.\n- `fit_predict` explicitly delegates to `super().fit_predict(X, y)`, which in `ClusterMixin` simply calls `self.fit(X)` and returns `self.labels_`.\n- The public function `affinity_propagation()` internally creates an `AffinityPropagation` instance with `affinity='precomputed'` and calls `.fit(S)`.\n- The `_parameter_constraints` dict validates `damping` with `Interval(Real, 0.5, 1.0, closed='left')` \u2014 meaning 0.5 is included but 1.0 is excluded.\n- When preference is None in `fit()`, it defaults to `np.median(self.affinity_matrix_)`.\n- `cluster_centers_` is only set when `affinity != 'precomputed'`, and is populated by indexing X with the cluster center indices and copying.\n- When the algorithm doesn't converge and K==0 (no exemplars), labels are set to `[-1] * n_samples` and `cluster_centers_indices` is an empty list.",
    "rubric": [
      "Inherits from ClusterMixin and BaseEstimator (in that MRO order)",
      "Core algorithm is in the private _affinity_propagation function",
      "Degeneracy check: if n_samples==1 or all similarities/preferences equal, short-circuits to return either n_samples clusters or a single cluster at index 0",
      "Preferences are placed on the diagonal of S; random noise is added using eps*S + tiny*100 multiplied by standard_normal",
      "Maintains R (responsibility) and A (availability) matrices, both initialized to zeros",
      "Responsibilities: computed as S[i,k] minus the max of (A+S) over other columns, with damping applied",
      "Availabilities: R is clipped at 0 (except diagonal), summed by columns, negated, off-diagonal positives clipped, then damped",
      "Convergence uses a circular buffer e of shape (n_samples, convergence_iter) tracking exemplar status; converges when all samples have stable exemplar status for convergence_iter iterations and K>0",
      "Post-convergence refinement: for each cluster, the point with max total similarity to cluster members becomes the exemplar",
      "Labels are made gapless via np.searchsorted(np.unique(labels), labels)",
      "When no exemplars found (K==0), labels are set to [-1]*n_samples and cluster_centers_indices is empty list",
      "damping parameter is validated with Interval(Real, 0.5, 1.0, closed='left') \u2014 0.5 included, 1.0 excluded",
      "Default preference (when None) is np.median of the affinity matrix",
      "cluster_centers_ is only set when affinity != 'precomputed'",
      "predict uses pairwise_distances_argmin with config_context(assume_finite=True)"
    ],
    "key_files": [
      "sklearn/cluster/_affinity_propagation.py",
      "sklearn/base.py",
      "sklearn/cluster/tests/test_affinity_propagation.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_affinity_propagation.py] sklearn.cluster._affinity_propagation.AffinityPropagation\nsklearn.cluster._affinity_propagation.AffinityPropagation:\n  Perform Affinity Propagation Clustering of data.\n\n    Read more in the :ref:`User Guide <affinity_propagation>`.\n\n    Parameters\n    ----------\n    damping : float, default=0.5\n        Damping factor in the range `[0.5, 1.0)` is the extent to\n        which the current value is maintained relative to\n        incoming values (weighted 1 - damping). This in order\n        to avoid numerical oscillations when updating these\n        values (messages).\n\n    max_iter : int, default=200\n        Maximum number of iterations.\n\n    convergence_iter : int, default=15\n        Number of iterations with no change in the number\n        of estimated clusters that stops the convergence.\n\n    copy : bool, default=True\n        Make a copy of input data.\n\n    preference : array-like of shape (n_samples,) or float, default=None\n        Preferences for each point - points with larger values of\n        preferences are more likely to be chosen as exemplars. The number\n        of exemplars, ie of clusters, is influenced by the input\n        preferences value. If the preferences are not passed as arguments,\n        they will be set to the median of the input similarities.\n\n    affinity : {'euclidean', 'precomputed'}, default='euclidean'\n        Which affinity to use. At the moment 'precomputed' and\n        ``euclidean`` are supported. 'euclidean' uses the\n        negative squared euclidean distance between points.\n\n    verbose : bool, default=False\n        Whether to be verbose.\n\n    random_state : int, RandomState instance or None, default=None\n        Pseudo-random number generator to control the starting state.\n        Use an int for reproducible results across function calls.\n        See the :term:`Glossary <random_state>`.\n\n        .. versionadded:: 0.23\n            this parameter was previously hardcoded as 0.\n\n    Attributes\n    ----------\n    cluster_centers_indices_ : ndarray of shape (n_clusters,)\n        Indices of cluster centers.\n\n    cluster_centers_ : ndarray of shape (n_clusters, n_features)\n        Cluster centers (if affinity != ``precomputed``).\n\n    labels_ : ndarray of shape (n_samples,)\n        Labels of each point.\n\n    affinity_matrix_ : ndarray of shape (n_samples, n_samples)\n        Stores the affinity matrix used in ``fit``.\n\n    n_iter_ : int\n        Number of iterations taken to converge.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    AgglomerativeClustering : Recursively merges the pair of\n        clusters that minimally increases a given linkage distance.\n    FeatureAgglomeration : Similar to AgglomerativeClustering,\n        but recursively merges features instead of samples.\n    KMeans : K-Means clustering.\n    MiniBatchKMeans : Mini-Batch K-Means clustering.\n    MeanShift : Mean shift clustering using a flat kernel.\n    SpectralClustering : Apply clustering to a projection\n        of the normalized Laplacian.\n\n    Notes\n    -----\n    The algorithmic complexity of affinity propagation is quadratic\n    in the number of points.\n\n    When the algorithm does not converge, it will still return an array of\n    ``cluster_center_indices`` and labels if there are any exemplars/clusters,\n    however they may be degenerate and should be used with caution.\n\n    When ``fit`` does not converge, ``cluster_centers_`` is still populated\n    however it may be degenerate. In such a case, proceed with caution.\n    If ``fit`` does not converge and fails to produce any ``cluster_centers_``\n    then ``predict`` will label every sample as ``-1``.\n\n    When all training samples have equal similarities and equal preferences,\n    the assignment of cluster centers and labels depends on the preference.\n    If the preference is smaller than the similarities, ``fit`` will result in\n    a single cluster center and label ``0`` for every sample. Otherwise, every\n    training sample becomes its own cluster center and is assigned a unique\n    label.\n\n    References\n    ----------\n\n    Brendan J. Frey and Delbert Dueck, \"Clustering by Passing Messages\n    Between Data Points\", Science Feb. 2007\n\n    Examples\n    --------\n    >>> from sklearn.cluster import AffinityPropagation\n    >>> import numpy as np\n    >>> X = np.array([[1, 2], [1, 4], [1, 0],\n    ...               [4, 2], [4, 4], [4, 0]])\n    >>> clustering = AffinityPropagation(random_state=5).fit(X)\n    >>> clustering\n    AffinityPropagation(random_state=5)\n    >>> clustering.labels_\n    array([0, 0, 0, 1, 1, 1])\n    >>> clustering.predict([[0, 0], [4, 4]])\n    array([0, 1])\n    >>> clustering.cluster_centers_\n    array([[1, 2],\n           [4, 2]])\n\n    For an example usage,\n    see :ref:`sphx_glr_auto_examples_cluster_plot_affinity_propagation.py`.\n\n    For a comparison of Affinity Propagation with other clustering algorithms, see\n    :ref:`sphx_glr_auto_examples_cluster_plot_cluster_comparison.py`\n\nsklearn.cluster._affinity_propagation.AffinityPropagation.fit:\n  Fit the clustering from features, or affinity matrix.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features), or                 array-like of shape (n_samples, n_samples)\n            Training instances to cluster, or similarities / affinities between\n            instances if ``affinity='precomputed'``. If a sparse feature matrix\n            is provided, it will be converted into a sparse ``csr_matrix``.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        Returns\n        -------\n        self\n            Returns the instance itself.\n\nsklearn.cluster._affinity_propagation.AffinityPropagation.predict:\n  Predict the closest cluster each sample in X belongs to.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            New data to predict. If a sparse matrix is provided, it will be\n            converted into a sparse ``csr_matrix``.\n\n        Returns\n        -------\n        labels : ndarray of shape (n_samples,)\n            Cluster labels.\n\nsklearn.cluster._affinity_propagation.AffinityPropagation.fit_predict:\n  Fit clustering from features/affinity matrix; return cluster labels.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features), or                 array-like of shape (n_samples, n_samples)\n            Training instances to cluster, or similarities / affinities between\n            instances if ``affinity='precomputed'``. If a sparse feature matrix\n            is provided, it will be converted into a sparse ``csr_matrix``.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        Returns\n        -------\n        labels : ndarray of shape (n_samples,)\n            Cluster labels.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many implementation details (e.g., random noise formula, message passing equations, circular buffer convergence check, __sklearn_tags__ method, pairwise_distances_argmin usage, _parameter_constraints dict) are not verifiable from the provided documentation alone, but they are plausible code-level details that don't contradict the documentation.",
      "The claim about damping range 'Interval(Real, 0.5, 1.0, closed=\"left\")' is consistent with the documentation stating damping is in range [0.5, 1.0).",
      "The claim about preference defaulting to median of input similarities matches the documentation.",
      "The claim about cluster_centers_ only being set when affinity != 'precomputed' is consistent with the documentation stating 'Cluster centers (if affinity != precomputed)'.",
      "The claim about non-convergence behavior (labels set to -1, etc.) is consistent with the Notes section of the documentation.",
      "The claim about fit_predict delegating to super().fit_predict is a plausible implementation detail consistent with the documented behavior."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor purpose explanations ('to remove degeneracies', 'for performance') were trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_17",
    "repo": "scikit_learn",
    "question": "What are the key components and code paths of the `ward_tree` function and how does it integrate with `AgglomerativeClustering`?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `ward_tree` function in `sklearn/cluster/_agglomerative.py` is the Ward linkage implementation for hierarchical agglomerative clustering. It has two distinct code paths depending on whether a connectivity matrix is provided:\n\n**Unstructured path (connectivity=None):** When no connectivity is given, it delegates entirely to `scipy.cluster.hierarchy.ward(X)` after calling `np.require(X, requirements='W')`. The output from scipy is parsed: children from columns 0-1 (cast to `np.intp`), distances from column 2. It always returns `n_connected_components=1`, `n_leaves=n_samples`, and `parents=None`.\n\n**Structured path (with connectivity):** When connectivity is provided, the function:\n1. Calls `_fix_connectivity(X, connectivity, affinity='euclidean')` which symmetrizes the matrix, converts to LIL format, counts connected components via `scipy.sparse.csgraph.connected_components`, and if >1 component exists, calls `_fix_connected_components` from `sklearn.utils.graph` to bridge disconnected components using nearest-neighbor edges in 'connectivity' mode.\n2. Builds two moment arrays: `moments_1` (initialized to 1 for leaves) and `moments_2` (initialized from X). These are used to compute Ward distances via the Cython function `compute_ward_dist` in `_hierarchical_fast.pyx`.\n3. `compute_ward_dist` computes: for each pair (row, col), the inertia as `n * sum((m_2[row]/m_1[row] - m_2[col]/m_1[col])^2)` where `n = (m_1[row] * m_1[col]) / (m_1[row] + m_1[col])`.\n4. Uses Python's `heapq` (heapify/heappop/heappush) with tuples of `(inertia_value, row_index, col_index)` for the priority queue.\n5. During each merge iteration, it updates the structure matrix `A` (a list of adjacency lists) via `_get_parents`, which traverses the parent array to find the current cluster head for each neighbor, using a `not_visited` boolean array to avoid duplicates.\n6. Children are reversed (`c[::-1]`) before returning.\n7. Final distances are scaled by `np.sqrt(2.0 * distances)`.\n\n**The `_TREE_BUILDERS` dictionary** maps `ward_tree` as the tree builder for the 'ward' linkage, alongside `_complete_linkage`, `_average_linkage`, and `_single_linkage` (which are wrappers around `linkage_tree`).\n\n**In `AgglomerativeClustering.fit`**, when `linkage='ward'`, it enforces that `metric` must be 'euclidean' or 'l2'. The tree builder is called with `memory.cache(tree_builder)` for optional joblib caching. The output tree is cut using `_hc_cut`, which uses a negated-index heap to find the top-k clusters by traversing from the root.\n\n**Parameter validation** is handled by the `@validate_params` decorator, which constrains `n_clusters` to `Interval(Integral, 1, None, closed='left')` \u2014 meaning n_clusters must be an integer \u2265 1.\n\n**The `n_nodes` calculation** differs based on `n_clusters`: if `n_clusters` is None, `n_nodes = 2 * n_samples - 1`; otherwise `n_nodes = 2 * n_samples - n_clusters`, and it raises a ValueError if `n_clusters > n_samples`.",
    "rubric": [
      "Explains two distinct code paths: unstructured (connectivity=None) delegates to scipy.cluster.hierarchy.ward, structured (with connectivity) uses a heapq-based merge loop",
      "Describes the unstructured path: calls np.require(X, requirements='W'), parses scipy output columns 0-1 as children (cast to np.intp) and column 2 as distances, always returns n_connected_components=1, n_leaves=n_samples, parents=None",
      "Describes _fix_connectivity: symmetrizes connectivity matrix, converts to LIL, counts connected components, and calls _fix_connected_components if >1 component",
      "Explains the moments-based Ward distance computation: moments_1 initialized to 1 for leaves, moments_2 initialized from X, and compute_ward_dist calculates inertia as n * sum((m_2[row]/m_1[row] - m_2[col]/m_1[col])^2) where n = (m_1[row]*m_1[col])/(m_1[row]+m_1[col])",
      "Describes the heapq priority queue usage with (inertia, row, col) tuples and the _get_parents Cython function for updating adjacency using parent traversal and not_visited array",
      "Mentions children are reversed (c[::-1]) and distances are scaled by np.sqrt(2.0 * distances)",
      "Describes the _TREE_BUILDERS dictionary mapping 'ward' to ward_tree and other linkage types to wrapper functions around linkage_tree",
      "Explains that AgglomerativeClustering.fit enforces metric must be 'euclidean' or 'l2' for ward linkage, uses memory.cache(tree_builder) for joblib caching, and cuts the tree using _hc_cut",
      "Mentions _hc_cut uses negated-index heap to find top-k clusters from the root",
      "Explains n_nodes calculation: 2*n_samples-1 when n_clusters is None, otherwise 2*n_samples-n_clusters, with ValueError if n_clusters > n_samples"
    ],
    "key_files": [
      "sklearn/cluster/_agglomerative.py",
      "sklearn/cluster/_hierarchical_fast.pyx",
      "sklearn/utils/graph.py",
      "sklearn/cluster/_hierarchical_fast.pxd"
    ],
    "source_doc": "[docstring: sklearn/cluster/_agglomerative.py] sklearn.cluster._agglomerative.ward_tree\nWard clustering based on a Feature matrix.\n\n    Recursively merges the pair of clusters that minimally increases\n    within-cluster variance.\n\n    The inertia matrix uses a Heapq-based representation.\n\n    This is the structured version, that takes into account some topological\n    structure between samples.\n\n    Read more in the :ref:`User Guide <hierarchical_clustering>`.\n\n    Parameters\n    ----------\n    X : array-like of shape (n_samples, n_features)\n        Feature matrix representing `n_samples` samples to be clustered.\n\n    connectivity : {array-like, sparse matrix}, default=None\n        Connectivity matrix. Defines for each sample the neighboring samples\n        following a given structure of the data. The matrix is assumed to\n        be symmetric and only the upper triangular half is used.\n        Default is None, i.e, the Ward algorithm is unstructured.\n\n    n_clusters : int, default=None\n        `n_clusters` should be less than `n_samples`.  Stop early the\n        construction of the tree at `n_clusters.` This is useful to decrease\n        computation time if the number of clusters is not small compared to the\n        number of samples. In this case, the complete tree is not computed, thus\n        the 'children' output is of limited use, and the 'parents' output should\n        rather be used. This option is valid only when specifying a connectivity\n        matrix.\n\n    return_distance : bool, default=False\n        If `True`, return the distance between the clusters.\n\n    Returns\n    -------\n    children : ndarray of shape (n_nodes-1, 2)\n        The children of each non-leaf node. Values less than `n_samples`\n        correspond to leaves of the tree which are the original samples.\n        A node `i` greater than or equal to `n_samples` is a non-leaf\n        node and has children `children_[i - n_samples]`. Alternatively\n        at the i-th iteration, children[i][0] and children[i][1]\n        are merged to form node `n_samples + i`.\n\n    n_connected_components : int\n        The number of connected components in the graph.\n\n    n_leaves : int\n        The number of leaves in the tree.\n\n    parents : ndarray of shape (n_nodes,) or None\n        The parent of each node. Only returned when a connectivity matrix\n        is specified, elsewhere 'None' is returned.\n\n    distances : ndarray of shape (n_nodes-1,)\n        Only returned if `return_distance` is set to `True` (for compatibility).\n        The distances between the centers of the nodes. `distances[i]`\n        corresponds to a weighted Euclidean distance between\n        the nodes `children[i, 1]` and `children[i, 2]`. If the nodes refer to\n        leaves of the tree, then `distances[i]` is their unweighted Euclidean\n        distance. Distances are updated in the following way\n        (from scipy.hierarchy.linkage):\n\n        The new entry :math:`d(u,v)` is computed as follows,\n\n        .. math::\n\n           d(u,v) = \\sqrt{\\frac{|v|+|s|}\n                               {T}d(v,s)^2\n                        + \\frac{|v|+|t|}\n                               {T}d(v,t)^2\n                        - \\frac{|v|}\n                               {T}d(s,t)^2}\n\n        where :math:`u` is the newly joined cluster consisting of\n        clusters :math:`s` and :math:`t`, :math:`v` is an unused\n        cluster in the forest, :math:`T=|v|+|s|+|t|`, and\n        :math:`|*|` is the cardinality of its argument. This is also\n        known as the incremental algorithm.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import ward_tree\n    >>> X = np.array([[1, 2], [1, 4], [1, 0],\n    ...               [4, 2], [4, 4], [4, 0]])\n    >>> children, n_connected_components, n_leaves, parents = ward_tree(X)\n    >>> children\n    array([[0, 1],\n           [3, 5],\n           [2, 6],\n           [4, 7],\n           [8, 9]])\n    >>> n_connected_components\n    1\n    >>> n_leaves\n    6",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states distances[i] corresponds to distance between children[i, 1] and children[i, 2], but the answer doesn't address this specific indexing detail.",
      "The answer contains many implementation-level claims (Cython functions, heapq usage, _fix_connectivity, _get_parents, etc.) that cannot be verified from the documentation alone \u2014 these are plausible code-level details but are extrapolations beyond the provided doc.",
      "The answer states n_clusters must be \u2265 1 via Interval(Integral, 1, None, closed='left'), but the documentation says 'n_clusters should be less than n_samples' \u2014 these are not contradictory but the closed='left' means \u22651, which is consistent.",
      "The answer says the unstructured path returns parents=None, which is consistent with the doc stating 'Only returned when a connectivity matrix is specified, elsewhere None is returned'.",
      "The documentation example shows ward_tree returns parents as the 4th element when no connectivity is given, but the doc also says parents is None when no connectivity is specified \u2014 the example confirms this pattern."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor documentation-dependent claims about purpose descriptions ('to ensure writable', 'full tree', 'scaling factor comment') were removed or adjusted; core algorithmic claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_18",
    "repo": "scikit_learn",
    "question": "Why does `FeatureAgglomeration` transpose the input matrix, disable `fit_predict` via a property, and use triple inheritance instead of a simpler design?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `FeatureAgglomeration` class applies clustering to features (columns) rather than samples (rows), as evidenced by its code structure. The key design elements are:\n\n1. **Transposing X to reuse clustering logic**: In `FeatureAgglomeration.fit()`, the data matrix X is transposed (`X.T`) before being passed to `AgglomerativeClustering._fit()`. This is the core trick: by transposing the matrix, what were columns (features) become rows (samples), allowing the exact same clustering algorithm to cluster features instead of samples without duplicating any code.\n\n2. **Disabling `fit_predict` via a property that raises `AttributeError`**: `FeatureAgglomeration` defines `fit_predict` as a `@property` that raises `AttributeError`. `FeatureAgglomeration` inherits from `AgglomerativeClustering` (which inherits from `ClusterMixin`), and `ClusterMixin` provides a `fit_predict` method. By making it a property that raises `AttributeError`, the method effectively becomes inaccessible \u2014 any call to `obj.fit_predict` triggers the property getter which raises the error.\n\n3. **Fast path optimization in `transform`**: The `AgglomerationTransform.transform()` method has a special fast path: when `pooling_func == np.mean` and X is not sparse, it uses `np.bincount` with weights to efficiently compute grouped means, avoiding the slower generic loop that applies `pooling_func` for each label group.\n\n4. **Validation differs from parent**: `FeatureAgglomeration.fit()` calls `validate_data` with `ensure_min_features=2`, while the parent `AgglomerativeClustering.fit()` uses `ensure_min_samples=2`.\n\n5. **Triple inheritance for composition**: The class inherits from `ClassNamePrefixFeaturesOutMixin`, `AgglomerationTransform`, and `AgglomerativeClustering`. This gives it: (a) auto-generated output feature names from `ClassNamePrefixFeaturesOutMixin`, (b) `transform` and `inverse_transform` methods from `AgglomerationTransform`, and (c) the full clustering machinery from `AgglomerativeClustering`. The `_n_features_out` is set to `self.n_clusters_` after fitting.\n\n6. **`inverse_transform` uses broadcasting**: The `inverse_transform` in `AgglomerationTransform` uses `np.unique(self.labels_, return_inverse=True)` and then fancy indexing with `X[..., inverse]` to expand the reduced representation back to the original feature space by duplicating cluster values to all original features in that cluster.",
    "rubric": [
      "Explains that transposing X (X.T) reuses AgglomerativeClustering's sample-clustering logic to cluster features (columns) instead of samples (rows) without duplicating code",
      "Explains that fit_predict is defined as a @property that raises AttributeError to block the inherited ClusterMixin.fit_predict method, since clustering features (not samples) makes fit_predict semantically invalid",
      "Identifies the three parent classes (ClassNamePrefixFeaturesOutMixin, AgglomerationTransform, AgglomerativeClustering) and explains each provides: auto-generated output feature names, transform/inverse_transform methods, and clustering machinery respectively",
      "Mentions the fast path optimization in AgglomerationTransform.transform that uses np.bincount for computing grouped means when pooling_func is np.mean and X is not sparse",
      "Notes that FeatureAgglomeration.fit validates with ensure_min_features=2 while AgglomerativeClustering.fit validates with ensure_min_samples=2, reflecting their different clustering targets",
      "Explains that inverse_transform uses np.unique with return_inverse and fancy indexing (X[..., inverse]) to broadcast cluster values back to the original feature space"
    ],
    "key_files": [
      "sklearn/cluster/_agglomerative.py",
      "sklearn/cluster/_feature_agglomeration.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_agglomerative.py] sklearn.cluster._agglomerative.FeatureAgglomeration\nsklearn.cluster._agglomerative.FeatureAgglomeration:\n  Agglomerate features.\n\n    Recursively merges pair of clusters of features.\n\n    Refer to\n    :ref:`sphx_glr_auto_examples_cluster_plot_feature_agglomeration_vs_univariate_selection.py`\n    for an example comparison of :class:`FeatureAgglomeration` strategy with a\n    univariate feature selection strategy (based on ANOVA).\n\n    Read more in the :ref:`User Guide <hierarchical_clustering>`.\n\n    Parameters\n    ----------\n    n_clusters : int or None, default=2\n        The number of clusters to find. It must be ``None`` if\n        ``distance_threshold`` is not ``None``.\n\n    metric : str or callable, default=\"euclidean\"\n        Metric used to compute the linkage. Can be \"euclidean\", \"l1\", \"l2\",\n        \"manhattan\", \"cosine\", or \"precomputed\". If linkage is \"ward\", only\n        \"euclidean\" and \"l2\" are accepted. If \"precomputed\", a distance matrix is needed\n        as input for the fit method.\n\n        .. versionadded:: 1.2\n\n    memory : str or object with the joblib.Memory interface, default=None\n        Used to cache the output of the computation of the tree.\n        By default, no caching is done. If a string is given, it is the\n        path to the caching directory.\n\n    connectivity : array-like, sparse matrix, or callable, default=None\n        Connectivity matrix. Defines for each feature the neighboring\n        features following a given structure of the data.\n        This can be a connectivity matrix itself or a callable that transforms\n        the data into a connectivity matrix, such as derived from\n        `kneighbors_graph`. Default is `None`, i.e, the\n        hierarchical clustering algorithm is unstructured.\n\n    compute_full_tree : 'auto' or bool, default='auto'\n        Stop early the construction of the tree at `n_clusters`. This is useful\n        to decrease computation time if the number of clusters is not small\n        compared to the number of features. This option is useful only when\n        specifying a connectivity matrix. Note also that when varying the\n        number of clusters and using caching, it may be advantageous to compute\n        the full tree. It must be ``True`` if ``distance_threshold`` is not\n        ``None``. By default `compute_full_tree` is \"auto\", which is equivalent\n        to `True` when `distance_threshold` is not `None` or that `n_clusters`\n        is inferior to the maximum between 100 or `0.02 * n_samples`.\n        Otherwise, \"auto\" is equivalent to `False`.\n\n    linkage : {\"ward\", \"complete\", \"average\", \"single\"}, default=\"ward\"\n        Which linkage criterion to use. The linkage criterion determines which\n        distance to use between sets of features. The algorithm will merge\n        the pairs of cluster that minimize this criterion.\n\n        - \"ward\" minimizes the variance of the clusters being merged.\n        - \"complete\" or maximum linkage uses the maximum distances between\n          all features of the two sets.\n        - \"average\" uses the average of the distances of each feature of\n          the two sets.\n        - \"single\" uses the minimum of the distances between all features\n          of the two sets.\n\n    pooling_func : callable, default=np.mean\n        This combines the values of agglomerated features into a single\n        value, and should accept an array of shape [M, N] and the keyword\n        argument `axis=1`, and reduce it to an array of size [M].\n\n    distance_threshold : float, default=None\n        The linkage distance threshold at or above which clusters will not be\n        merged. If not ``None``, ``n_clusters`` must be ``None`` and\n        ``compute_full_tree`` must be ``True``.\n\n        .. versionadded:: 0.21\n\n    compute_distances : bool, default=False\n        Computes distances between clusters even if `distance_threshold` is not\n        used. This can be used to make dendrogram visualization, but introduces\n        a computational and memory overhead.\n\n        .. versionadded:: 0.24\n\n    Attributes\n    ----------\n    n_clusters_ : int\n        The number of clusters found by the algorithm. If\n        ``distance_threshold=None``, it will be equal to the given\n        ``n_clusters``.\n\n    labels_ : array-like of (n_features,)\n        Cluster labels for each feature.\n\n    n_leaves_ : int\n        Number of leaves in the hierarchical tree.\n\n    n_connected_components_ : int\n        The estimated number of connected components in the graph.\n\n        .. versionadded:: 0.21\n            ``n_connected_components_`` was added to replace ``n_components_``.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    children_ : array-like of shape (n_nodes-1, 2)\n        The children of each non-leaf node. Values less than `n_features`\n        correspond to leaves of the tree which are the original samples.\n        A node `i` greater than or equal to `n_features` is a non-leaf\n        node and has children `children_[i - n_features]`. Alternatively\n        at the i-th iteration, children[i][0] and children[i][1]\n        are merged to form node `n_features + i`.\n\n    distances_ : array-like of shape (n_nodes-1,)\n        Distances between nodes in the corresponding place in `children_`.\n        Only computed if `distance_threshold` is used or `compute_distances`\n        is set to `True`.\n\n    See Also\n    --------\n    AgglomerativeClustering : Agglomerative clustering samples instead of\n        features.\n    ward_tree : Hierarchical clustering with ward linkage.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn import datasets, cluster\n    >>> digits = datasets.load_digits()\n    >>> images = digits.images\n    >>> X = np.reshape(images, (len(images), -1))\n    >>> agglo = cluster.FeatureAgglomeration(n_clusters=32)\n    >>> agglo.fit(X)\n    FeatureAgglomeration(n_clusters=32)\n    >>> X_reduced = agglo.transform(X)\n    >>> X_reduced.shape\n    (1797, 32)\n\nsklearn.cluster._agglomerative.FeatureAgglomeration.fit:\n  Fit the hierarchical clustering on the data.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            The data.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        Returns\n        -------\n        self : object\n            Returns the transformer.\n\nsklearn.cluster._agglomerative.FeatureAgglomeration.fit_predict:\n  Fit and return the result of each sample's clustering assignment.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 2 about `fit_predict` being a property that raises `AttributeError` is not supported by the documentation. The documentation actually lists `fit_predict` as a method with the docstring 'Fit and return the result of each sample's clustering assignment.' This contradicts the claim that it raises AttributeError.",
      "Claims about transposing X, fast path optimization with np.bincount, triple inheritance from specific mixins, validate_data with ensure_min_features=2, and inverse_transform implementation details are all implementation-level claims not verifiable from the provided documentation, though they are plausible code-level details.",
      "The documentation explicitly shows `fit_predict` as a valid method signature, contradicting the claim that it's disabled via a property raising AttributeError."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed to remove design rationale phrasing not directly in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_20",
    "repo": "scikit_learn",
    "question": "How does SpectralBiclustering implement its full fitting pipeline, from normalization through SVD to final bicluster assignment?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `SpectralBiclustering` algorithm is implemented through the following steps:\n\n1. **Normalization** (`_fit` method): Depending on the `method` parameter:\n   - `'bistochastic'`: Uses `_bistochastic_normalize()` which iteratively applies `_scale_normalize()` up to `max_iter=1000` iterations with `tol=1e-5` convergence tolerance. Each iteration computes row and column scaling as `1/sqrt(row_sums)` and `1/sqrt(col_sums)`, replacing NaN values with 0.\n   - `'scale'`: A single application of `_scale_normalize()` which normalizes using `row_diag[:, np.newaxis] * X * col_diag` for dense matrices or `r @ X @ c` using `dia_array` for sparse ones.\n   - `'log'`: Uses `_log_normalize()` which first ensures minimum value is 1 via `make_nonnegative(X, min_value=1)`, then computes `L - row_avg - col_avg + avg` where L = log(X).\n\n2. **SVD computation** (`_svd` method): For 'bistochastic' and 'scale', `n_sv = n_components + 1` singular vectors are computed and the first is discarded (`n_discard=1`). For 'log', `n_sv = n_components` and `n_discard=0`. If using 'arpack' and NaN values appear in u or vt, a fallback computes eigenvectors of `A*A.T` or `A.T*A` using `eigsh` with a random initialization vector sampled uniformly from [-1, 1].\n\n3. **Best piecewise constant vector selection** (`_fit_best_piecewise` method): For each singular vector (row in the `vectors` matrix), k-means is applied to the 1D reshaped vector (shape `(-1, 1)`) with `n_clusters` clusters. The resulting centroid values are assigned back to each element based on its cluster label, creating a piecewise constant approximation. The Euclidean distance (using `scipy.linalg.norm`) between each original vector and its piecewise approximation is computed. The `n_best` vectors with the **smallest** distances (best approximated by piecewise constants) are selected via `np.argsort(dists)[:n_best]`.\n\n4. **Projection and clustering** (`_project_and_cluster` method): Row labels are determined by projecting the original data X onto the best right singular vectors (`best_vt.T`) using `safe_sparse_dot`, then clustering the projected data with k-means into `n_row_clusters`. Column labels are determined by projecting X.T onto the best left singular vectors (`best_ut.T`) and clustering into `n_col_clusters`.\n\n5. **Checkerboard bicluster construction**: The final `rows_` array has `n_row_clusters * n_col_clusters` rows, constructed by repeating each row-label indicator `n_col_clusters` times. Similarly, `columns_` repeats each column-label indicator `n_row_clusters` times. This creates the outer-product checkerboard structure where each row belongs to `n_col_clusters` biclusters and each column belongs to `n_row_clusters` biclusters.\n\nKey implementation detail: The `_k_means` helper in `BaseSpectral` either uses `KMeans` or `MiniBatchKMeans` depending on the `mini_batch` flag, passing `init`, `n_init`, and `random_state` parameters. It returns both `cluster_centers_` and `labels_` from the fitted model.",
    "rubric": [
      "Explains the three normalization methods: bistochastic (iterative scale_normalize up to 1000 iterations), scale (single scale_normalize), and log (log transform minus row/column/overall averages)",
      "Describes that scale_normalize computes row and column scaling as 1/sqrt(sums) with NaN replaced by 0, applied differently for sparse vs dense",
      "Notes that bistochastic and scale methods compute n_components+1 SVD vectors and discard the first one (n_discard=1), while log computes n_components with n_discard=0",
      "Explains the ARPACK fallback: when NaN appears in u or vt, eigsh is used on A*A.T or A.T*A with a random v0 from uniform[-1,1]",
      "Describes _fit_best_piecewise: k-means on each 1D singular vector to create piecewise constant approximations, then selects n_best vectors with smallest Euclidean distances",
      "Explains _project_and_cluster: projects data onto best singular vectors using safe_sparse_dot, then clusters with k-means",
      "Describes the checkerboard structure of rows_ and columns_: each row label is repeated n_col_clusters times and each column label is repeated n_row_clusters times to form the outer product pattern"
    ],
    "key_files": [
      "sklearn/cluster/_bicluster.py",
      "sklearn/utils/extmath.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_bicluster.py] sklearn.cluster._bicluster.SpectralBiclustering\nsklearn.cluster._bicluster.SpectralBiclustering:\n  Spectral biclustering (Kluger, 2003) [1]_.\n\n    Partitions rows and columns under the assumption that the data has\n    an underlying checkerboard structure. For instance, if there are\n    two row partitions and three column partitions, each row will\n    belong to three biclusters, and each column will belong to two\n    biclusters. The outer product of the corresponding row and column\n    label vectors gives this checkerboard structure.\n\n    Read more in the :ref:`User Guide <spectral_biclustering>`.\n\n    Parameters\n    ----------\n    n_clusters : int or tuple (n_row_clusters, n_column_clusters), default=3\n        The number of row and column clusters in the checkerboard\n        structure.\n\n    method : {'bistochastic', 'scale', 'log'}, default='bistochastic'\n        Method of normalizing and converting singular vectors into\n        biclusters. May be one of 'scale', 'bistochastic', or 'log'.\n        The authors recommend using 'log'. If the data is sparse,\n        however, log normalization will not work, which is why the\n        default is 'bistochastic'.\n\n        .. warning::\n           if `method='log'`, the data must not be sparse.\n\n    n_components : int, default=6\n        Number of singular vectors to check.\n\n    n_best : int, default=3\n        Number of best singular vectors to which to project the data\n        for clustering.\n\n    svd_method : {'randomized', 'arpack'}, default='randomized'\n        Selects the algorithm for finding singular vectors. May be\n        'randomized' or 'arpack'. If 'randomized', uses\n        :func:`~sklearn.utils.extmath.randomized_svd`, which may be faster\n        for large matrices. If 'arpack', uses\n        `scipy.sparse.linalg.svds`, which is more accurate, but\n        possibly slower in some cases.\n\n    n_svd_vecs : int, default=None\n        Number of vectors to use in calculating the SVD. Corresponds\n        to `ncv` when `svd_method=arpack` and `n_oversamples` when\n        `svd_method` is 'randomized`.\n\n    mini_batch : bool, default=False\n        Whether to use mini-batch k-means, which is faster but may get\n        different results.\n\n    init : {'k-means++', 'random'} or ndarray of shape (n_clusters, n_features),             default='k-means++'\n        Method for initialization of k-means algorithm; defaults to\n        'k-means++'.\n\n    n_init : int, default=10\n        Number of random initializations that are tried with the\n        k-means algorithm.\n\n        If mini-batch k-means is used, the best initialization is\n        chosen and the algorithm runs once. Otherwise, the algorithm\n        is run for each initialization and the best solution chosen.\n\n    random_state : int, RandomState instance, default=None\n        Used for randomizing the singular value decomposition and the k-means\n        initialization. Use an int to make the randomness deterministic.\n        See :term:`Glossary <random_state>`.\n\n    Attributes\n    ----------\n    rows_ : array-like of shape (n_row_clusters, n_rows)\n        Results of the clustering. `rows[i, r]` is True if\n        cluster `i` contains row `r`. Available only after calling ``fit``.\n\n    columns_ : array-like of shape (n_column_clusters, n_columns)\n        Results of the clustering, like `rows`.\n\n    row_labels_ : array-like of shape (n_rows,)\n        Row partition labels.\n\n    column_labels_ : array-like of shape (n_cols,)\n        Column partition labels.\n\n    biclusters_ : tuple of two ndarrays\n        The tuple contains the `rows_` and `columns_` arrays.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    SpectralCoclustering : Clusters rows and columns of an array `X` to solve the\n        relaxed normalized cut of the bipartite graph created from `X`.\n\n    References\n    ----------\n\n    .. [1] :doi:`Kluger, Yuval, et. al., 2003. Spectral biclustering of microarray\n           data: coclustering genes and conditions.\n           <10.1101/gr.648603>`\n\n    Examples\n    --------\n    >>> from sklearn.cluster import SpectralBiclustering\n    >>> import numpy as np\n    >>> X = np.array([[1, 1], [2, 1], [1, 0],\n    ...               [4, 7], [3, 5], [3, 6]])\n    >>> clustering = SpectralBiclustering(n_clusters=2, random_state=0).fit(X)\n    >>> clustering.row_labels_\n    array([1, 1, 1, 0, 0, 0], dtype=int32)\n    >>> clustering.column_labels_\n    array([1, 0], dtype=int32)\n    >>> clustering\n    SpectralBiclustering(n_clusters=2, random_state=0)\n\n    For a more detailed example, see\n    :ref:`sphx_glr_auto_examples_bicluster_plot_spectral_biclustering.py`\n\nsklearn.cluster._bicluster.SpectralBiclustering._fit_best_piecewise:\n  Find the ``n_best`` vectors that are best approximated by piecewise\n        constant vectors.\n\n        The piecewise vectors are found by k-means; the best is chosen\n        according to Euclidean distance.\n\nsklearn.cluster._bicluster.SpectralBiclustering._project_and_cluster:\n  Project ``data`` to ``vectors`` and cluster the result.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, convergence tolerances, specific formulas, fallback mechanisms) that are not present in the provided documentation and cannot be verified from it. These appear to be derived from source code inspection rather than the documentation chunk provided.",
      "The claim about n_components default being used with n_best default is consistent with the documentation (n_components=6, n_best=3).",
      "The description of the checkerboard structure ('each row belongs to n_col_clusters biclusters and each column belongs to n_row_clusters biclusters') is consistent with the documentation's description.",
      "The description of _fit_best_piecewise finding vectors 'best approximated by piecewise constant vectors' using k-means and Euclidean distance is consistent with the docstring provided.",
      "The description of _project_and_cluster projecting data to vectors and clustering is consistent with the docstring provided.",
      "The method options ('bistochastic', 'scale', 'log') and their descriptions are consistent with the documentation.",
      "The documentation says rows_ shape is (n_row_clusters, n_rows), but the answer implies rows_ has n_row_clusters * n_col_clusters rows, which would contradict the documented shape. However, looking more carefully at the documentation, it says 'Results of the clustering. rows[i, r] is True if cluster i contains row r' - the number of biclusters in the checkerboard structure is indeed n_row_clusters * n_col_clusters, so the shape would be (n_biclusters, n_rows). The documentation's attribute description says shape (n_row_clusters, n_rows) which may be imprecise or the answer's interpretation may be wrong.",
      "The answer's claim about rows_ having n_row_clusters * n_col_clusters rows potentially contradicts the documented shape of (n_row_clusters, n_rows)"
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor adjustments to remove line number references that cannot be verified from stripped code; all algorithmic claims are verifiable from the source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_19",
    "repo": "scikit_learn",
    "question": "Where is the SpectralCoclustering fitting logic implemented, and where do its key steps like normalization, SVD, and clustering happen?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `SpectralCoclustering._fit` method is located in `sklearn/cluster/_bicluster.py`. It performs its core computation through the following call chain:\n\n1. It first normalizes the data via `_scale_normalize(X)`, which is a module-level function in the same file that computes row and column diagonal scaling factors using `1.0/sqrt(X.sum(axis=...))`, handles NaN values from zero-sum rows/columns by replacing them with 0 (using `np.where`), and supports sparse matrices via `dia_array` diagonal multiplication.\n\n2. The number of singular vectors to compute is determined by `n_sv = 1 + int(np.ceil(np.log2(self.n_clusters)))`, where 1 is then discarded (`n_discard=1`), leaving `ceil(log2(n_clusters))` vectors.\n\n3. SVD is performed via `self._svd()` (defined in `BaseSpectral`), which dispatches to either `_randomized_svd` from `sklearn.utils.extmath` (with `n_oversamples` parameter) or `scipy.sparse.linalg.svds` (with `ncv` parameter). When using arpack, there's a NaN fallback that uses `eigsh` with `v0 = random_state.uniform(-1, 1, A.shape[0])`.\n\n4. The stacked vector `z = np.vstack((row_diag[:, np.newaxis] * u, col_diag[:, np.newaxis] * v))` combines scaled row and column singular vectors into a single matrix that is then clustered.\n\n5. K-means clustering is performed via `self._k_means()` (defined in `BaseSpectral`), which creates either a `KMeans` or `MiniBatchKMeans` instance from `sklearn.cluster._kmeans`.\n\n6. The `biclusters_` attribute is a property defined in `BiclusterMixin` (in `sklearn/base.py`) that simply returns `(self.rows_, self.columns_)` as a tuple.\n\nThe `BaseSpectral` class inherits from both `BiclusterMixin` and `BaseEstimator` and uses `ABCMeta` as its metaclass. The `fit` method in `BaseSpectral` is decorated with `@_fit_context(prefer_skip_nested_validation=True)`. Input data is validated with `accept_sparse=\"csr\"` and `dtype=np.float64`.",
    "rubric": [
      "Identifies that _fit is defined in SpectralCoclustering and the fit method is in BaseSpectral (both in the bicluster module)",
      "Mentions _scale_normalize as the normalization step, computing row/col diagonal scaling with 1/sqrt(sum) and handling NaN via np.where",
      "Explains the SVD count formula: n_sv = 1 + ceil(log2(n_clusters)), with n_discard=1",
      "Identifies _svd in BaseSpectral dispatching to either _randomized_svd (randomized) or scipy svds (arpack), with eigsh NaN fallback for arpack",
      "Describes the stacked vector z = vstack(row_diag * u, col_diag * v) combining scaled row and column singular vectors",
      "Identifies _k_means in BaseSpectral creating KMeans or MiniBatchKMeans based on self.mini_batch",
      "Notes that biclusters_ is a property in BiclusterMixin (in sklearn/base.py) returning (rows_, columns_)",
      "Mentions BaseSpectral inherits from BiclusterMixin and BaseEstimator with ABCMeta metaclass"
    ],
    "key_files": [
      "sklearn/cluster/_bicluster.py",
      "sklearn/base.py",
      "sklearn/utils/extmath.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_bicluster.py] sklearn.cluster._bicluster.SpectralCoclustering\nsklearn.cluster._bicluster.SpectralCoclustering:\n  Spectral Co-Clustering algorithm (Dhillon, 2001) [1]_.\n\n    Clusters rows and columns of an array `X` to solve the relaxed\n    normalized cut of the bipartite graph created from `X` as follows:\n    the edge between row vertex `i` and column vertex `j` has weight\n    `X[i, j]`.\n\n    The resulting bicluster structure is block-diagonal, since each\n    row and each column belongs to exactly one bicluster.\n\n    Supports sparse matrices, as long as they are nonnegative.\n\n    Read more in the :ref:`User Guide <spectral_coclustering>`.\n\n    Parameters\n    ----------\n    n_clusters : int, default=3\n        The number of biclusters to find.\n\n    svd_method : {'randomized', 'arpack'}, default='randomized'\n        Selects the algorithm for finding singular vectors. May be\n        'randomized' or 'arpack'. If 'randomized', use\n        :func:`sklearn.utils.extmath.randomized_svd`, which may be faster\n        for large matrices. If 'arpack', use\n        :func:`scipy.sparse.linalg.svds`, which is more accurate, but\n        possibly slower in some cases.\n\n    n_svd_vecs : int, default=None\n        Number of vectors to use in calculating the SVD. Corresponds\n        to `ncv` when `svd_method=arpack` and `n_oversamples` when\n        `svd_method` is 'randomized`.\n\n    mini_batch : bool, default=False\n        Whether to use mini-batch k-means, which is faster but may get\n        different results.\n\n    init : {'k-means++', 'random'}, or ndarray of shape             (n_clusters, n_features), default='k-means++'\n        Method for initialization of k-means algorithm; defaults to\n        'k-means++'.\n\n    n_init : int, default=10\n        Number of random initializations that are tried with the\n        k-means algorithm.\n\n        If mini-batch k-means is used, the best initialization is\n        chosen and the algorithm runs once. Otherwise, the algorithm\n        is run for each initialization and the best solution chosen.\n\n    random_state : int, RandomState instance, default=None\n        Used for randomizing the singular value decomposition and the k-means\n        initialization. Use an int to make the randomness deterministic.\n        See :term:`Glossary <random_state>`.\n\n    Attributes\n    ----------\n    rows_ : array-like of shape (n_row_clusters, n_rows)\n        Results of the clustering. `rows[i, r]` is True if\n        cluster `i` contains row `r`. Available only after calling ``fit``.\n\n    columns_ : array-like of shape (n_column_clusters, n_columns)\n        Results of the clustering, like `rows`.\n\n    row_labels_ : array-like of shape (n_rows,)\n        The bicluster label of each row.\n\n    column_labels_ : array-like of shape (n_cols,)\n        The bicluster label of each column.\n\n    biclusters_ : tuple of two ndarrays\n        The tuple contains the `rows_` and `columns_` arrays.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    SpectralBiclustering : Partitions rows and columns under the assumption\n        that the data has an underlying checkerboard structure.\n\n    References\n    ----------\n    .. [1] :doi:`Dhillon, Inderjit S, 2001. Co-clustering documents and words using\n           bipartite spectral graph partitioning.\n           <10.1145/502512.502550>`\n\n    Examples\n    --------\n    >>> from sklearn.cluster import SpectralCoclustering\n    >>> import numpy as np\n    >>> X = np.array([[1, 1], [2, 1], [1, 0],\n    ...               [4, 7], [3, 5], [3, 6]])\n    >>> clustering = SpectralCoclustering(n_clusters=2, random_state=0).fit(X)\n    >>> clustering.row_labels_ #doctest: +SKIP\n    array([0, 1, 1, 0, 0, 0], dtype=int32)\n    >>> clustering.column_labels_ #doctest: +SKIP\n    array([0, 0], dtype=int32)\n    >>> clustering\n    SpectralCoclustering(n_clusters=2, random_state=0)\n\n    For a more detailed example, see the following:\n    :ref:`sphx_glr_auto_examples_bicluster_plot_spectral_coclustering.py`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most of the answer describes internal implementation details (line numbers, internal method calls, code-level mechanics) that are not present in the provided documentation. These cannot be verified from the documentation alone, but they are plausible code-level details rather than contradictions.",
      "The answer states n_sv = 1 + int(np.ceil(np.log2(self.n_clusters))) \u2014 this is not in the documentation but is a plausible implementation detail.",
      "The answer describes _scale_normalize, _svd dispatch logic, NaN fallback with eigsh, stacked vector construction, etc. \u2014 none of these are documented but are plausible implementation specifics.",
      "The answer's description of n_svd_vecs mapping to 'ncv' for arpack and 'n_oversamples' for randomized is consistent with the documentation's description of the n_svd_vecs parameter.",
      "The answer's description of mini_batch controlling KMeans vs MiniBatchKMeans is consistent with the documentation's mini_batch parameter description.",
      "The answer mentions biclusters_ returns (self.rows_, self.columns_) which is consistent with the documented attribute 'The tuple contains the rows_ and columns_ arrays.'"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: line numbers are unverifiable from stripped code, and the internal behavior of _fit_context decorator is not visible in the provided code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_23",
    "repo": "scikit_learn",
    "question": "Where is `_brute_mst` defined, how does it compute the MST for dense vs sparse inputs, and how is its output consumed downstream?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `_brute_mst` function is defined in `sklearn/cluster/_hdbscan/hdbscan.py` at line 87. It is called exclusively from the `_hdbscan_brute` function (line 267 of the same file), which itself is selected as `mst_func` in the HDBSCAN class's `fit` method when `algorithm='brute'` or when the metric is not in FAST_METRICS or the input is sparse.\n\nFor the dense path, `_brute_mst` dispatches to `mst_from_mutual_reachability` in `sklearn/cluster/_hdbscan/_linkage.pyx` (line 61), which implements an MST algorithm starting at node 0. It maintains a `min_reachability` array initialized to infinity, iterates `n_samples - 1` times, and at each step uses `np.minimum` to update reachabilities and `np.argmin` to select the next node.\n\nFor the sparse path, `_brute_mst` first checks that every row has at least `min_samples` non-zero elements (using `indptr`), then verifies the graph has exactly one connected component via `scipy.sparse.csgraph.connected_components`, and finally calls `scipy.sparse.csgraph.minimum_spanning_tree`. The result is converted to a record array using `np.rec.fromarrays` with the `MST_edge_dtype`.\n\nThe `MST_edge_dtype` is defined in `_linkage.pyx` (line 48) as a NumPy structured dtype with three fields: `(\"current_node\", np.int64)`, `(\"next_node\", np.int64)`, and `(\"distance\", np.float64)`.\n\nThe output of `_brute_mst` is consumed by `_process_mst` (line 148), which sorts the MST edges by `\"distance\"` using `np.argsort` and then calls `make_single_linkage` \u2014 a Cython function that uses `UnionFind` (from `sklearn/cluster/_hierarchical_fast.pxd`) to build a single-linkage dendrogram.",
    "rubric": [
      "`_brute_mst` is defined in the HDBSCAN module and is called from `_hdbscan_brute`",
      "For dense inputs, it dispatches to `mst_from_mutual_reachability` which uses a Prim-like algorithm with `np.minimum` and `np.argmin`",
      "For sparse inputs, it checks that each row has at least `min_samples` non-zero elements, verifies one connected component, then calls `scipy.sparse.csgraph.minimum_spanning_tree`",
      "The result is a record array with `MST_edge_dtype` containing fields: current_node (int64), next_node (int64), and distance (float64)",
      "The output is consumed by `_process_mst`, which sorts edges by distance and calls `make_single_linkage` using a UnionFind structure to build a single-linkage dendrogram"
    ],
    "key_files": [
      "sklearn/cluster/_hdbscan/hdbscan.py",
      "sklearn/cluster/_hdbscan/_linkage.pyx",
      "sklearn/cluster/_hdbscan/_reachability.pyx",
      "sklearn/cluster/_hierarchical_fast.pxd"
    ],
    "source_doc": "[docstring: sklearn/cluster/_hdbscan/hdbscan.py] sklearn.cluster._hdbscan.hdbscan._brute_mst\nBuilds a minimum spanning tree (MST) from the provided mutual-reachability\n    values. This function dispatches to a custom Cython implementation for\n    dense arrays, and `scipy.sparse.csgraph.minimum_spanning_tree` for sparse\n    arrays/matrices.\n\n    Parameters\n    ----------\n    mututal_reachability_graph: {ndarray, sparse matrix} of shape             (n_samples, n_samples)\n        Weighted adjacency matrix of the mutual reachability graph.\n\n    min_samples : int, default=None\n        The number of samples in a neighborhood for a point\n        to be considered as a core point. This includes the point itself.\n\n    Returns\n    -------\n    mst : ndarray of shape (n_samples - 1,), dtype=MST_edge_dtype\n        The MST representation of the mutual-reachability graph. The MST is\n        represented as a collection of edges.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor wording about 'Prim's algorithm' was partially doc-derived and was softened.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_21",
    "repo": "scikit_learn",
    "question": "What classes, mixins, and internal data structures does the Birch clustering estimator depend on, and how do they interact?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `Birch` class in `sklearn/cluster/_birch.py` depends on several key internal classes and external utilities:\n\n**Class Hierarchy and MRO:**\nThe `Birch` class inherits from `ClassNamePrefixFeaturesOutMixin, ClusterMixin, TransformerMixin, BaseEstimator` (in that order). `TransformerMixin` itself inherits from `_SetOutputMixin`. `ClusterMixin` sets the estimator type to `\"clusterer\"` via `__sklearn_tags__` and provides `fit_predict`. The `ClassNamePrefixFeaturesOutMixin` generates feature names like `\"birch0\"`, `\"birch1\"`, etc., relying on a `_n_features_out` attribute set during fitting.\n\n**Internal Tree Classes:**\n- `_CFNode`: Initializes `init_centroids_` as a numpy array of shape `(branching_factor + 1, n_features)`. The `centroids_` and `squared_norm_` attributes are views (slices) of `init_centroids_` and `init_sq_norm_` respectively.\n- `_CFSubcluster`: When initialized with `linear_sum=None`, sets `n_samples_=0`, `squared_sum_=0.0`, and `centroid_=linear_sum_=0` (integer zero). The `merge_subcluster` method uses the formula `sq_radius = new_ss / new_n - new_sq_norm` and merges if `sq_radius <= threshold**2`. The `radius` property uses `sqrt(max(0, sq_radius))`.\n\n**Distance Computation in `insert_cf_subcluster`:**\nThe closest subcluster is found using a partial squared Euclidean distance: `dist_matrix = np.dot(self.centroids_, subcluster.centroid_) * -2.0 + self.squared_norm_`. This omits the `sq_norm_` of the query subcluster.\n\n**`_split_node` function:**\nUses `euclidean_distances` with `squared=True` and `Y_norm_squared=node.squared_norm_` to compute pairwise distances between subclusters in the node. The farthest pair is found via `np.unravel_index(dist.argmax(), ...)`. The code sets `node1_closer[farthest_idx[0]] = True` after computing the closer assignments.\n\n**Parameter Validation:**\nThe `_parameter_constraints` dictionary validates `n_clusters` against `[None, ClusterMixin, Interval(Integral, 1, None, closed=\"left\")]`.\n\n**Global Clustering (`_global_clustering`):**\nWhen `n_clusters` is an integer, an `AgglomerativeClustering` instance is created with that `n_clusters`. If the number of subclusters is less than the requested `n_clusters`, a `ConvergenceWarning` is raised and labels are assigned sequentially via `np.arange(len(centroids))`. The method computes `_subcluster_norms` using `row_norms(self.subcluster_centers_, squared=True)` for reuse in `_predict`.\n\n**`_predict` method:**\nUses `pairwise_distances_argmin` with `metric_kwargs={\"Y_norm_squared\": self._subcluster_norms}` inside `config_context(assume_finite=True)`.\n\n**Data Handling:**\nThe `_fit` method accepts both dense and CSR sparse matrices. For sparse data, it uses `_iterate_sparse_X` which densifies each row individually (accessing `X.indices`, `X.data`, `X.indptr` directly) rather than using sparse row slicing. Data is validated with `dtype=[np.float64, np.float32]`.\n\n**`__sklearn_tags__`:**\nSets `tags.transformer_tags.preserves_dtype = [\"float64\", \"float32\"]` and `tags.input_tags.sparse = True`.",
    "rubric": [
      "Birch inherits from ClassNamePrefixFeaturesOutMixin, ClusterMixin, TransformerMixin, and BaseEstimator (in that MRO order)",
      "_CFNode initializes init_centroids_ as shape (branching_factor + 1, n_features) and centroids_/squared_norm_ are views (slices) of init_centroids_/init_sq_norm_",
      "_CFSubcluster initialized with linear_sum=None sets n_samples_=0, squared_sum_=0.0, and centroid_=linear_sum_=0 (integer zero)",
      "merge_subcluster computes sq_radius = new_ss / new_n - new_sq_norm and merges only if sq_radius <= threshold**2",
      "insert_cf_subcluster finds closest subcluster via partial squared Euclidean distance: dot(centroids_, centroid_) * -2.0 + squared_norm_ (omitting query's squared norm)",
      "_split_node uses euclidean_distances with squared=True and Y_norm_squared, then np.unravel_index(dist.argmax()) to find the farthest pair",
      "_parameter_constraints validates n_clusters against [None, ClusterMixin, Interval(Integral, 1, None, closed='left')]",
      "_global_clustering creates AgglomerativeClustering when n_clusters is an int, and raises ConvergenceWarning with sequential labels if subclusters < n_clusters",
      "_predict uses pairwise_distances_argmin with metric_kwargs containing precomputed Y_norm_squared inside config_context(assume_finite=True)",
      "Sparse data is handled by _iterate_sparse_X which densifies rows individually using X.indices, X.data, X.indptr rather than sparse row slicing",
      "__sklearn_tags__ sets preserves_dtype=['float64', 'float32'] and sparse=True"
    ],
    "key_files": [
      "sklearn/cluster/_birch.py",
      "sklearn/base.py",
      "sklearn/utils/extmath.py",
      "sklearn/cluster/tests/test_birch.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_birch.py] sklearn.cluster._birch.Birch\nsklearn.cluster._birch.Birch:\n  Implements the BIRCH clustering algorithm.\n\n    It is a memory-efficient, online-learning algorithm provided as an\n    alternative to :class:`MiniBatchKMeans`. It constructs a tree\n    data structure with the cluster centroids being read off the leaf.\n    These can be either the final cluster centroids or can be provided as input\n    to another clustering algorithm such as :class:`AgglomerativeClustering`.\n\n    Read more in the :ref:`User Guide <birch>`.\n\n    .. versionadded:: 0.16\n\n    Parameters\n    ----------\n    threshold : float, default=0.5\n        The radius of the subcluster obtained by merging a new sample and the\n        closest subcluster should be lesser than the threshold. Otherwise a new\n        subcluster is started. Setting this value to be very low promotes\n        splitting and vice-versa.\n\n    branching_factor : int, default=50\n        Maximum number of CF subclusters in each node. If a new samples enters\n        such that the number of subclusters exceed the branching_factor then\n        that node is split into two nodes with the subclusters redistributed\n        in each. The parent subcluster of that node is removed and two new\n        subclusters are added as parents of the 2 split nodes.\n\n    n_clusters : int, instance of sklearn.cluster model or None, default=3\n        Number of clusters after the final clustering step, which treats the\n        subclusters from the leaves as new samples.\n\n        - `None` : the final clustering step is not performed and the\n          subclusters are returned as they are.\n\n        - :mod:`sklearn.cluster` Estimator : If a model is provided, the model\n          is fit treating the subclusters as new samples and the initial data\n          is mapped to the label of the closest subcluster.\n\n        - `int` : the model fit is :class:`AgglomerativeClustering` with\n          `n_clusters` set to be equal to the int.\n\n    compute_labels : bool, default=True\n        Whether or not to compute labels for each fit.\n\n    Attributes\n    ----------\n    root_ : _CFNode\n        Root of the CFTree.\n\n    dummy_leaf_ : _CFNode\n        Start pointer to all the leaves.\n\n    subcluster_centers_ : ndarray\n        Centroids of all subclusters read directly from the leaves.\n\n    subcluster_labels_ : ndarray\n        Labels assigned to the centroids of the subclusters after\n        they are clustered globally.\n\n    labels_ : ndarray of shape (n_samples,)\n        Array of labels assigned to the input data.\n        if partial_fit is used instead of fit, they are assigned to the\n        last batch of data.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    MiniBatchKMeans : Alternative implementation that does incremental updates\n        of the centers' positions using mini-batches.\n\n    Notes\n    -----\n    The tree data structure consists of nodes with each node consisting of\n    a number of subclusters. The maximum number of subclusters in a node\n    is determined by the branching factor. Each subcluster maintains a\n    linear sum, squared sum and the number of samples in that subcluster.\n    In addition, each subcluster can also have a node as its child, if the\n    subcluster is not a member of a leaf node.\n\n    For a new point entering the root, it is merged with the subcluster closest\n    to it and the linear sum, squared sum and the number of samples of that\n    subcluster are updated. This is done recursively till the properties of\n    the leaf node are updated.\n\n    See :ref:`sphx_glr_auto_examples_cluster_plot_birch_vs_minibatchkmeans.py` for a\n    comparison with :class:`~sklearn.cluster.MiniBatchKMeans`.\n\n    References\n    ----------\n    * Tian Zhang, Raghu Ramakrishnan, Maron Livny\n      BIRCH: An efficient data clustering method for large databases.\n      https://www.cs.sfu.ca/CourseCentral/459/han/papers/zhang96.pdf\n\n    * Roberto Perdisci\n      JBirch - Java implementation of BIRCH clustering algorithm\n      https://code.google.com/archive/p/jbirch\n\n    Examples\n    --------\n    >>> from sklearn.cluster import Birch\n    >>> X = [[0, 1], [0.3, 1], [-0.3, 1], [0, -1], [0.3, -1], [-0.3, -1]]\n    >>> brc = Birch(n_clusters=None)\n    >>> brc.fit(X)\n    Birch(n_clusters=None)\n    >>> brc.predict(X)\n    array([0, 0, 0, 1, 1, 1])\n\n    For a comparison of the BIRCH clustering algorithm with other clustering algorithms,\n    see :ref:`sphx_glr_auto_examples_cluster_plot_cluster_comparison.py`\n\nsklearn.cluster._birch.Birch.fit:\n  Build a CF Tree for the input data.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            Input data.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        Returns\n        -------\n        self\n            Fitted estimator.\n\nsklearn.cluster._birch.Birch._get_leaves:\n  Retrieve the leaves of the CF Node.\n\n        Returns\n        -------\n        leaves : list of shape (n_leaves,)\n            List of the leaf nodes.\n\nsklearn.cluster._birch.Birch.partial_fit:\n  Online learning. Prevents rebuilding of CFTree from scratch.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features),             default=None\n            Input data. If X is not provided, only the global clustering\n            step is done.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        Returns\n        -------\n        self\n            Fitted estimator.\n\nsklearn.cluster._birch.Birch.predict:\n  Predict data using the ``centroids_`` of subclusters.\n\n        Avoid computation of the row norms of X.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            Input data.\n\n        Returns\n        -------\n        labels : ndarray of shape(n_samples,)\n            Labelled data.\n\nsklearn.cluster._birch.Birch._predict:\n  Predict data using the ``centroids_`` of subclusters.\n\nsklearn.cluster._birch.Birch.transform:\n  Transform X into subcluster centroids dimension.\n\n        Each dimension represents the distance from the sample point to each\n        cluster centroid.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            Input data.\n\n        Returns\n        -------\n        X_trans : {array-like, sparse matrix} of shape (n_samples, n_clusters)\n            Transformed data.\n\nsklearn.cluster._birch.Birch._global_clustering:\n  Global clustering for the subclusters obtained after fitting",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer go far beyond what is documented (e.g., internal details about _CFNode, _CFSubcluster, _split_node, distance computation formulas, _parameter_constraints, sparse data handling, __sklearn_tags__). These are plausible code-level details but cannot be verified from the provided documentation alone.",
      "The claim about MRO (ClassNamePrefixFeaturesOutMixin, ClusterMixin, TransformerMixin, BaseEstimator) is not mentioned in the documentation and cannot be verified from it.",
      "The claim about _predict using config_context(assume_finite=True) is not in the documentation.",
      "The claim about _iterate_sparse_X densifying each row individually is not in the documentation.",
      "The description of _CFSubcluster merge formula and radius property are not in the documentation.",
      "The description of _split_node using euclidean_distances is not in the documentation.",
      "However, no claims directly contradict the provided documentation. The documented behavior (parameters, attributes, methods, their signatures and descriptions) is consistent with the answer where they overlap."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor rewrites needed for interpretive rationales about design purposes.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_24",
    "repo": "scikit_learn",
    "question": "How does HDBSCAN's brute-force algorithm build a single-linkage tree from raw data, and how does it differ from the Prim's-based alternative?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `_hdbscan_brute` function builds a single-linkage tree (SLT) through a multi-step pipeline:\n\n**Step 1: Distance Matrix Computation**\n- If `metric=\"precomputed\"`, it validates that X is square and symmetric using `_allclose_dense_sparse` (from `sklearn.utils.validation`), which checks symmetry by comparing `X` with `X.T`.\n- If not precomputed, it calls `sklearn.metrics.pairwise_distances(X, metric=metric, n_jobs=n_jobs, **metric_params)` to compute the full pairwise distance matrix.\n- The distance matrix is then scaled by dividing by `alpha` in-place: `distance_matrix /= alpha`.\n\n**Step 2: max_distance extraction**\n- A `max_distance` parameter is extracted from `metric_params` with a default of `0.0`. This is used only for sparse matrices in the sparse mutual reachability computation, where infinite mutual-reachability distances are truncated to `max_distance` if `max_distance > 0`.\n\n**Step 3: Sparse format conversion**\n- If the distance matrix is sparse and not in CSR format, it's converted to CSR.\n\n**Step 4: Mutual Reachability Graph**\n- Calls `mutual_reachability_graph()` from `_reachability.pyx`, which computes core distances as the (min_samples - 1)-th nearest neighbor distance using `np.partition`. For dense matrices, it uses `_dense_mutual_reachability_graph` which replaces each entry `d(i,j)` with `max(core_distances[i], core_distances[j], d(i,j))` in a nogil loop. For sparse matrices, `_sparse_mutual_reachability_graph` does the same but truncates infinite results to `max_distance` when positive. All modifications are done in-place on the distance matrix.\n\n**Step 5: MST Construction via `_brute_mst`**\n- For dense arrays: dispatches to `mst_from_mutual_reachability()` (Cython in `_linkage.pyx`), which implements Prim's algorithm \u2014 starts at node 0, tracks `min_reachability` initialized to infinity, and iterates n_samples-1 times, each time selecting the minimum-reachability edge to an unvisited node.\n- For sparse arrays: first validates that every row has at least `min_samples` non-zero elements (checking CSR `indptr` differences), then verifies the graph is connected via `scipy.sparse.csgraph.connected_components`. If disconnected, raises a `ValueError`. Then uses `scipy.sparse.csgraph.minimum_spanning_tree` and packages results into an array with `MST_edge_dtype` (fields: `current_node` int64, `next_node` int64, `distance` float64).\n\n**Step 6: Warning for infinite edges**\n- If any MST edge has infinite distance, a `UserWarning` is issued about missing distances.\n\n**Step 7: MST to SLT via `_process_mst`**\n- Sorts MST edges by `distance` field using `np.argsort`.\n- Calls `make_single_linkage()` (Cython), which uses a `UnionFind` data structure (from `sklearn.cluster._hierarchical_fast`) with `fast_find` and `union` operations to build the dendrogram. Each entry in the output has fields: `left_node` (intp), `right_node` (intp), `value` (float64), `cluster_size` (intp) \u2014 defined as `HIERARCHY_dtype` in `_tree.pyx`.\n\n**Integration with HDBSCAN class:**\n- `_hdbscan_brute` is selected when `algorithm=\"brute\"` explicitly, or when `algorithm=\"auto\"` and either the data is sparse or the metric is not in `FAST_METRICS` (the union of `KDTree.valid_metrics` and `BallTree.valid_metrics`). The `\"precomputed\"` metric always uses brute since it's not in FAST_METRICS.\n- The alternative path `_hdbscan_prims` uses `NearestNeighbors` (KDTree or BallTree) to compute core distances and then `mst_from_data_matrix` which computes mutual reachability implicitly during Prim's MST construction, avoiding the full pairwise distance matrix.",
    "rubric": [
      "Explains distance matrix computation: precomputed (with symmetry validation) vs calling pairwise_distances, followed by dividing by alpha",
      "Describes mutual reachability graph construction: core distances computed via np.partition at the (min_samples-1)-th neighbor, then replacing each distance with max(core_i, core_j, d(i,j)) in-place",
      "Explains dense MST construction uses Prim's algorithm (mst_from_mutual_reachability) starting at node 0 with min_reachability initialized to infinity",
      "Explains sparse MST construction validates min_samples neighbors per row and graph connectivity, then delegates to scipy.sparse.csgraph.minimum_spanning_tree",
      "Describes MST-to-SLT conversion: edges sorted by distance, then make_single_linkage uses UnionFind with fast_find/union to build dendrogram entries (left_node, right_node, value, cluster_size)",
      "Mentions the warning issued when MST contains infinite-distance edges",
      "Contrasts with _hdbscan_prims which uses NearestNeighbors for core distances and mst_from_data_matrix that computes mutual reachability on-the-fly during Prim's MST, avoiding the full pairwise distance matrix",
      "Explains when brute is selected: explicitly via algorithm='brute', or auto when data is sparse or metric not in FAST_METRICS (KDTree + BallTree valid metrics)"
    ],
    "key_files": [
      "sklearn/cluster/_hdbscan/hdbscan.py",
      "sklearn/cluster/_hdbscan/_reachability.pyx",
      "sklearn/cluster/_hdbscan/_linkage.pyx",
      "sklearn/cluster/_hdbscan/_tree.pyx",
      "sklearn/cluster/_hierarchical_fast.pxd"
    ],
    "source_doc": "[docstring: sklearn/cluster/_hdbscan/hdbscan.py] sklearn.cluster._hdbscan.hdbscan._hdbscan_brute\nBuilds a single-linkage tree (SLT) from the input data `X`. If\n    `metric=\"precomputed\"` then `X` must be a symmetric array of distances.\n    Otherwise, the pairwise distances are calculated directly and passed to\n    `mutual_reachability_graph`.\n\n    Parameters\n    ----------\n    X : ndarray of shape (n_samples, n_features) or (n_samples, n_samples)\n        Either the raw data from which to compute the pairwise distances,\n        or the precomputed distances.\n\n    min_samples : int, default=None\n        The number of samples in a neighborhood for a point\n        to be considered as a core point. This includes the point itself.\n\n    alpha : float, default=1.0\n        A distance scaling parameter as used in robust single linkage.\n\n    metric : str or callable, default='euclidean'\n        The metric to use when calculating distance between instances in a\n        feature array.\n\n        - If metric is a string or callable, it must be one of\n          the options allowed by :func:`~sklearn.metrics.pairwise_distances`\n          for its metric parameter.\n\n        - If metric is \"precomputed\", X is assumed to be a distance matrix and\n          must be square.\n\n    n_jobs : int, default=None\n        The number of jobs to use for computing the pairwise distances. This\n        works by breaking down the pairwise matrix into n_jobs even slices and\n        computing them in parallel. This parameter is passed directly to\n        :func:`~sklearn.metrics.pairwise_distances`.\n\n        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n        for more details.\n\n    copy : bool, default=False\n        If `copy=True` then any time an in-place modifications would be made\n        that would overwrite `X`, a copy will first be made, guaranteeing that\n        the original data will be unchanged. Currently, it only applies when\n        `metric=\"precomputed\"`, when passing a dense array or a CSR sparse\n        array/matrix.\n\n    metric_params : dict, default=None\n        Arguments passed to the distance metric.\n\n    Returns\n    -------\n    single_linkage : ndarray of shape (n_samples - 1,), dtype=HIERARCHY_dtype\n        The single-linkage tree tree (dendrogram) built from the MST.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The return type description in the answer mentions fields `left_node`, `right_node`, `value`, `cluster_size` for HIERARCHY_dtype, while the documentation only states the return is 'ndarray of shape (n_samples - 1,), dtype=HIERARCHY_dtype' without specifying the fields \u2014 this is plausible but unverifiable from the doc alone.",
      "Many implementation details (e.g., Prim's algorithm specifics, UnionFind with path compression, _allclose_dense_sparse tolerances, np.partition for core distances, sparse validation checks, max_distance truncation behavior) go far beyond what the documentation states. These are plausible code-level details but cannot be verified from the provided docstring.",
      "The answer claims the distance matrix is scaled by dividing by alpha in-place. The doc only says alpha is 'a distance scaling parameter as used in robust single linkage' \u2014 the specific operation (division) is plausible but not documented.",
      "The answer's description of algorithm selection logic (FAST_METRICS, auto mode behavior) is not covered by this docstring and represents extrapolation from broader codebase knowledge.",
      "The answer states symmetry is checked using `_allclose_dense_sparse` with specific tolerances \u2014 this is not mentioned in the documentation and is an implementation detail that could be inaccurate."
    ],
    "strip_verify_leakage": 0.09,
    "strip_verify_summary": "Very few claims rely on documentation: specific _allclose_dense_sparse implementation details and 'path compression' annotation. Almost all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_22",
    "repo": "scikit_learn",
    "question": "Why does scikit-learn's DBSCAN implementation use a Cython DFS with a C++ stack, skip distances in neighborhood queries, and square the radius for euclidean brute-force?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The DBSCAN implementation uses several performance optimizations that explain key design decisions:\n\n**1. Cython inner loop with C++ stack (dbscan_inner):** The cluster-labeling phase is implemented in Cython (`_dbscan_inner.pyx`) rather than pure Python. It uses a C++ `std::vector<intp_t>` as an explicit stack for depth-first search (DFS) instead of Python recursion. The DFS only expands neighborhoods of core points (checking `is_core[i]` before expanding), so non-core points are labeled but their neighborhoods aren't traversed\u2014this is a critical optimization that limits the work done.\n\n**2. Bulk neighborhood computation with `return_distance=False`:** The `fit()` method calls `neighbors_model.radius_neighbors(X, return_distance=False)`. By passing `return_distance=False`, it skips computing and storing actual distance values, since the code only uses the neighbor indices (not distances) for determining core samples and labeling.\n\n**3. Squared Euclidean distance optimization:** In `_base.py`, when the effective metric is euclidean and brute-force is used, there is logic to square the radius and compute distances with `squared=True`, deferring the square root operation. (This logic is in the `radius_neighbors` method.)\n\n**4. RadiusNeighbors fast path:** The `RadiusNeighbors` class is imported from `_pairwise_distances_reduction`, and `radius_neighbors()` can use `RadiusNeighbors.is_usable_for()` to check if a fast path is available before falling back to `pairwise_distances_chunked`.\n\n**5. Precomputed sparse diagonal handling:** For precomputed sparse distance matrices, the diagonal is explicitly set using `X.setdiag(X.diagonal())`. The `SparseEfficiencyWarning` is suppressed with `warnings.simplefilter('ignore', sparse.SparseEfficiencyWarning)` around this operation.\n\n**6. Chunked distance computation for memory control:** The `pairwise_distances_chunked` function (imported and used in the codebase) computes distances in chunks. The `get_chunk_n_rows` utility is imported to determine appropriate chunk sizes based on available working memory.\n\n**7. Thread-based parallelism for tree methods:** The `Parallel` and `delayed` utilities are imported from `sklearn.utils.parallel` for parallelized neighbor queries in `_base.py`.\n\n**8. Default metric difference:** The standalone `dbscan()` function defaults to `metric='minkowski'` with `p=2`, while the `DBSCAN` class defaults to `metric='euclidean'` with `p=None`. In `_base.py`, the `_fit` method converts minkowski with p=2 to an effective metric of 'euclidean', making them functionally equivalent.",
    "rubric": [
      "Explains that dbscan_inner is implemented in Cython using a C++ std::vector as an explicit stack for DFS instead of Python recursion to avoid Python overhead",
      "Notes that the DFS only expands neighborhoods of core points (checks is_core before pushing neighbors), so non-core points are labeled but not expanded",
      "Explains that return_distance=False is passed to radius_neighbors to skip computing/storing actual distance values since only indices are needed",
      "Describes the squared euclidean optimization: when effective_metric_ is euclidean and brute-force is used, the radius is squared and distances are computed with squared=True to avoid computing square roots",
      "Mentions the RadiusNeighbors fast path from _pairwise_distances_reduction that is checked via is_usable_for() before falling back to chunked computation",
      "Notes the precomputed sparse matrix diagonal handling with setdiag and suppression of SparseEfficiencyWarning",
      "Mentions chunked distance computation via pairwise_distances_chunked for memory control",
      "Notes that tree-based methods use thread-based parallelism via Parallel with prefer='threads'"
    ],
    "key_files": [
      "sklearn/cluster/_dbscan.py",
      "sklearn/cluster/_dbscan_inner.pyx",
      "sklearn/neighbors/_base.py",
      "sklearn/metrics/pairwise.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_dbscan.py] sklearn.cluster._dbscan.dbscan\nPerform DBSCAN clustering from vector array or distance matrix.\n\n    This function is a wrapper around :class:`~cluster.DBSCAN`, suitable for\n    quick, standalone clustering tasks. For estimator-based workflows, where\n    estimator attributes or pipeline integration is required, prefer\n    :class:`~cluster.DBSCAN`.\n\n    DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a\n    density-based clustering algorithm that groups together points that are\n    closely packed while marking points in low-density regions as outliers.\n\n    Read more in the :ref:`User Guide <dbscan>`.\n\n    Parameters\n    ----------\n    X : {array-like, scipy sparse matrix} of shape (n_samples, n_features) or             (n_samples, n_samples)\n        A feature array, or array of distances between samples if\n        ``metric='precomputed'``. When using precomputed distances, X must\n        be a square symmetric matrix.\n\n    eps : float, default=0.5\n        The maximum distance between two samples for one to be considered\n        as in the neighborhood of the other. This is not a maximum bound\n        on the distances of points within a cluster. This is the most\n        important DBSCAN parameter to choose appropriately for your data set\n        and distance function. Smaller values result in more clusters,\n        while larger values result in fewer, larger clusters.\n\n    min_samples : int, default=5\n        The number of samples (or total weight) in a neighborhood for a point\n        to be considered as a core point. This includes the point itself.\n        Higher values yield fewer, denser clusters, while lower values yield\n        more, sparser clusters.\n\n    metric : str or callable, default='minkowski'\n        The metric to use when calculating distance between instances in a\n        feature array. If metric is a string or callable, it must be one of\n        the options allowed by :func:`sklearn.metrics.pairwise_distances` for\n        its metric parameter.\n        If metric is \"precomputed\", X is assumed to be a distance matrix and\n        must be square during fit.\n        X may be a :term:`sparse graph <sparse graph>`,\n        in which case only \"nonzero\" elements may be considered neighbors.\n\n    metric_params : dict, default=None\n        Additional keyword arguments for the metric function.\n\n        .. versionadded:: 0.19\n\n    algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto'\n        The algorithm to be used by the NearestNeighbors module\n        to compute pointwise distances and find nearest neighbors.\n        'auto' will attempt to decide the most appropriate algorithm\n        based on the values passed to :meth:`fit` method.\n        See :class:`~sklearn.neighbors.NearestNeighbors` documentation for\n        details.\n\n    leaf_size : int, default=30\n        Leaf size passed to BallTree or cKDTree. This can affect the speed\n        of the construction and query, as well as the memory required\n        to store the tree. The optimal value depends\n        on the nature of the problem. Generally, smaller leaf sizes\n        lead to faster queries but slower construction.\n\n    p : float, default=2\n        Power parameter for the Minkowski metric. When p = 1, this is equivalent\n        to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2.\n        For arbitrary p, minkowski_distance (l_p) is used. This parameter is expected\n        to be positive.\n\n    sample_weight : array-like of shape (n_samples,), default=None\n        Weight of each sample, such that a sample with a weight of at least\n        ``min_samples`` is by itself a core sample; a sample with negative\n        weight may inhibit its eps-neighbor from being core.\n        Note that weights are absolute, and default to 1.\n\n    n_jobs : int, default=None\n        The number of parallel jobs to run for neighbors search. ``None`` means\n        1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means\n        using all processors. See :term:`Glossary <n_jobs>` for more details.\n        If precomputed distances are used, parallel execution is not available\n        and thus n_jobs will have no effect.\n\n    Returns\n    -------\n    core_samples : ndarray of shape (n_core_samples,)\n        Indices of core samples.\n\n    labels : ndarray of shape (n_samples,)\n        Cluster labels for each point. Noisy samples are given the label -1.\n        Non-negative integers indicate cluster membership.\n\n    See Also\n    --------\n    DBSCAN : An estimator interface for this clustering algorithm.\n    OPTICS : A similar estimator interface clustering at multiple values of\n        eps. Our implementation is optimized for memory usage.\n\n    Notes\n    -----\n    For an example, see :ref:`sphx_glr_auto_examples_cluster_plot_dbscan.py`.\n\n    This implementation bulk-computes all neighborhood queries, which increases\n    the memory complexity to O(n.d) where d is the average number of neighbors,\n    while original DBSCAN had memory complexity O(n). It may attract a higher\n    memory complexity when querying these nearest neighborhoods, depending\n    on the ``algorithm``.\n\n    One way to avoid the query complexity is to pre-compute sparse\n    neighborhoods in chunks using\n    :func:`NearestNeighbors.radius_neighbors_graph\n    <sklearn.neighbors.NearestNeighbors.radius_neighbors_graph>` with\n    ``mode='distance'``, then using ``metric='precomputed'`` here.\n\n    Another way to reduce memory and computation time is to remove\n    (near-)duplicate points and use ``sample_weight`` instead.\n\n    :class:`~sklearn.cluster.OPTICS` provides a similar clustering with lower\n    memory usage.\n\n    References\n    ----------\n    Ester, M., H. P. Kriegel, J. Sander, and X. Xu, `\"A Density-Based\n    Algorithm for Discovering Clusters in Large Spatial Databases with Noise\"\n    <https://www.dbs.ifi.lmu.de/Publikationen/Papers/KDD-96.final.frame.pdf>`_.\n    In: Proceedings of the 2nd International Conference on Knowledge Discovery\n    and Data Mining, Portland, OR, AAAI Press, pp. 226-231. 1996\n\n    Schubert, E., Sander, J., Ester, M., Kriegel, H. P., & Xu, X. (2017).\n    :doi:`\"DBSCAN revisited, revisited: why and how you should (still) use DBSCAN.\"\n    <10.1145/3068335>`\n    ACM Transactions on Database Systems (TODS), 42(3), 19.\n\n    Examples\n    --------\n    >>> from sklearn.cluster import dbscan\n    >>> X = [[1, 2], [2, 2], [2, 3], [8, 7], [8, 8], [25, 80]]\n    >>> core_samples, labels = dbscan(X, eps=3, min_samples=2)\n    >>> core_samples\n    array([0, 1, 2, 3, 4])\n    >>> labels\n    array([ 0,  0,  0,  1,  1, -1])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer discusses many implementation details (Cython inner loop, C++ stack, squared Euclidean optimization, RadiusNeighbors fast path, chunked distance computation, thread-based parallelism) that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the given doc chunk alone.",
      "Claim 8 states the DBSCAN class defaults to metric='euclidean' with p=None, while the standalone function defaults to metric='minkowski' with p=2. The documentation only shows the standalone function's defaults (metric='minkowski', p=2). The claim about the class defaults cannot be verified from the provided documentation, though it is plausible.",
      "The documentation states memory complexity is O(n.d) where d is the average number of neighbors, while the answer mentions 'worst case O(n^2) memory complexity' attributed to a code comment. These are compatible (O(n*d) worst case when d=n is O(n^2)), but the answer's framing is slightly different from the doc's explicit statement.",
      "The answer's detailed claims about specific line numbers, file paths, and internal implementation mechanisms go well beyond what the documentation covers. While plausible, they are unverifiable from the provided documentation alone."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims rely on design rationale from comments/docstrings or reference truncated code sections; these were removed or rewritten to only include code-verifiable facts.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_27",
    "repo": "scikit_learn",
    "question": "Where does the data flow through `estimate_bandwidth` and how is the result used downstream in `MeanShift`?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `estimate_bandwidth` function in `sklearn/cluster/_mean_shift.py` flows data through the following path:\n\n1. **Input validation**: `X` is validated via `check_array(X)` with no additional arguments.\n\n2. **Subsampling**: If `n_samples` is provided, the function uses `random_state.permutation(X.shape[0])[:n_samples]` to randomly select a subset of indices and replaces `X` with `X[idx]`. The default `random_state` parameter is `0`.\n\n3. **Neighbor computation**: `n_neighbors` is computed as `int(X.shape[0] * quantile)`, clamped to a minimum of 1. A `NearestNeighbors` object is created with this `n_neighbors` and default algorithm='auto', metric='minkowski', p=2 (Euclidean distance). This `NearestNeighbors` is fitted on the (potentially subsampled) `X`.\n\n4. **Batched distance computation**: The function iterates over batches of size 500 (using `gen_batches(len(X), 500)` from `sklearn.utils._chunking`). For each batch, it calls `nbrs.kneighbors(X[batch, :], return_distance=True)`, takes `np.max(d, axis=1).sum()` (the maximum distance to the k-th neighbor for each point in the batch), and accumulates this into the total bandwidth.\n\n5. **Final bandwidth**: The accumulated sum is divided by `X.shape[0]` to produce the average max k-neighbor distance.\n\n6. **Downstream usage**: When `MeanShift.fit()` is called with `bandwidth=None`, it calls `estimate_bandwidth(X, n_jobs=self.n_jobs)` with default `quantile=0.3` and `random_state=0`. The returned bandwidth is then used to construct a `NearestNeighbors(radius=bandwidth, n_jobs=1)` for the hill-climbing step in `_mean_shift_single_seed`. In `_mean_shift_single_seed`, the bandwidth is retrieved via `nbrs.get_params()[\"radius\"]` and the convergence threshold is set to `1e-3 * bandwidth`.\n\n7. **gen_batches**: Located in `sklearn/utils/_chunking.py`, it generates slices of the given batch_size. It also supports a `min_batch_size` parameter \u2014 when `end + min_batch_size > n`, the current batch is skipped via `continue`, and the remaining elements are yielded as a single final slice.",
    "rubric": [
      "Input X is validated via check_array(X)",
      "Subsampling uses random_state.permutation(X.shape[0])[:n_samples] with default random_state=0",
      "n_neighbors is computed as int(X.shape[0] * quantile) clamped to minimum of 1",
      "NearestNeighbors is created with the computed n_neighbors and fitted on (potentially subsampled) X",
      "Distances are computed in batches of 500 using gen_batches, taking np.max(d, axis=1).sum() per batch",
      "Final bandwidth is the accumulated sum divided by X.shape[0]",
      "MeanShift.fit() calls estimate_bandwidth with default quantile=0.3 when bandwidth is None",
      "The returned bandwidth is used as the radius parameter in a NearestNeighbors for the hill-climbing step",
      "In _mean_shift_single_seed, bandwidth is retrieved via nbrs.get_params()['radius'] and convergence threshold is 1e-3 * bandwidth",
      "gen_batches is located in sklearn/utils/_chunking.py and handles min_batch_size by merging remaining elements into the final slice"
    ],
    "key_files": [
      "sklearn/cluster/_mean_shift.py",
      "sklearn/utils/_chunking.py",
      "sklearn/neighbors/_unsupervised.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_mean_shift.py] sklearn.cluster._mean_shift.estimate_bandwidth\nEstimate the bandwidth to use with the mean-shift algorithm.\n\n    This function takes time at least quadratic in `n_samples`. For large\n    datasets, it is wise to subsample by setting `n_samples`. Alternatively,\n    the parameter `bandwidth` can be set to a small value without estimating\n    it.\n\n    Parameters\n    ----------\n    X : array-like of shape (n_samples, n_features)\n        Input points.\n\n    quantile : float, default=0.3\n        Should be between [0, 1]\n        0.5 means that the median of all pairwise distances is used.\n\n    n_samples : int, default=None\n        The number of samples to use. If not given, all samples are used.\n\n    random_state : int, RandomState instance, default=None\n        The generator used to randomly select the samples from input points\n        for bandwidth estimation. Use an int to make the randomness\n        deterministic.\n        See :term:`Glossary <random_state>`.\n\n    n_jobs : int, default=None\n        The number of parallel jobs to run for neighbors search.\n        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n        for more details.\n\n    Returns\n    -------\n    bandwidth : float\n        The bandwidth parameter.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import estimate_bandwidth\n    >>> X = np.array([[1, 1], [2, 1], [1, 0],\n    ...               [4, 7], [3, 5], [3, 6]])\n    >>> estimate_bandwidth(X, quantile=0.5)\n    np.float64(1.61)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 'The default random_state parameter is 0 (not None as stated in the docstring signature \u2014 the actual function signature has random_state=0)'. The documentation explicitly states 'random_state : int, RandomState instance, default=None'. This is a direct contradiction of the documentation, though the answer may be correct about the actual source code \u2014 this is a claim that cannot be verified from the provided documentation alone.",
      "Many implementation details (batched computation with size 500, gen_batches, NearestNeighbors with algorithm='auto', metric='minkowski', p=2, convergence threshold 1e-3*bandwidth, MeanShift.fit() calling estimate_bandwidth with specific parameters) are not verifiable from the provided documentation chunk and appear to be extrapolations from source code knowledge."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc-leakage in check_array default parameter descriptions and a docstring-related parenthetical about random_state; otherwise nearly all claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_28",
    "repo": "scikit_learn",
    "question": "How does the MeanShift clustering algorithm work end-to-end, from bandwidth estimation through label assignment?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `mean_shift` function works as follows:\n\n## Overall Architecture\nThe `mean_shift` standalone function is a thin wrapper that creates a `MeanShift` class instance, calls `.fit(X)`, and returns `(model.cluster_centers_, model.labels_)` (lines 235-244). The `MeanShift` class inherits from `ClusterMixin` and `BaseEstimator`.\n\n## Algorithm Steps (in `MeanShift.fit()`):\n\n### 1. Bandwidth Estimation\nIf bandwidth is None, it calls `estimate_bandwidth(X, n_jobs=self.n_jobs)`. This function computes `n_neighbors = int(n_samples * quantile)` (default quantile=0.3), fits a `NearestNeighbors` with that many neighbors, queries in batches of 500 using `gen_batches(len(X), 500)`, takes the max distance per row, sums them, and divides by `n_samples`. The default `random_state` for `estimate_bandwidth` is 0 (not None).\n\n### 2. Seed Selection\n- If `seeds` is None and `bin_seeding=False`: all data points X are used as seeds.\n- If `seeds` is None and `bin_seeding=True`: `get_bin_seeds(X, bandwidth, min_bin_freq)` is called, which bins points by dividing by `bin_size` and rounding (`np.round(point / bin_size)`), counts frequency per bin using a `defaultdict(int)`, filters bins with at least `min_bin_freq` points, creates an array with `dtype=np.float32`, and multiplies back by `bin_size`. If `bin_size == 0`, it returns X unchanged. If binning produces the same number of seeds as data points, it issues a warning and falls back to using X directly.\n\n### 3. Hill-Climbing (Parallel)\nA `NearestNeighbors(radius=bandwidth, n_jobs=1)` is fitted on X (n_jobs=1 is set explicitly). Then `_mean_shift_single_seed` is called in parallel for each seed via `joblib.Parallel`.\n\nEach seed's hill-climbing (`_mean_shift_single_seed`):\n- Retrieves bandwidth via `nbrs.get_params()[\"radius\"]`\n- Sets convergence threshold: `stop_thresh = 1e-3 * bandwidth`\n- In a while-True loop: finds all points within the bandwidth using `radius_neighbors`, computes the mean of those points, checks if `np.linalg.norm(my_mean - my_old_mean) <= stop_thresh` or if `completed_iterations == max_iter`, breaks if either is true\n- Returns a tuple: `(tuple(my_mean), len(points_within), completed_iterations)`\n- If no points are within bandwidth of a seed, the loop breaks early\n\n### 4. Post-Processing: Duplicate Removal\nResults are collected into a dict mapping center-tuple \u2192 intensity (number of points within bandwidth at convergence). Centers with 0 points within are excluded. Centers are sorted by `(intensity, center_coordinates)` in descending order. A new `NearestNeighbors(radius=bandwidth)` is fitted on sorted centers. For each center (in order of decreasing intensity), all neighbors within bandwidth are marked as non-unique, but the current center is preserved. This ensures higher-intensity centers survive.\n\n### 5. Label Assignment\nA `NearestNeighbors(n_neighbors=1)` is fitted on the final cluster centers. If `cluster_all=True`, every point is assigned to its nearest center. If `cluster_all=False`, labels are initialized to -1, and only points whose distance to the nearest center is \u2264 bandwidth are assigned.\n\n### 6. Predict\nThe `predict` method uses `pairwise_distances_argmin(X, self.cluster_centers_)` with `config_context(assume_finite=True)`.\n\n### 7. Parameter Constraints\nThe bandwidth constraint is `Interval(Real, 0, None, closed=\"neither\")` \u2014 meaning bandwidth must be strictly greater than 0 (open interval, not including 0). The `max_iter` constraint is `Interval(Integral, 0, None, closed=\"left\")`, meaning it must be \u2265 0.\n\n`n_iter_` is computed as the maximum of `completed_iterations` across all seeds.",
    "rubric": [
      "Explains that `mean_shift` function is a wrapper that creates a MeanShift instance, calls fit(X), and returns cluster_centers_ and labels_",
      "Describes bandwidth estimation: computes n_neighbors as int(n_samples * quantile), uses NearestNeighbors to find k-nearest distances in batches of 500, takes max per row, sums and divides by n_samples, with default random_state=0",
      "Explains seed selection logic: all data points if bin_seeding=False, otherwise get_bin_seeds bins points by rounding(point/bin_size), counts via defaultdict, filters by min_bin_freq, stores as float32, and multiplies back by bin_size",
      "Describes the hill-climbing loop in _mean_shift_single_seed: convergence threshold is 1e-3 * bandwidth, uses radius_neighbors to find points within bandwidth, computes mean, checks norm against threshold or max_iter",
      "Explains post-processing duplicate removal: centers sorted by (intensity, coordinates) descending, NearestNeighbors fitted on sorted centers, neighbors within bandwidth marked non-unique while preserving the current (higher-intensity) center",
      "Describes label assignment: NearestNeighbors(n_neighbors=1) fitted on final centers; if cluster_all=True assigns all points to nearest center, otherwise only assigns points within bandwidth (rest get -1)",
      "Mentions that hill-climbing is parallelized via joblib.Parallel, and NearestNeighbors for the hill-climbing uses n_jobs=1 explicitly since it's nested under the parallel calls",
      "Notes that n_iter_ is the maximum of completed_iterations across all seeds"
    ],
    "key_files": [
      "sklearn/cluster/_mean_shift.py",
      "sklearn/neighbors/_unsupervised.py",
      "sklearn/utils/_chunking.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_mean_shift.py] sklearn.cluster._mean_shift.mean_shift\nPerform mean shift clustering of data using a flat kernel.\n\n    Read more in the :ref:`User Guide <mean_shift>`.\n\n    Parameters\n    ----------\n\n    X : array-like of shape (n_samples, n_features)\n        Input data.\n\n    bandwidth : float, default=None\n        Kernel bandwidth. If not None, must be in the range [0, +inf).\n\n        If None, the bandwidth is determined using a heuristic based on\n        the median of all pairwise distances. This will take quadratic time in\n        the number of samples. The sklearn.cluster.estimate_bandwidth function\n        can be used to do this more efficiently.\n\n    seeds : array-like of shape (n_seeds, n_features) or None\n        Point used as initial kernel locations. If None and bin_seeding=False,\n        each data point is used as a seed. If None and bin_seeding=True,\n        see bin_seeding.\n\n    bin_seeding : bool, default=False\n        If true, initial kernel locations are not locations of all\n        points, but rather the location of the discretized version of\n        points, where points are binned onto a grid whose coarseness\n        corresponds to the bandwidth. Setting this option to True will speed\n        up the algorithm because fewer seeds will be initialized.\n        Ignored if seeds argument is not None.\n\n    min_bin_freq : int, default=1\n       To speed up the algorithm, accept only those bins with at least\n       min_bin_freq points as seeds.\n\n    cluster_all : bool, default=True\n        If true, then all points are clustered, even those orphans that are\n        not within any kernel. Orphans are assigned to the nearest kernel.\n        If false, then orphans are given cluster label -1.\n\n    max_iter : int, default=300\n        Maximum number of iterations, per seed point before the clustering\n        operation terminates (for that seed point), if has not converged yet.\n\n    n_jobs : int, default=None\n        The number of jobs to use for the computation. The following tasks benefit\n        from the parallelization:\n\n        - The search of nearest neighbors for bandwidth estimation and label\n          assignments. See the details in the docstring of the\n          ``NearestNeighbors`` class.\n        - Hill-climbing optimization for all seeds.\n\n        See :term:`Glossary <n_jobs>` for more details.\n\n        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n        for more details.\n\n        .. versionadded:: 0.17\n           Parallel Execution using *n_jobs*.\n\n    Returns\n    -------\n\n    cluster_centers : ndarray of shape (n_clusters, n_features)\n        Coordinates of cluster centers.\n\n    labels : ndarray of shape (n_samples,)\n        Cluster labels for each point.\n\n    Notes\n    -----\n    For a usage example, see\n    :ref:`sphx_glr_auto_examples_cluster_plot_mean_shift.py`.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import mean_shift\n    >>> X = np.array([[1, 1], [2, 1], [1, 0],\n    ...               [4, 7], [3, 5], [3, 6]])\n    >>> cluster_centers, labels = mean_shift(X, bandwidth=2)\n    >>> cluster_centers\n    array([[3.33, 6.     ],\n           [1.33, 0.66]])\n    >>> labels\n    array([1, 1, 1, 0, 0, 0])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states bandwidth range is [0, +inf) (closed='left'), but the answer claims the constraint is Interval(Real, 0, None, closed='neither'), meaning strictly greater than 0. The doc says 'must be in the range [0, +inf)' which includes 0, contradicting the answer's claim of open interval. However, this is a code-level detail that might reflect actual implementation vs documentation wording.",
      "Many implementation details (e.g., estimate_bandwidth internals, get_bin_seeds using np.round, stop_thresh=1e-3*bandwidth, NearestNeighbors with n_jobs=1, duplicate removal logic, predict method using pairwise_distances_argmin, n_iter_ attribute) are not verifiable from the provided documentation alone \u2014 they appear to be based on source code inspection rather than the docstring.",
      "The answer states bandwidth constraint is strictly greater than 0, but the documentation says 'must be in the range [0, +inf)' which includes 0. This is a potential contradiction, though it may reflect a discrepancy between the docstring and actual code constraints."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites for design rationale about n_jobs=1 and default metric assumptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_25",
    "repo": "scikit_learn",
    "question": "What is the architecture of `kmeans_plusplus` \u2014 how does its public/private split work, what algorithm does it use, and where else in sklearn is it consumed?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `kmeans_plusplus` function in `sklearn/cluster/_kmeans.py` is a public wrapper around the private `_kmeans_plusplus` function. Here's the architecture:\n\n**Public function `kmeans_plusplus`** (line 74):\n- Decorated with `@validate_params` that enforces `n_clusters` must be an `Interval(Integral, 1, None, closed='left')` and `n_local_trials` must be either `None` or `Interval(Integral, 1, None, closed='left')`.\n- Validates input using `check_array(X, accept_sparse='csr', dtype=[np.float64, np.float32])`, meaning only CSR sparse matrices are accepted and only float32/float64 dtypes.\n- Calls `_check_sample_weight(sample_weight, X, dtype=X.dtype)` to validate and normalize weights.\n- Computes `x_squared_norms` via `row_norms(X, squared=True)` from `sklearn.utils.extmath` if not provided. `row_norms` uses `np.einsum('ij,ij->i', X, X)` for dense arrays and `csr_row_norms` for sparse.\n- Delegates to private `_kmeans_plusplus(X, n_clusters, x_squared_norms, sample_weight, random_state, n_local_trials)`.\n\n**Private function `_kmeans_plusplus`** (line 180):\n- When `n_local_trials` is None, it defaults to `2 + int(np.log(n_clusters))` (natural log, not log2).\n- The first center is chosen via `random_state.choice(n_samples, p=sample_weight / sample_weight.sum())` \u2014 weighted random selection.\n- The `indices` array is initialized with `np.full(n_clusters, -1, dtype=int)`.\n- Uses `_euclidean_distances` from `sklearn.metrics.pairwise` (not the public `euclidean_distances`) with `squared=True` for distance computations. This private function handles float32 data by upcasting to float64 in chunks via `_euclidean_distances_upcast`.\n- The potential (`current_pot`) is computed as `closest_dist_sq @ sample_weight` (dot product of squared distances with sample weights).\n- Candidate sampling uses `np.searchsorted(np.cumsum(sample_weight * closest_dist_sq), rand_vals)` with `np.clip` to handle numerical imprecision.\n- After computing distances to all candidates, uses `np.minimum` to track closest distances and `np.argmin(candidates_pot)` to select the best candidate.\n- For sparse X, converts rows via `.toarray()` when assigning centers.\n\n**Integration with the broader system:**\n- `_BaseKMeans` inherits from `ClassNamePrefixFeaturesOutMixin, TransformerMixin, ClusterMixin, BaseEstimator, ABC`.\n- Both `KMeans` and `MiniBatchKMeans` extend `_BaseKMeans`.\n- `sklearn.mixture._base` imports `kmeans_plusplus` directly (public API) to initialize Gaussian Mixture Models when `init_params='k-means++'`, using it with `self.n_components` as the number of clusters.\n- The function is exported from `sklearn.cluster.__init__` and listed in `__all__`.",
    "rubric": [
      "Public `kmeans_plusplus` is decorated with `@validate_params` enforcing `n_clusters` and `n_local_trials` as Interval(Integral, 1, None, closed='left'), with n_local_trials also accepting None",
      "Public wrapper validates input via `check_array` accepting only CSR sparse and float32/float64 dtypes, and calls `_check_sample_weight` and `row_norms(X, squared=True)` for squared norms",
      "Public wrapper delegates to private `_kmeans_plusplus` which contains the actual algorithm logic",
      "When `n_local_trials` is None, it defaults to `2 + int(np.log(n_clusters))` (natural log)",
      "First center is chosen via weighted random selection using `random_state.choice` with probabilities proportional to sample weights",
      "Uses `_euclidean_distances` (private, not public `euclidean_distances`) with `squared=True` for distance computation",
      "Candidate sampling uses `np.searchsorted` on cumulative sum of `sample_weight * closest_dist_sq`, with `np.clip` to handle numerical imprecision",
      "Best candidate is selected via `np.argmin(candidates_pot)` where potential is the dot product of squared distances with sample weights",
      "For sparse X, centers are assigned via `.toarray()` conversion",
      "`_BaseKMeans` (parent of `KMeans` and `MiniBatchKMeans`) inherits from ClassNamePrefixFeaturesOutMixin, TransformerMixin, ClusterMixin, BaseEstimator, ABC",
      "`sklearn.mixture._base` imports and uses `kmeans_plusplus` to initialize Gaussian Mixture Models when `init_params='k-means++'` using `self.n_components` as the number of clusters"
    ],
    "key_files": [
      "sklearn/cluster/_kmeans.py",
      "sklearn/metrics/pairwise.py",
      "sklearn/utils/extmath.py",
      "sklearn/mixture/_base.py",
      "sklearn/cluster/__init__.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_kmeans.py] sklearn.cluster._kmeans.kmeans_plusplus\nInit n_clusters seeds according to k-means++.\n\n    .. versionadded:: 0.24\n\n    Parameters\n    ----------\n    X : {array-like, sparse matrix} of shape (n_samples, n_features)\n        The data to pick seeds from.\n\n    n_clusters : int\n        The number of centroids to initialize.\n\n    sample_weight : array-like of shape (n_samples,), default=None\n        The weights for each observation in `X`. If `None`, all observations\n        are assigned equal weight. `sample_weight` is ignored if `init`\n        is a callable or a user provided array.\n\n        .. versionadded:: 1.3\n\n    x_squared_norms : array-like of shape (n_samples,), default=None\n        Squared Euclidean norm of each data point.\n\n    random_state : int or RandomState instance, default=None\n        Determines random number generation for centroid initialization. Pass\n        an int for reproducible output across multiple function calls.\n        See :term:`Glossary <random_state>`.\n\n    n_local_trials : int, default=None\n        The number of seeding trials for each center (except the first),\n        of which the one reducing inertia the most is greedily chosen.\n        Set to None to make the number of trials depend logarithmically\n        on the number of seeds (2+log(k)) which is the recommended setting.\n        Setting to 1 disables the greedy cluster selection and recovers the\n        vanilla k-means++ algorithm which was empirically shown to work less\n        well than its greedy variant.\n\n    Returns\n    -------\n    centers : ndarray of shape (n_clusters, n_features)\n        The initial centers for k-means.\n\n    indices : ndarray of shape (n_clusters,)\n        The index location of the chosen centers in the data array X. For a\n        given index and center, X[index] = center.\n\n    Notes\n    -----\n    Selects initial cluster centers for k-mean clustering in a smart way\n    to speed up convergence. see: Arthur, D. and Vassilvitskii, S.\n    \"k-means++: the advantages of careful seeding\". ACM-SIAM symposium\n    on Discrete algorithms. 2007\n\n    Examples\n    --------\n\n    >>> from sklearn.cluster import kmeans_plusplus\n    >>> import numpy as np\n    >>> X = np.array([[1, 2], [1, 4], [1, 0],\n    ...               [10, 2], [10, 4], [10, 0]])\n    >>> centers, indices = kmeans_plusplus(X, n_clusters=2, random_state=0)\n    >>> centers\n    array([[10,  2],\n           [ 1,  0]])\n    >>> indices\n    array([3, 2])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor removals for doc-dependent descriptions and unshown code references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_26",
    "repo": "scikit_learn",
    "question": "Why does sklearn's KMeans implementation use both label-equality and tolerance-based convergence checks, center the data, limit BLAS threads only for Lloyd, and rerun the E-step on non-strict convergence?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `k_means` function is a thin wrapper that delegates entirely to the `KMeans` class. Here are the key design decisions found in the actual code:\n\n**1. Delegation pattern:** The `k_means` function simply instantiates a `KMeans` estimator and calls `.fit()`, then extracts attributes (`cluster_centers_`, `labels_`, `inertia_`, `n_iter_`).\n\n**2. Dual convergence checking (both lloyd and elkan):** Both `_kmeans_single_lloyd` and `_kmeans_single_elkan` employ a two-tier convergence strategy. First, they check for *strict convergence* via `np.array_equal(labels, labels_old)` \u2014 if labels haven't changed, the algorithm has truly converged. Only if labels differ do they fall back to the tolerance-based check: `center_shift_tot = (center_shift**2).sum()` compared against `tol`. The `_center_shift` Cython function computes the *non-squared* Euclidean distance per cluster (calling `_euclidean_dense_dense` with `squared=False`), so the convergence metric is actually the sum of squared Euclidean distances (Frobenius norm squared) of center movements. The tolerance (`_tol`) is itself data-dependent: `_tolerance()` computes `np.mean(variances) * tol`, making it relative to the data's scale.\n\n**3. E-step rerun for label consistency:** When convergence is *not* strict (i.e., it converged by tolerance or hit max_iter), both lloyd and elkan rerun the E-step with `update_centers=False`.\n\n**4. Data centering:** For dense data, `KMeans.fit()` subtracts the data mean (`X -= X_mean`) before running the algorithm. The mean is added back to the best centers after all runs complete. If `copy_x=False`, the original data is also restored (`X += X_mean`), though with a warning about small numerical differences.\n\n**5. `_is_same_clustering` for run selection:** When selecting the best run among `n_init` runs, the code uses `_is_same_clustering` (a Cython function in `_k_means_common.pyx`) to check if two label arrays represent the same clustering up to label permutation. A new run replaces the best only if it has better inertia AND a genuinely different clustering.\n\n**6. BLAS thread limiting only on Lloyd:** `_kmeans_single_lloyd` is decorated with `@_threadpool_controller_decorator(limits=1, user_api=\"blas\")` to limit BLAS threads to 1. `_kmeans_single_elkan` does NOT have this decorator. Lloyd's `_update_chunk_dense` uses `_gemm` (BLAS) for distance computation, which explains the need for thread limiting on Lloyd.\n\n**7. Empty cluster relocation:** The `_relocate_empty_clusters_dense` function (in `_k_means_common.pyx`) handles clusters that lose all members during an iteration by reassigning the sample farthest from its center to the empty cluster. If all distances are zero, relocation is skipped. The `_average_centers` function also handles empty clusters by placing them at the location of the largest cluster.\n\n**8. `n_init='auto'` logic:** When `n_init='auto'`, the code sets `_n_init=1` for 'k-means++' and array-like init, but uses `default_n_init` for 'random' and callable init (which need more restarts). Additionally, if an array-like init is passed with `n_init > 1`, a RuntimeWarning is issued and `_n_init` is forced to 1.\n\n**9. Lloyd uses BLAS GEMM for distance computation:** The `_update_chunk_dense` function computes `||X-C||\u00b2 = ||X||\u00b2 - 2X\u00b7C^T + ||C||\u00b2` but optimizes by only computing `-2X\u00b7C^T + ||C||\u00b2` since `||X||\u00b2` doesn't affect the argmin. The `-2X\u00b7C^T` term is computed via `_gemm` (BLAS), making it highly optimized for dense data.\n\n**10. CHUNK_SIZE = 256:** Defined in `_k_means_common.pyx` as a global constant. This controls data chunking for parallel processing. The MKL/vcomp memory leak check (`_check_mkl_vcomp`) specifically uses this to compute `n_active_threads = ceil(n_samples / CHUNK_SIZE)`.",
    "rubric": [
      "Explains the dual convergence strategy: strict convergence via np.array_equal on labels is checked first, falling back to tolerance-based center_shift check only when labels differ",
      "Explains that the tolerance is data-dependent, computed as np.mean(variances) * tol via the _tolerance function",
      "Explains data centering (subtracting X_mean) is done for numerical accuracy in distance computations, with the mean added back to centers after fitting",
      "Explains the E-step rerun with update_centers=False when convergence is not strict ensures labels are consistent with final centers",
      "Explains that only Lloyd (not Elkan) has the @_threadpool_controller_decorator limiting BLAS threads to 1, because Lloyd uses BLAS GEMM for distance computation which would cause oversubscription in nested parallelism",
      "Explains the _is_same_clustering check when selecting best run: a new run replaces best only if it has better inertia AND represents a genuinely different clustering (to handle permuted labels with slightly better inertia due to rounding)",
      "Explains that n_init='auto' sets _n_init=1 for k-means++ and array-like init, but uses default_n_init for random and callable init",
      "Explains that k_means function is a thin wrapper that delegates to KMeans.fit() and extracts cluster_centers_, labels_, inertia_, and optionally n_iter_"
    ],
    "key_files": [
      "sklearn/cluster/_kmeans.py",
      "sklearn/cluster/_k_means_common.pyx",
      "sklearn/cluster/_k_means_lloyd.pyx",
      "sklearn/utils/parallel.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_kmeans.py] sklearn.cluster._kmeans.k_means\nPerform K-means clustering algorithm.\n\n    Read more in the :ref:`User Guide <k_means>`.\n\n    Parameters\n    ----------\n    X : {array-like, sparse matrix} of shape (n_samples, n_features)\n        The observations to cluster. It must be noted that the data\n        will be converted to C ordering, which will cause a memory copy\n        if the given data is not C-contiguous.\n\n    n_clusters : int\n        The number of clusters to form as well as the number of\n        centroids to generate.\n\n    sample_weight : array-like of shape (n_samples,), default=None\n        The weights for each observation in `X`. If `None`, all observations\n        are assigned equal weight. `sample_weight` is not used during\n        initialization if `init` is a callable or a user provided array.\n\n    init : {'k-means++', 'random'}, callable or array-like of shape             (n_clusters, n_features), default='k-means++'\n        Method for initialization:\n\n        - `'k-means++'` : selects initial cluster centers for k-mean\n          clustering in a smart way to speed up convergence. See section\n          Notes in k_init for more details.\n        - `'random'`: choose `n_clusters` observations (rows) at random from data\n          for the initial centroids.\n        - If an array is passed, it should be of shape `(n_clusters, n_features)`\n          and gives the initial centers.\n        - If a callable is passed, it should take arguments `X`, `n_clusters` and a\n          random state and return an initialization.\n\n    n_init : 'auto' or int, default=\"auto\"\n        Number of time the k-means algorithm will be run with different\n        centroid seeds. The final results will be the best output of\n        n_init consecutive runs in terms of inertia.\n\n        When `n_init='auto'`, the number of runs depends on the value of init:\n        10 if using `init='random'` or `init` is a callable;\n        1 if using `init='k-means++'` or `init` is an array-like.\n\n        .. versionadded:: 1.2\n           Added 'auto' option for `n_init`.\n\n        .. versionchanged:: 1.4\n           Default value for `n_init` changed to `'auto'`.\n\n    max_iter : int, default=300\n        Maximum number of iterations of the k-means algorithm to run.\n\n    verbose : bool, default=False\n        Verbosity mode.\n\n    tol : float, default=1e-4\n        Relative tolerance with regards to Frobenius norm of the difference\n        in the cluster centers of two consecutive iterations to declare\n        convergence.\n\n    random_state : int, RandomState instance or None, default=None\n        Determines random number generation for centroid initialization. Use\n        an int to make the randomness deterministic.\n        See :term:`Glossary <random_state>`.\n\n    copy_x : bool, default=True\n        When pre-computing distances it is more numerically accurate to center\n        the data first. If `copy_x` is True (default), then the original data is\n        not modified. If False, the original data is modified, and put back\n        before the function returns, but small numerical differences may be\n        introduced by subtracting and then adding the data mean. Note that if\n        the original data is not C-contiguous, a copy will be made even if\n        `copy_x` is False. If the original data is sparse, but not in CSR format,\n        a copy will be made even if `copy_x` is False.\n\n    algorithm : {\"lloyd\", \"elkan\"}, default=\"lloyd\"\n        K-means algorithm to use. The classical EM-style algorithm is `\"lloyd\"`.\n        The `\"elkan\"` variation can be more efficient on some datasets with\n        well-defined clusters, by using the triangle inequality. However it's\n        more memory intensive due to the allocation of an extra array of shape\n        `(n_samples, n_clusters)`.\n\n        .. versionchanged:: 0.18\n            Added Elkan algorithm\n\n        .. versionchanged:: 1.1\n            Renamed \"full\" to \"lloyd\", and deprecated \"auto\" and \"full\".\n            Changed \"auto\" to use \"lloyd\" instead of \"elkan\".\n\n    return_n_iter : bool, default=False\n        Whether or not to return the number of iterations.\n\n    Returns\n    -------\n    centroid : ndarray of shape (n_clusters, n_features)\n        Centroids found at the last iteration of k-means.\n\n    label : ndarray of shape (n_samples,)\n        The `label[i]` is the code or index of the centroid the\n        i'th observation is closest to.\n\n    inertia : float\n        The final value of the inertia criterion (sum of squared distances to\n        the closest centroid for all observations in the training set).\n\n    best_n_iter : int\n        Number of iterations corresponding to the best results.\n        Returned only if `return_n_iter` is set to True.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import k_means\n    >>> X = np.array([[1, 2], [1, 4], [1, 0],\n    ...               [10, 2], [10, 4], [10, 0]])\n    >>> centroid, label, inertia = k_means(\n    ...     X, n_clusters=2, n_init=\"auto\", random_state=0\n    ... )\n    >>> centroid\n    array([[10.,  2.],\n           [ 1.,  2.]])\n    >>> label\n    array([1, 1, 1, 0, 0, 0], dtype=int32)\n    >>> inertia\n    16.0",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., 'lines 445-459', 'lines 1486-1493') that cannot be verified from the documentation alone - these are plausible but unverifiable claims about source code.",
      "Point 5 claims '_is_same_clustering' prevents replacing best run if clustering is the same despite better inertia. The documentation doesn't mention this behavior, making it an unverifiable implementation detail.",
      "Point 6 about BLAS thread limiting decorator is an implementation detail not covered by documentation - plausible but unverifiable.",
      "Point 9's claim about Lloyd using GEMM for distance computation is an implementation detail not in the documentation.",
      "Point 10 about CHUNK_SIZE=256 is an implementation detail not mentioned in the documentation.",
      "The answer claims the function 'simply instantiates a KMeans estimator and calls .fit()' - this is a plausible implementation detail but not stated in the documentation.",
      "The n_init='auto' description in point 8 is consistent with the documentation (10 for random/callable, 1 for k-means++/array-like).",
      "The convergence tolerance description mentions '_tolerance() computes np.mean(variances) * tol' - the documentation only says 'Relative tolerance with regards to Frobenius norm of the difference in the cluster centers' which is consistent but the specific formula is unverifiable from the doc alone."
    ],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rationale explanations and direct comment quotes were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_29",
    "repo": "scikit_learn",
    "question": "What is `compute_optics_graph` and how does it relate to the OPTICS class's fit method?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`compute_optics_graph` is a standalone function (decorated with `@validate_params`) that computes the OPTICS reachability graph (as indicated by its name and return values: ordering, core_distances_, reachability_, predecessor_). It is the core computation engine behind the `OPTICS` class's `fit` method. Here is what the function does, based on actual code inspection:\n\n**Architecture and Call Chain:**\n- The `OPTICS` class (which inherits from `ClusterMixin` and `BaseEstimator`) calls `compute_optics_graph` in its `fit()` method, wrapping it through `memory.cache(compute_optics_graph)(...)` to enable optional joblib caching via the `memory` parameter.\n- All parameters to `compute_optics_graph` are keyword-only (after `X`, there's a `*` separator in the signature).\n\n**Key Implementation Details:**\n\n1. **Initialization**: Reachability distances are initialized as a NumPy array filled with `np.inf`, and predecessors are initialized to `-1` (all as int dtype). The `processed` array is a boolean array of zeros.\n\n2. **min_samples conversion**: When `min_samples` is a float \u2264 1, it is converted to `max(2, int(min_samples * n_samples))`. The `_validate_size` helper raises a `ValueError` if `min_samples > n_samples`.\n\n3. **Core distance computation**: `_compute_core_distances_` uses `NearestNeighbors.kneighbors()` in chunks. The row byte calculation uses `16 * min_samples` bytes per row, and chunks are generated via `get_chunk_n_rows()` from `sklearn.utils._chunking`, which respects `sklearn.get_config()['working_memory']`. The core distance for each point is the distance to its k-th nearest neighbor (the last column of the kneighbors distance matrix: `neighbors.kneighbors(X[sl], min_samples)[0][:, -1]`).\n\n4. **Rounding**: Both core distances and reachability distances are explicitly rounded using `np.around` to `np.finfo(dtype).precision` decimal places.\n\n5. **Core distances capping**: After computing core distances, any value exceeding `max_eps` is set to `np.inf`.\n\n6. **Main loop (O(n\u00b2))**: The main OPTICS loop iterates `n_samples` times. At each step, it selects the unprocessed point with the smallest reachability distance (using `np.argmin` on unprocessed indices), marks it as processed, adds it to the ordering, and then calls `_set_reach_dist` if the point's core distance is not infinite.\n\n7. **`_set_reach_dist` function**: This private function first finds all neighbors within `max_eps` radius using `nbrs.radius_neighbors(P, radius=max_eps, return_distance=False)`. It then filters to only unprocessed neighbors using `np.compress(~np.take(processed, indices), indices)`. Distances to unprocessed points are computed separately (either from the precomputed matrix or via `pairwise_distances`). Reachability distances are the maximum of the pairwise distance and the current point's core distance (`np.maximum(dists, core_distances_[point_index])`). Only improved (smaller) reachability values are updated.\n\n8. **Minkowski p handling in `_set_reach_dist`**: When metric is 'minkowski' and 'p' is not in `metric_params`, the `p` parameter is explicitly added to the distance computation params.\n\n9. **Warning**: After the main loop, if all reachability values remain `np.inf`, a `UserWarning` is issued suggesting a larger `max_eps`.\n\n10. **Boolean metrics**: In the `OPTICS.fit()` method (but not in `compute_optics_graph` itself), when the metric is a boolean function (from `PAIRWISE_BOOLEAN_FUNCTIONS` like 'dice', 'jaccard', 'rogerstanimoto', 'russellrao', 'sokalsneath', 'yule'), data is converted to boolean dtype, with a `DataConversionWarning` if the input wasn't already boolean.\n\n11. **Sparse precomputed handling**: In `OPTICS.fit()`, if metric is 'precomputed' and X is sparse, X is copied and its diagonal is explicitly set (`X.setdiag(X.diagonal())`), with `SparseEfficiencyWarning` suppressed.\n\n12. **Validation constraints differ**: The `@validate_params` decorator for `compute_optics_graph` allows `p` to be `None` or `Interval(Real, 0, None, closed=\"right\")`, while the `OPTICS` class constrains `p` as `Interval(Real, 1, None, closed=\"left\")`.\n\n13. **NearestNeighbors class**: `NearestNeighbors` inherits from `KNeighborsMixin`, `RadiusNeighborsMixin`, and `NeighborsBase`, defined in `sklearn/neighbors/_unsupervised.py`.",
    "rubric": [
      "Explains that compute_optics_graph is a standalone function (decorated with @validate_params) that computes the OPTICS reachability graph",
      "Mentions that OPTICS.fit() calls compute_optics_graph, optionally wrapped with memory.cache for joblib caching",
      "Describes that reachability distances are initialized to np.inf and predecessors to -1",
      "Explains min_samples float-to-int conversion using max(2, int(min_samples * n_samples))",
      "Describes core distance computation via NearestNeighbors.kneighbors using the distance to the k-th nearest neighbor",
      "Mentions that core distances and reachability distances are rounded using np.around to the dtype's precision",
      "Describes the main loop selecting the unprocessed point with smallest reachability distance at each iteration",
      "Explains _set_reach_dist finds radius neighbors within max_eps, filters to unprocessed points, and updates reachability as max(pairwise_distance, core_distance)",
      "Mentions that a UserWarning is issued if all reachability values remain infinity after the loop",
      "Notes that the function returns ordering, core_distances_, reachability_, and predecessor_"
    ],
    "key_files": [
      "sklearn/cluster/_optics.py",
      "sklearn/neighbors/_unsupervised.py",
      "sklearn/utils/_chunking.py",
      "sklearn/metrics/pairwise.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_optics.py] sklearn.cluster._optics.compute_optics_graph\nCompute the OPTICS reachability graph.\n\n    Read more in the :ref:`User Guide <optics>`.\n\n    Parameters\n    ----------\n    X : {ndarray, sparse matrix} of shape (n_samples, n_features), or             (n_samples, n_samples) if metric='precomputed'\n        A feature array, or array of distances between samples if\n        metric='precomputed'.\n\n    min_samples : int > 1 or float between 0 and 1\n        The number of samples in a neighborhood for a point to be considered\n        as a core point. Expressed as an absolute number or a fraction of the\n        number of samples (rounded to be at least 2).\n\n    max_eps : float, default=np.inf\n        The maximum distance between two samples for one to be considered as\n        in the neighborhood of the other. Default value of ``np.inf`` will\n        identify clusters across all scales; reducing ``max_eps`` will result\n        in shorter run times.\n\n    metric : str or callable, default='minkowski'\n        Metric to use for distance computation. Any metric from scikit-learn\n        or scipy.spatial.distance can be used.\n\n        If metric is a callable function, it is called on each\n        pair of instances (rows) and the resulting value recorded. The callable\n        should take two arrays as input and return one value indicating the\n        distance between them. This works for Scipy's metrics, but is less\n        efficient than passing the metric name as a string. If metric is\n        \"precomputed\", X is assumed to be a distance matrix and must be square.\n\n        Valid values for metric are:\n\n        - from scikit-learn: ['cityblock', 'cosine', 'euclidean', 'l1', 'l2',\n          'manhattan']\n\n        - from scipy.spatial.distance: ['braycurtis', 'canberra', 'chebyshev',\n          'correlation', 'dice', 'hamming', 'jaccard', 'kulsinski',\n          'mahalanobis', 'minkowski', 'rogerstanimoto', 'russellrao',\n          'seuclidean', 'sokalmichener', 'sokalsneath', 'sqeuclidean',\n          'yule']\n\n        See the documentation for scipy.spatial.distance for details on these\n        metrics.\n\n        .. note::\n           `'kulsinski'` is deprecated from SciPy 1.9 and will be removed in SciPy 1.11.\n\n    p : float, default=2\n        Parameter for the Minkowski metric from\n        :class:`~sklearn.metrics.pairwise_distances`. When p = 1, this is\n        equivalent to using manhattan_distance (l1), and euclidean_distance\n        (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.\n\n    metric_params : dict, default=None\n        Additional keyword arguments for the metric function.\n\n    algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto'\n        Algorithm used to compute the nearest neighbors:\n\n        - 'ball_tree' will use :class:`~sklearn.neighbors.BallTree`.\n        - 'kd_tree' will use :class:`~sklearn.neighbors.KDTree`.\n        - 'brute' will use a brute-force search.\n        - 'auto' will attempt to decide the most appropriate algorithm\n          based on the values passed to `fit` method. (default)\n\n        Note: fitting on sparse input will override the setting of\n        this parameter, using brute force.\n\n    leaf_size : int, default=30\n        Leaf size passed to :class:`~sklearn.neighbors.BallTree` or\n        :class:`~sklearn.neighbors.KDTree`. This can affect the speed of the\n        construction and query, as well as the memory required to store the\n        tree. The optimal value depends on the nature of the problem.\n\n    n_jobs : int, default=None\n        The number of parallel jobs to run for neighbors search.\n        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n        for more details.\n\n    Returns\n    -------\n    ordering_ : array of shape (n_samples,)\n        The cluster ordered list of sample indices.\n\n    core_distances_ : array of shape (n_samples,)\n        Distance at which each sample becomes a core point, indexed by object\n        order. Points which will never be core have a distance of inf. Use\n        ``clust.core_distances_[clust.ordering_]`` to access in cluster order.\n\n    reachability_ : array of shape (n_samples,)\n        Reachability distances per sample, indexed by object order. Use\n        ``clust.reachability_[clust.ordering_]`` to access in cluster order.\n\n    predecessor_ : array of shape (n_samples,)\n        Point that a sample was reached from, indexed by object order.\n        Seed points have a predecessor of -1.\n\n    References\n    ----------\n    .. [1] Ankerst, Mihael, Markus M. Breunig, Hans-Peter Kriegel,\n       and J\u00f6rg Sander. \"OPTICS: ordering points to identify the clustering\n       structure.\" ACM SIGMOD Record 28, no. 2 (1999): 49-60.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import compute_optics_graph\n    >>> X = np.array([[1, 2], [2, 5], [3, 6],\n    ...               [8, 7], [8, 8], [7, 3]])\n    >>> ordering, core_distances, reachability, predecessor = compute_optics_graph(\n    ...     X,\n    ...     min_samples=2,\n    ...     max_eps=np.inf,\n    ...     metric=\"minkowski\",\n    ...     p=2,\n    ...     metric_params=None,\n    ...     algorithm=\"auto\",\n    ...     leaf_size=30,\n    ...     n_jobs=None,\n    ... )\n    >>> ordering\n    array([0, 1, 2, 5, 3, 4])\n    >>> core_distances\n    array([3.16, 1.41, 1.41, 1.        , 1.        ,\n           4.12])\n    >>> reachability\n    array([       inf, 3.16, 1.41, 4.12, 1.        ,\n           5.        ])\n    >>> predecessor\n    array([-1,  0,  1,  5,  3,  2])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Very minor doc leakage: a few rationale phrases ('for memory efficiency', 'to avoid floating-point artifacts', 'to ensure each point is its own neighbor', 'mirrors NearestNeighbors behavior') and one claim about __init__.py export were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_30",
    "repo": "scikit_learn",
    "question": "Why does `cluster_optics_dbscan` use `np.cumsum` on boolean masks instead of iterating, and how does its two-step labeling handle noise points?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `cluster_optics_dbscan` function uses a vectorized, linear-time algorithm to extract DBSCAN-like clusters from precomputed OPTICS results, rather than re-running the full DBSCAN algorithm. The key design decisions are:\n\n1. **Why use `cumsum` with boolean masks instead of iterative cluster assignment**: The implementation uses `np.cumsum(far_reach[ordering] & near_core[ordering]) - 1` to assign cluster labels. This exploits the OPTICS ordering property: in the ordered sequence, a new cluster boundary is detected whenever a point has reachability > eps (far_reach) AND core_distance <= eps (near_core). Such a point starts a new cluster. The cumulative sum increments the cluster ID at each such boundary, achieving O(n) cluster assignment through vectorized numpy operations rather than looping.\n\n2. **Why points with `far_reach & ~near_core` are labeled as noise (-1)**: Points that have reachability > eps AND are not core points (core_distance > eps) are explicitly overwritten as noise (label = -1) via `labels[far_reach & ~near_core] = -1`. This two-step approach (first assign via cumsum, then correct non-core far-reach points) avoids complex conditional logic.\n\n3. **Why the function is decoupled from the OPTICS graph computation**: The function takes precomputed `reachability`, `core_distances`, and `ordering` arrays, allowing users to extract clusters at multiple epsilon values without re-running the expensive O(n\u00b2) OPTICS graph computation. This is demonstrated in `examples/cluster/plot_optics.py` where `cluster_optics_dbscan` is called twice with eps=0.5 and eps=2 on the same OPTICS results.\n\n4. **Why there's no `eps > max_eps` validation in the standalone function**: Unlike the OPTICS.fit() method which validates `eps <= max_eps` and raises a ValueError, the standalone `cluster_optics_dbscan` function only validates that eps >= 0 via `@validate_params` with `Interval(Real, 0, None, closed='both')`. The OPTICS class handles this constraint because it knows `max_eps`, but the standalone function has no max_eps parameter and works with arbitrary precomputed arrays.\n\n5. **Why `core_distances > max_eps` are set to inf in `compute_optics_graph`**: Core distances exceeding `max_eps` are set to infinity via `core_distances_[core_distances_ > max_eps] = np.inf`. This means in `cluster_optics_dbscan`, the `near_core = core_distances <= eps` check will always be False for such points (since eps should be < max_eps), causing them to never start new clusters \u2014 they can only be noise or assigned to a cluster if their reachability is <= eps.\n\n6. **Why all parameters are keyword-only (using `*`)**: The function signature `def cluster_optics_dbscan(*, reachability, core_distances, ordering, eps)` forces all arguments to be keyword-only using the `*` syntax.",
    "rubric": [
      "Explains that np.cumsum on `far_reach[ordering] & near_core[ordering]` detects cluster boundaries where reachability > eps AND core_distance <= eps, incrementing a cluster ID at each boundary in O(n) vectorized fashion.",
      "Explains the two-step approach: first assigns all labels via cumsum, then overwrites points with far_reach & ~near_core as noise (-1), avoiding complex conditional logic.",
      "Mentions the function is decoupled from compute_optics_graph, taking precomputed arrays so users can extract clusters at multiple epsilon values without re-running the expensive OPTICS computation.",
      "Notes the difference in eps validation: the OPTICS class checks eps <= max_eps and raises ValueError, while the standalone function only validates eps >= 0 since it has no knowledge of max_eps.",
      "Explains that core_distances > max_eps are set to np.inf in compute_optics_graph, so near_core will always be False for those points and they can only be noise or passively assigned to a cluster.",
      "Notes the keyword-only parameter design using * in the function signature."
    ],
    "key_files": [
      "sklearn/cluster/_optics.py",
      "examples/cluster/plot_optics.py",
      "sklearn/cluster/__init__.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_optics.py] sklearn.cluster._optics.cluster_optics_dbscan\nPerform DBSCAN extraction for an arbitrary epsilon.\n\n    Extracting the clusters runs in linear time. Note that this results in\n    ``labels_`` which are close to a :class:`~sklearn.cluster.DBSCAN` with\n    similar settings and ``eps``, only if ``eps`` is close to ``max_eps``.\n\n    Parameters\n    ----------\n    reachability : ndarray of shape (n_samples,)\n        Reachability distances calculated by OPTICS (``reachability_``).\n\n    core_distances : ndarray of shape (n_samples,)\n        Distances at which points become core (``core_distances_``).\n\n    ordering : ndarray of shape (n_samples,)\n        OPTICS ordered point indices (``ordering_``).\n\n    eps : float\n        DBSCAN ``eps`` parameter. Must be set to < ``max_eps``. Results\n        will be close to DBSCAN algorithm if ``eps`` and ``max_eps`` are close\n        to one another.\n\n    Returns\n    -------\n    labels_ : array of shape (n_samples,)\n        The estimated labels.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.cluster import cluster_optics_dbscan, compute_optics_graph\n    >>> X = np.array([[1, 2], [2, 5], [3, 6],\n    ...               [8, 7], [8, 8], [7, 3]])\n    >>> ordering, core_distances, reachability, predecessor = compute_optics_graph(\n    ...     X,\n    ...     min_samples=2,\n    ...     max_eps=np.inf,\n    ...     metric=\"minkowski\",\n    ...     p=2,\n    ...     metric_params=None,\n    ...     algorithm=\"auto\",\n    ...     leaf_size=30,\n    ...     n_jobs=None,\n    ... )\n    >>> eps = 4.5\n    >>> labels = cluster_optics_dbscan(\n    ...     reachability=reachability,\n    ...     core_distances=core_distances,\n    ...     ordering=ordering,\n    ...     eps=eps,\n    ... )\n    >>> labels\n    array([0, 0, 0, 1, 1, 1])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (785-788, 626, 381-384) and implementation details that cannot be verified from the provided documentation alone - these are plausible but unverifiable claims about the source code.",
      "The answer claims the function signature uses `*` to make all parameters keyword-only. The documentation's example does show keyword arguments being used (`reachability=reachability, core_distances=core_distances, ordering=ordering, eps=eps`), which is consistent with this claim, but the signature itself is not shown in the provided documentation.",
      "The answer claims `@validate_params` with `Interval(Real, 0, None, closed='both')` for eps validation - this is a specific implementation detail not present in the documentation.",
      "The answer references `examples/cluster/plot_optics.py` with specific eps values of 0.5 and 2 - not verifiable from the provided documentation.",
      "The documentation states 'Must be set to < max_eps' for the eps parameter, while the answer in point 4 claims the standalone function doesn't validate eps > max_eps - this is consistent with the documentation which only states the requirement without specifying enforcement mechanism.",
      "The answer's claim about cumsum-based implementation is plausible but entirely based on source code not shown in the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor design rationale phrasings needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_31",
    "repo": "scikit_learn",
    "question": "Where does the spectral clustering pipeline go from receiving an affinity matrix to returning cluster labels, and what are the key steps along the way?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `spectral_clustering` function in `sklearn/cluster/_spectral.py` is a thin wrapper that delegates entirely to `SpectralClustering.fit()`. Here is the complete call chain and where each piece lives:\n\n1. **`spectral_clustering()` (line 194, `sklearn/cluster/_spectral.py`)**: Creates a `SpectralClustering` instance with `affinity=\"precomputed\"`, calls `.fit(affinity)`, and returns `clusterer.labels_`. The default `n_clusters` is 8.\n\n2. **`SpectralClustering.fit()` (line 672, `sklearn/cluster/_spectral.py`)**: Constructs the affinity matrix (for 'precomputed' it just sets `self.affinity_matrix_ = X`), then calls `_spectral_embedding()` from `sklearn/manifold/_spectral_embedding.py` with `drop_first=False` (unlike `spectral_embedding()` which defaults to `drop_first=True`).\n\n3. **`_spectral_embedding()` (line 295, `sklearn/manifold/_spectral_embedding.py`)**: This is the internal (non-validated) version. It first checks symmetry with `check_symmetric(adjacency)`, then checks graph connectivity via `_graph_is_connected()` (issuing a warning if not fully connected). It computes the graph Laplacian using `csgraph_laplacian` (imported from `sklearn.utils.fixes`, which uses scipy >= 1.12's version or an internal copy for older scipy). For the `arpack` solver, it uses shift-invert mode with `sigma=1.0` and `which='LM'` via `scipy.sparse.linalg.eigsh`, negating the Laplacian first (`laplacian *= -1`). The initial vector `v0` is created by `_init_arpack_v0()` which samples uniformly from [-1, 1]. If ARPACK raises a `RuntimeError`, it falls back to the `lobpcg` solver. For `amg`, it adds a diagonal shift of `1e-5 * I` to the Laplacian before building the AMG preconditioner via `pyamg.smoothed_aggregation_solver`, then removes it before calling `lobpcg`. For `lobpcg` with small graphs (`n_nodes < 5 * n_components + 1`), it falls back to `scipy.linalg.eigh`. After computing the embedding, `_deterministic_vector_sign_flip()` is applied (from `sklearn/utils/extmath.py`), which flips each eigenvector's sign so the element with the largest absolute value is positive.\n\n4. **Label assignment**: After embedding, labels are assigned via one of three strategies:\n   - `'kmeans'`: calls `k_means()` from `sklearn/cluster/_kmeans.py` \u2014 SpectralClustering unpacks the result as `_, self.labels_, _ = k_means(...)`, keeping only `labels_`.\n   - `'discretize'`: calls `discretize()` in the same file, which iteratively searches for the closest discrete partition matrix using SVD, with `max_svd_restarts=30` and `n_iter_max=20`.\n   - `'cluster_qr'`: calls `cluster_qr()` which uses `scipy.linalg.qr` with pivoting and `scipy.linalg.svd` on a k\u00d7k submatrix (no iteration or random initialization).\n\n5. **`_set_diag()`**: When the sparse Laplacian has 7 or fewer unique diagonals, it converts to `dia` format; otherwise it converts to `csr` format.\n\n6. The `SpectralClustering` class inherits from both `ClusterMixin` and `BaseEstimator`. Its `__sklearn_tags__` method sets `tags.input_tags.pairwise = True` when `affinity` is `\"precomputed\"` or `\"precomputed_nearest_neighbors\"`.",
    "rubric": [
      "Identifies that `spectral_clustering()` is a wrapper that creates a `SpectralClustering` instance with `affinity='precomputed'` and returns `clusterer.labels_`",
      "Identifies that `SpectralClustering.fit()` calls `_spectral_embedding()` from the manifold module with `drop_first=False`",
      "Describes that `_spectral_embedding()` checks symmetry and graph connectivity, computes the graph Laplacian, and solves an eigenvalue problem (arpack uses shift-invert with sigma=1.0 and which='LM')",
      "Mentions the fallback behavior: ARPACK falls back to lobpcg on RuntimeError; lobpcg falls back to scipy.linalg.eigh for small graphs",
      "Mentions that `_deterministic_vector_sign_flip()` is applied to eigenvectors after computing the embedding",
      "Identifies the three label assignment strategies: 'kmeans' (calls k_means), 'discretize' (iterative SVD-based), and 'cluster_qr' (QR with pivoting)",
      "Mentions the AMG solver adds a diagonal shift of 1e-5*I before building the preconditioner and removes it before calling lobpcg"
    ],
    "key_files": [
      "sklearn/cluster/_spectral.py",
      "sklearn/manifold/_spectral_embedding.py",
      "sklearn/utils/extmath.py",
      "sklearn/utils/_arpack.py",
      "sklearn/utils/fixes.py"
    ],
    "source_doc": "[docstring: sklearn/cluster/_spectral.py] sklearn.cluster._spectral.spectral_clustering\nApply clustering to a projection of the normalized Laplacian.\n\n    In practice Spectral Clustering is very useful when the structure of\n    the individual clusters is highly non-convex or more generally when\n    a measure of the center and spread of the cluster is not a suitable\n    description of the complete cluster. For instance, when clusters are\n    nested circles on the 2D plane.\n\n    If affinity is the adjacency matrix of a graph, this method can be\n    used to find normalized graph cuts [1]_, [2]_.\n\n    Read more in the :ref:`User Guide <spectral_clustering>`.\n\n    Parameters\n    ----------\n    affinity : {array-like, sparse matrix} of shape (n_samples, n_samples)\n        The affinity matrix describing the relationship of the samples to\n        embed. **Must be symmetric**.\n\n        Possible examples:\n          - adjacency matrix of a graph,\n          - heat kernel of the pairwise distance matrix of the samples,\n          - symmetric k-nearest neighbours connectivity matrix of the samples.\n\n    n_clusters : int, default=None\n        Number of clusters to extract.\n\n    n_components : int, default=n_clusters\n        Number of eigenvectors to use for the spectral embedding.\n\n    eigen_solver : {None, 'arpack', 'lobpcg', or 'amg'}\n        The eigenvalue decomposition method. If None then ``'arpack'`` is used.\n        See [4]_ for more details regarding ``'lobpcg'``.\n        Eigensolver ``'amg'`` runs ``'lobpcg'`` with optional\n        Algebraic MultiGrid preconditioning and requires pyamg to be installed.\n        It can be faster on very large sparse problems [6]_ and [7]_.\n\n    random_state : int, RandomState instance, default=None\n        A pseudo random number generator used for the initialization\n        of the lobpcg eigenvectors decomposition when `eigen_solver ==\n        'amg'`, and for the K-Means initialization. Use an int to make\n        the results deterministic across calls (See\n        :term:`Glossary <random_state>`).\n\n        .. note::\n            When using `eigen_solver == 'amg'`,\n            it is necessary to also fix the global numpy seed with\n            `np.random.seed(int)` to get deterministic results. See\n            https://github.com/pyamg/pyamg/issues/139 for further\n            information.\n\n    n_init : int, default=10\n        Number of time the k-means algorithm will be run with different\n        centroid seeds. The final results will be the best output of n_init\n        consecutive runs in terms of inertia. Only used if\n        ``assign_labels='kmeans'``.\n\n    eigen_tol : float, default=\"auto\"\n        Stopping criterion for eigendecomposition of the Laplacian matrix.\n        If `eigen_tol=\"auto\"` then the passed tolerance will depend on the\n        `eigen_solver`:\n\n        - If `eigen_solver=\"arpack\"`, then `eigen_tol=0.0`;\n        - If `eigen_solver=\"lobpcg\"` or `eigen_solver=\"amg\"`, then\n          `eigen_tol=None` which configures the underlying `lobpcg` solver to\n          automatically resolve the value according to their heuristics. See,\n          :func:`scipy.sparse.linalg.lobpcg` for details.\n\n        Note that when using `eigen_solver=\"lobpcg\"` or `eigen_solver=\"amg\"`\n        values of `tol<1e-5` may lead to convergence issues and should be\n        avoided.\n\n        .. versionadded:: 1.2\n           Added 'auto' option.\n\n    assign_labels : {'kmeans', 'discretize', 'cluster_qr'}, default='kmeans'\n        The strategy to use to assign labels in the embedding\n        space.  There are three ways to assign labels after the Laplacian\n        embedding.  k-means can be applied and is a popular choice. But it can\n        also be sensitive to initialization. Discretization is another\n        approach which is less sensitive to random initialization [3]_.\n        The cluster_qr method [5]_ directly extracts clusters from eigenvectors\n        in spectral clustering. In contrast to k-means and discretization, cluster_qr\n        has no tuning parameters and is not an iterative method, yet may outperform\n        k-means and discretization in terms of both quality and speed. For a detailed\n        comparison of clustering strategies, refer to the following example:\n        :ref:`sphx_glr_auto_examples_cluster_plot_coin_segmentation.py`.\n\n        .. versionchanged:: 1.1\n           Added new labeling method 'cluster_qr'.\n\n    verbose : bool, default=False\n        Verbosity mode.\n\n        .. versionadded:: 0.24\n\n    Returns\n    -------\n    labels : array of integers, shape: n_samples\n        The labels of the clusters.\n\n    Notes\n    -----\n    The graph should contain only one connected component, elsewhere\n    the results make little sense.\n\n    This algorithm solves the normalized cut for `k=2`: it is a\n    normalized spectral clustering.\n\n    References\n    ----------\n\n    .. [1] :doi:`Normalized cuts and image segmentation, 2000\n           Jianbo Shi, Jitendra Malik\n           <10.1109/34.868688>`\n\n    .. [2] :doi:`A Tutorial on Spectral Clustering, 2007\n           Ulrike von Luxburg\n           <10.1007/s11222-007-9033-z>`\n\n    .. [3] `Multiclass spectral clustering, 2003\n           Stella X. Yu, Jianbo Shi\n           <https://people.eecs.berkeley.edu/~jordan/courses/281B-spring04/readings/yu-shi.pdf>`_\n\n    .. [4] :doi:`Toward the Optimal Preconditioned Eigensolver:\n           Locally Optimal Block Preconditioned Conjugate Gradient Method, 2001\n           A. V. Knyazev\n           SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541.\n           <10.1137/S1064827500366124>`\n\n    .. [5] :doi:`Simple, direct, and efficient multi-way spectral clustering, 2019\n           Anil Damle, Victor Minden, Lexing Ying\n           <10.1093/imaiai/iay008>`\n\n    .. [6] :doi:`Multiscale Spectral Image Segmentation Multiscale preconditioning\n           for computing eigenvalues of graph Laplacians in image segmentation, 2006\n           Andrew Knyazev\n           <10.13140/RG.2.2.35280.02565>`\n\n    .. [7] :doi:`Preconditioned spectral clustering for stochastic block partition\n           streaming graph challenge (Preliminary version at arXiv.)\n           David Zhuzhunashvili, Andrew Knyazev\n           <10.1109/HPEC.2017.8091045>`\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.metrics.pairwise import pairwise_kernels\n    >>> from sklearn.cluster import spectral_clustering\n    >>> X = np.array([[1, 1], [2, 1], [1, 0],\n    ...               [4, 7], [3, 5], [3, 6]])\n    >>> affinity = pairwise_kernels(X, metric='rbf')\n    >>> spectral_clustering(\n    ...     affinity=affinity, n_clusters=2, assign_labels=\"discretize\", random_state=0\n    ... )\n    array([1, 1, 1, 0, 0, 0])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 'The default n_clusters is 8 (not None as the docstring's parameter description suggests).' The documentation clearly states 'n_clusters : int, default=None'. While the actual implementation might use 8 as a default in the class, the function docstring explicitly says default=None, so this claim contradicts the documentation.",
      "Most of the detailed implementation claims (line numbers, internal function behavior, fallback logic, etc.) go far beyond what the documentation covers and cannot be verified from it alone. These are plausible code-level details but are unverifiable from the provided doc."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from a code comment about eigenvectors, design rationale for dia format, and internal details of k_means not shown in provided code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_34",
    "repo": "scikit_learn",
    "question": "Why does TransformedTargetRegressor end up checking the inverse twice when using func/inverse_func, and what other performance-related design choices does it make?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `TransformedTargetRegressor` uses several performance-related design decisions:\n\n1. **Subsampling for inverse check**: In `_fit_transformer`, the `check_inverse` verification only operates on a subset of the data, using `slice(None, None, max(1, y.shape[0] // 10))` \u2014 effectively sampling every 10th element (up to ~10% of data). Notably, this uses a *different* sampling ratio than `FunctionTransformer._check_inverse_transform`, which samples every 100th element (`X.shape[0] // 100`). The TTR check is thus less aggressive in subsampling (checks more data).\n\n2. **Double-checking when using func/inverse_func**: When `func` and `inverse_func` are provided (not a `transformer` object), the code creates a `FunctionTransformer` with `check_inverse=self.check_inverse`. This means `FunctionTransformer.fit()` runs its own `_check_inverse_transform`, and *then* `TransformedTargetRegressor._fit_transformer` runs a *second* inverse check. The inverse is thus checked twice with different sampling strategies.\n\n3. **`set_output(transform=\"default\")` call**: After creating the `FunctionTransformer`, the code explicitly calls `self.transformer_.set_output(transform=\"default\")`. This ensures the transformer outputs numpy arrays regardless of the global `set_output` configuration, as confirmed by the test `test_transform_target_regressor_not_warns_with_global_output_set`.\n\n4. **`_get_regressor(get_clone=False)`**: The `_get_regressor` method has a `get_clone` parameter; when `False`, it returns the regressor directly without cloning. This default is used in `__sklearn_tags__` and `get_metadata_routing`. Only during `fit` is `get_clone=True` used, which triggers `clone(self.regressor)`.\n\n5. **`poor_score = True` tag**: The `__sklearn_tags__` method sets `tags.regressor_tags.poor_score = True`.\n\n6. **Single fit guarantee**: The transformer's `fit` is called exactly once. The `check_inverse` block only calls `transform` and `inverse_transform` after fit, never triggering a second `fit`. This was explicitly tested in `test_transform_target_regressor_count_fit`.",
    "rubric": [
      "Explains that _fit_transformer subsamples using y.shape[0] // 10 (every 10th element) for its inverse check, which is different from FunctionTransformer's X.shape[0] // 100 (every 100th element)",
      "Identifies the double inverse check: FunctionTransformer.fit() runs its own _check_inverse_transform, and then _fit_transformer runs a second inverse check with different sampling",
      "Mentions that set_output(transform='default') is called on the FunctionTransformer to ensure numpy array output regardless of global configuration",
      "Explains the _get_regressor method's get_clone parameter: False returns regressor directly (used in __sklearn_tags__ and get_metadata_routing), True triggers clone (used only during fit)",
      "Notes the poor_score = True tag set in __sklearn_tags__",
      "Explains that the transformer's fit is called exactly once (the check_inverse block only calls transform/inverse_transform, not fit again)"
    ],
    "key_files": [
      "sklearn/compose/_target.py",
      "sklearn/preprocessing/_function_transformer.py",
      "sklearn/compose/tests/test_target.py"
    ],
    "source_doc": "[docstring: sklearn/compose/_target.py] sklearn.compose._target.TransformedTargetRegressor\nsklearn.compose._target.TransformedTargetRegressor:\n  Meta-estimator to regress on a transformed target.\n\n    Useful for applying a non-linear transformation to the target `y` in\n    regression problems. This transformation can be given as a Transformer\n    such as the :class:`~sklearn.preprocessing.QuantileTransformer` or as a\n    function and its inverse such as `np.log` and `np.exp`.\n\n    The computation during :meth:`fit` is::\n\n        regressor.fit(X, func(y))\n\n    or::\n\n        regressor.fit(X, transformer.transform(y))\n\n    The computation during :meth:`predict` is::\n\n        inverse_func(regressor.predict(X))\n\n    or::\n\n        transformer.inverse_transform(regressor.predict(X))\n\n    Read more in the :ref:`User Guide <transformed_target_regressor>`.\n\n    .. versionadded:: 0.20\n\n    Parameters\n    ----------\n    regressor : object, default=None\n        Regressor object such as derived from\n        :class:`~sklearn.base.RegressorMixin`. This regressor will\n        automatically be cloned each time prior to fitting. If `regressor is\n        None`, :class:`~sklearn.linear_model.LinearRegression` is created and used.\n\n    transformer : object, default=None\n        Estimator object such as derived from\n        :class:`~sklearn.base.TransformerMixin`. Cannot be set at the same time\n        as `func` and `inverse_func`. If `transformer is None` as well as\n        `func` and `inverse_func`, the transformer will be an identity\n        transformer. Note that the transformer will be cloned during fitting.\n        Also, the transformer is restricting `y` to be a numpy array.\n\n    func : function, default=None\n        Function to apply to `y` before passing to :meth:`fit`. Cannot be set\n        at the same time as `transformer`. If `func is None`, the function used will be\n        the identity function. If `func` is set, `inverse_func` also needs to be\n        provided. The function needs to return a 2-dimensional array.\n\n    inverse_func : function, default=None\n        Function to apply to the prediction of the regressor. Cannot be set at\n        the same time as `transformer`. The inverse function is used to return\n        predictions to the same space of the original training labels. If\n        `inverse_func` is set, `func` also needs to be provided. The inverse\n        function needs to return a 2-dimensional array.\n\n    check_inverse : bool, default=True\n        Whether to check that `transform` followed by `inverse_transform`\n        or `func` followed by `inverse_func` leads to the original targets.\n\n    Attributes\n    ----------\n    regressor_ : object\n        Fitted regressor.\n\n    transformer_ : object\n        Transformer used in :meth:`fit` and :meth:`predict`.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`. Only defined if the\n        underlying regressor exposes such an attribute when fit.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    sklearn.preprocessing.FunctionTransformer : Construct a transformer from an\n        arbitrary callable.\n\n    Notes\n    -----\n    Internally, the target `y` is always converted into a 2-dimensional array\n    to be used by scikit-learn transformers. At the time of prediction, the\n    output will be reshaped to a have the same number of dimensions as `y`.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.linear_model import LinearRegression\n    >>> from sklearn.compose import TransformedTargetRegressor\n    >>> tt = TransformedTargetRegressor(regressor=LinearRegression(),\n    ...                                 func=np.log, inverse_func=np.exp)\n    >>> X = np.arange(4).reshape(-1, 1)\n    >>> y = np.exp(2 * X).ravel()\n    >>> tt.fit(X, y)\n    TransformedTargetRegressor(...)\n    >>> tt.score(X, y)\n    1.0\n    >>> tt.regressor_.coef_\n    array([2.])\n\n    For a more detailed example use case refer to\n    :ref:`sphx_glr_auto_examples_compose_plot_transformed_target.py`.\n\nsklearn.compose._target.TransformedTargetRegressor._fit_transformer:\n  Check transformer and fit transformer.\n\n        Create the default transformer, fit it and make additional inverse\n        check on a subset (optional).\n\nsklearn.compose._target.TransformedTargetRegressor.fit:\n  Fit the model according to the given training data.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            Training vector, where `n_samples` is the number of samples and\n            `n_features` is the number of features.\n\n        y : array-like of shape (n_samples,)\n            Target values.\n\n        **fit_params : dict\n            - If `enable_metadata_routing=False` (default): Parameters directly passed\n              to the `fit` method of the underlying regressor.\n\n            - If `enable_metadata_routing=True`: Parameters safely routed to the `fit`\n              method of the underlying regressor.\n\n            .. versionchanged:: 1.6\n                See :ref:`Metadata Routing User Guide <metadata_routing>` for\n                more details.\n\n        Returns\n        -------\n        self : object\n            Fitted estimator.\n\nsklearn.compose._target.TransformedTargetRegressor.predict:\n  Predict using the base regressor, applying inverse.\n\n        The regressor is used to predict and the `inverse_func` or\n        `inverse_transform` is applied before returning the prediction.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n            Samples.\n\n        **predict_params : dict of str -> object\n            - If `enable_metadata_routing=False` (default): Parameters directly passed\n              to the `predict` method of the underlying regressor.\n\n            - If `enable_metadata_routing=True`: Parameters safely routed to the\n              `predict` method of the underlying regressor.\n\n            .. versionchanged:: 1.6\n                See :ref:`Metadata Routing User Guide <metadata_routing>`\n                for more details.\n\n        Returns\n        -------\n        y_hat : ndarray of shape (n_samples,)\n            Predicted values.\n\nsklearn.compose._target.TransformedTargetRegressor.n_features_in_:\n  Number of features seen during :term:`fit`.\n\nsklearn.compose._target.TransformedTargetRegressor.get_metadata_routing:\n  Get metadata routing of this object.\n\n        Please check :ref:`User Guide <metadata_routing>` on how the routing\n        mechanism works.\n\n        .. versionadded:: 1.6\n\n        Returns\n        -------\n        routing : MetadataRouter\n            A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating\n            routing information.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claims about specific implementation details (subsampling ratios, line numbers, internal method parameters like `get_clone`, `set_output` calls) are not verifiable from the provided documentation - they appear to be derived from source code inspection rather than the documentation chunk provided.",
      "The claim about 'redundant double-checking' when using func/inverse_func is an implementation detail not mentioned in the documentation and cannot be verified from it.",
      "The claim about `poor_score = True` tag and `__sklearn_tags__` is not mentioned in the documentation.",
      "The claim about `_get_regressor(get_clone=False)` method is not mentioned in the documentation.",
      "The reference to issue #11618 and `test_transform_target_regressor_count_fit` cannot be verified from the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code; some design rationale and issue references are doc-only and need removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_33",
    "repo": "scikit_learn",
    "question": "What is `make_column_selector`, how is it implemented internally, and how does it integrate with `ColumnTransformer` during fit and transform?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`make_column_selector` is implemented as a **class** (not a function) in `sklearn/compose/_column_transformer.py`. Despite its function-like naming convention (`make_*`), it's a class with `__init__` and `__call__` methods that makes instances callable.\n\n**Key implementation details:**\n\n1. **Validation mechanism**: The `__call__` method checks `hasattr(df, \"iloc\")` to verify the input is a pandas DataFrame. If this check fails, it raises a `ValueError` with message \"make_column_selector can only be applied to pandas dataframes\". This means it does NOT support Polars DataFrames or PyArrow tables (which don't have `iloc`).\n\n2. **Performance optimization**: The `__call__` method slices only the first row using `df.iloc[:1]` before calling `select_dtypes()`.\n\n3. **Pattern matching**: When `pattern` is not None, it uses `cols.str.contains(self.pattern, regex=True)` on the pandas Index, meaning it performs regex matching by default (the `regex=True` parameter is explicitly set).\n\n4. **Return type**: The method returns `cols.tolist()`, which produces a Python list of column name strings.\n\n5. **Integration with ColumnTransformer**: The callable is invoked during `_validate_column_callables(X)` which is called in `fit_transform()`. The callable is called with the full X: `columns = columns(X)`. The returned list of column names is then passed to `_get_column_indices(X, columns)` to convert them to integer indices, which are stored in `self._transformer_to_input_indices`. The callable is NOT re-invoked at transform time.\n\n6. **Pickling support**: The class stores only `pattern`, `dtype_include`, and `dtype_exclude` as instance attributes (set in `__init__`), making instances easily picklable (verified by test `test_make_column_selector_pickle`).\n\n7. **No inheritance**: `make_column_selector` doesn't inherit from any scikit-learn base class - it's a plain Python class.\n\n8. **Public API**: It's exported via `sklearn/compose/__init__.py` which imports it from `sklearn.compose._column_transformer` and includes it in `__all__`.",
    "rubric": [
      "Identifies that make_column_selector is a class (not a function), with __init__ and __call__ methods making instances callable",
      "Explains the DataFrame validation: checks hasattr(df, 'iloc') and raises ValueError if it fails, meaning it only supports pandas DataFrames",
      "Mentions the performance optimization of slicing only the first row via df.iloc[:1] before calling select_dtypes()",
      "Describes that pattern matching uses cols.str.contains(self.pattern, regex=True), i.e., regex matching is explicitly enabled",
      "Notes the return type is cols.tolist(), producing a plain Python list of column names",
      "Explains integration: the callable is invoked during _validate_column_callables(X) where columns = columns(X), and results are converted to integer indices via _get_column_indices and stored in _transformer_to_input_indices",
      "States that the callable is NOT re-invoked at transform time \u2014 only the resolved column indices are reused",
      "Mentions pickling support: only pattern, dtype_include, and dtype_exclude are stored as instance attributes, making it easily picklable"
    ],
    "key_files": [
      "sklearn/compose/_column_transformer.py",
      "sklearn/utils/_indexing.py",
      "sklearn/compose/__init__.py",
      "sklearn/compose/tests/test_column_transformer.py"
    ],
    "source_doc": "[docstring: sklearn/compose/_column_transformer.py] sklearn.compose._column_transformer.make_column_selector\nsklearn.compose._column_transformer.make_column_selector:\n  Create a callable to select columns to be used with\n    :class:`ColumnTransformer`.\n\n    :func:`make_column_selector` can select columns based on datatype or the\n    columns name with a regex. When using multiple selection criteria, **all**\n    criteria must match for a column to be selected.\n\n    For an example of how to use :func:`make_column_selector` within a\n    :class:`ColumnTransformer` to select columns based on data type (i.e.\n    `dtype`), refer to\n    :ref:`sphx_glr_auto_examples_compose_plot_column_transformer_mixed_types.py`.\n\n    Parameters\n    ----------\n    pattern : str, default=None\n        Name of columns containing this regex pattern will be included. If\n        None, column selection will not be selected based on pattern.\n\n    dtype_include : column dtype or list of column dtypes, default=None\n        A selection of dtypes to include. For more details, see\n        :meth:`pandas.DataFrame.select_dtypes`.\n\n    dtype_exclude : column dtype or list of column dtypes, default=None\n        A selection of dtypes to exclude. For more details, see\n        :meth:`pandas.DataFrame.select_dtypes`.\n\n    Returns\n    -------\n    selector : callable\n        Callable for column selection to be used by a\n        :class:`ColumnTransformer`.\n\n    See Also\n    --------\n    ColumnTransformer : Class that allows combining the\n        outputs of multiple transformer objects used on column subsets\n        of the data into a single feature space.\n\n    Examples\n    --------\n    >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder\n    >>> from sklearn.compose import make_column_transformer\n    >>> from sklearn.compose import make_column_selector\n    >>> import numpy as np\n    >>> import pandas as pd  # doctest: +SKIP\n    >>> X = pd.DataFrame({'city': ['London', 'London', 'Paris', 'Sallisaw'],\n    ...                   'rating': [5, 3, 4, 5]})  # doctest: +SKIP\n    >>> ct = make_column_transformer(\n    ...       (StandardScaler(),\n    ...        make_column_selector(dtype_include=np.number)),  # rating\n    ...       (OneHotEncoder(),\n    ...        make_column_selector(dtype_include=[object, \"string\"])))  # city\n    >>> ct.fit_transform(X)  # doctest: +SKIP\n    array([[ 0.90453403,  1.        ,  0.        ,  0.        ],\n           [-1.50755672,  1.        ,  0.        ,  0.        ],\n           [-0.30151134,  0.        ,  1.        ,  0.        ],\n           [ 0.90453403,  0.        ,  0.        ,  1.        ]])\n\nsklearn.compose._column_transformer.make_column_selector.__call__:\n  Callable for column selection to be used by a\n        :class:`ColumnTransformer`.\n\n        Parameters\n        ----------\n        df : dataframe of shape (n_features, n_samples)\n            DataFrame to select columns from.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation describes `__call__`'s parameter `df` as having shape `(n_features, n_samples)`, which is unusual (typically it would be `(n_samples, n_features)`). The generated answer doesn't address this discrepancy.",
      "Claims about specific line numbers (line 1490, line 997), `hasattr(df, 'iloc')` check, `df.iloc[:1]` optimization, `cols.str.contains(self.pattern, regex=True)`, `_validate_column_callables`, `_get_column_indices`, `_transformer_to_input_indices`, pickling test name, and lack of inheritance are all implementation details not present in the documentation - they are plausible but unverifiable from the given doc alone.",
      "The claim that it does NOT support Polars DataFrames or PyArrow tables is an implementation detail not mentioned in documentation, though the documentation only references pandas DataFrames."
    ],
    "strip_verify_leakage": 0.06,
    "strip_verify_summary": "Almost all claims are verifiable from the stripped code; only a design rationale comment about performance optimization and specific line numbers needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_35",
    "repo": "scikit_learn",
    "question": "Where is `EllipticEnvelope` defined, what is its full inheritance chain, and how do its key methods like `score_samples`, `score`, and `predict` work?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `EllipticEnvelope` class is defined in `sklearn/covariance/_elliptic_envelope.py`. Its class declaration is `class EllipticEnvelope(OutlierMixin, MinCovDet)`, inheriting from `OutlierMixin` (defined in `sklearn/base.py`) and `MinCovDet` (defined in `sklearn/covariance/_robust_covariance.py`). `MinCovDet` in turn inherits from `EmpiricalCovariance` (defined in `sklearn/covariance/_empirical_covariance.py`), which inherits from `BaseEstimator`.\n\nThe full inheritance chain is: EllipticEnvelope \u2192 OutlierMixin + MinCovDet \u2192 EmpiricalCovariance \u2192 BaseEstimator.\n\nKey implementation details:\n\n1. **`score_samples`** simply returns `-self.mahalanobis(X)`, where `mahalanobis` is inherited from `EmpiricalCovariance` and computes *squared* Mahalanobis distances using `pairwise_distances` with metric='mahalanobis' and the precision matrix as `VI`, then squares the result (`return np.reshape(dist, (len(X),)) ** 2`).\n\n2. **`offset_`** is calculated in `fit` as `np.percentile(-self.dist_, 100.0 * self.contamination)`, where `self.dist_` is the Mahalanobis distances of training samples computed by `MinCovDet.fit`.\n\n3. **`EllipticEnvelope.score`** overrides `EmpiricalCovariance.score` with a completely different implementation: it uses `accuracy_score(y, self.predict(X), sample_weight=sample_weight)` (returning classification accuracy) rather than the log-likelihood computation in the parent class.\n\n4. **`OutlierMixin`** provides `fit_predict` which calls `self.fit(X, **kwargs).predict(X)`, and sets the estimator_type tag to `\"outlier_detector\"` in `__sklearn_tags__`.\n\n5. **`MinCovDet.reweight_covariance`** uses a hardcoded `quantile_threshold = 0.025` with `chi2(n_features).isf(quantile_threshold)` from scipy.stats to determine which observations are inliers for reweighting.\n\n6. **`_parameter_constraints`** for `EllipticEnvelope` extends `MinCovDet._parameter_constraints` (which extends `EmpiricalCovariance._parameter_constraints`) by adding a `contamination` constraint as `Interval(Real, 0, 0.5, closed=\"right\")`.\n\n7. **`MinCovDet._nonrobust_covariance`** is defined as `staticmethod(empirical_covariance)` and used throughout the fit process.\n\n8. The `predict` method uses `np.full(values.shape[0], -1, dtype=int)` to create an array pre-filled with -1, then sets positions where `decision_function >= 0` to 1.",
    "rubric": [
      "EllipticEnvelope is defined in sklearn/covariance/_elliptic_envelope.py and inherits from OutlierMixin and MinCovDet",
      "MinCovDet inherits from EmpiricalCovariance, which inherits from BaseEstimator (full chain: EllipticEnvelope \u2192 OutlierMixin + MinCovDet \u2192 EmpiricalCovariance \u2192 BaseEstimator)",
      "score_samples returns -self.mahalanobis(X), where mahalanobis is inherited from EmpiricalCovariance",
      "offset_ is computed in fit as np.percentile(-self.dist_, 100.0 * self.contamination)",
      "score overrides the parent's log-likelihood-based score with accuracy_score(y, self.predict(X), sample_weight=sample_weight)",
      "predict creates an array pre-filled with -1 and sets positions where decision_function >= 0 to 1",
      "OutlierMixin (from sklearn/base.py) provides fit_predict and sets estimator_type tag to 'outlier_detector'",
      "MinCovDet.reweight_covariance uses a hardcoded quantile_threshold of 0.025 with chi2(n_features).isf()",
      "_parameter_constraints extends MinCovDet's constraints by adding contamination as Interval(Real, 0, 0.5, closed='right')"
    ],
    "key_files": [
      "sklearn/covariance/_elliptic_envelope.py",
      "sklearn/covariance/_robust_covariance.py",
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_elliptic_envelope.py] sklearn.covariance._elliptic_envelope.EllipticEnvelope\nsklearn.covariance._elliptic_envelope.EllipticEnvelope:\n  An object for detecting outliers in a Gaussian distributed dataset.\n\n    Read more in the :ref:`User Guide <outlier_detection>`.\n\n    Parameters\n    ----------\n    store_precision : bool, default=True\n        Specify if the estimated precision is stored.\n\n    assume_centered : bool, default=False\n        If True, the support of robust location and covariance estimates\n        is computed, and a covariance estimate is recomputed from it,\n        without centering the data.\n        Useful to work with data whose mean is significantly equal to\n        zero but is not exactly zero.\n        If False, the robust location and covariance are directly computed\n        with the FastMCD algorithm without additional treatment.\n\n    support_fraction : float, default=None\n        The proportion of points to be included in the support of the raw\n        MCD estimate. If None, the minimum value of support_fraction will\n        be used within the algorithm: `(n_samples + n_features + 1) / 2 * n_samples`.\n        Range is (0, 1).\n\n    contamination : float, default=0.1\n        The amount of contamination of the data set, i.e. the proportion\n        of outliers in the data set. Range is (0, 0.5].\n\n    random_state : int, RandomState instance or None, default=None\n        Determines the pseudo random number generator for shuffling\n        the data. Pass an int for reproducible results across multiple function\n        calls. See :term:`Glossary <random_state>`.\n\n    Attributes\n    ----------\n    location_ : ndarray of shape (n_features,)\n        Estimated robust location.\n\n    covariance_ : ndarray of shape (n_features, n_features)\n        Estimated robust covariance matrix.\n\n    precision_ : ndarray of shape (n_features, n_features)\n        Estimated pseudo inverse matrix.\n        (stored only if store_precision is True)\n\n    support_ : ndarray of shape (n_samples,)\n        A mask of the observations that have been used to compute the\n        robust estimates of location and shape.\n\n    offset_ : float\n        Offset used to define the decision function from the raw scores.\n        We have the relation: ``decision_function = score_samples - offset_``.\n        The offset depends on the contamination parameter and is defined in\n        such a way we obtain the expected number of outliers (samples with\n        decision function < 0) in training.\n\n        .. versionadded:: 0.20\n\n    raw_location_ : ndarray of shape (n_features,)\n        The raw robust estimated location before correction and re-weighting.\n\n    raw_covariance_ : ndarray of shape (n_features, n_features)\n        The raw robust estimated covariance before correction and re-weighting.\n\n    raw_support_ : ndarray of shape (n_samples,)\n        A mask of the observations that have been used to compute\n        the raw robust estimates of location and shape, before correction\n        and re-weighting.\n\n    dist_ : ndarray of shape (n_samples,)\n        Mahalanobis distances of the training set (on which :meth:`fit` is\n        called) observations.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    EmpiricalCovariance : Maximum likelihood covariance estimator.\n    GraphicalLasso : Sparse inverse covariance estimation\n        with an l1-penalized estimator.\n    LedoitWolf : LedoitWolf Estimator.\n    MinCovDet : Minimum Covariance Determinant\n        (robust estimator of covariance).\n    OAS : Oracle Approximating Shrinkage Estimator.\n    ShrunkCovariance : Covariance estimator with shrinkage.\n\n    Notes\n    -----\n    Outlier detection from covariance estimation may break or not\n    perform well in high-dimensional settings. In particular, one will\n    always take care to work with ``n_samples > n_features ** 2``.\n\n    References\n    ----------\n    .. [1] Rousseeuw, P.J., Van Driessen, K. \"A fast algorithm for the\n       minimum covariance determinant estimator\" Technometrics 41(3), 212\n       (1999)\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.covariance import EllipticEnvelope\n    >>> true_cov = np.array([[.8, .3],\n    ...                      [.3, .4]])\n    >>> X = np.random.RandomState(0).multivariate_normal(mean=[0, 0],\n    ...                                                  cov=true_cov,\n    ...                                                  size=500)\n    >>> cov = EllipticEnvelope(random_state=0).fit(X)\n    >>> # predict returns 1 for an inlier and -1 for an outlier\n    >>> cov.predict([[0, 0],\n    ...              [3, 3]])\n    array([ 1, -1])\n    >>> cov.covariance_\n    array([[0.8102, 0.2736],\n           [0.2736, 0.3330]])\n    >>> cov.location_\n    array([0.0769 , 0.0397])\n\nsklearn.covariance._elliptic_envelope.EllipticEnvelope.fit:\n  Fit the EllipticEnvelope model.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            Training data.\n\n        y : Ignored\n            Not used, present for API consistency by convention.\n\n        Returns\n        -------\n        self : object\n            Returns the instance itself.\n\nsklearn.covariance._elliptic_envelope.EllipticEnvelope.decision_function:\n  Compute the decision function of the given observations.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            The data matrix.\n\n        Returns\n        -------\n        decision : ndarray of shape (n_samples,)\n            Decision function of the samples.\n            It is equal to the shifted Mahalanobis distances.\n            The threshold for being an outlier is 0, which ensures a\n            compatibility with other outlier detection algorithms.\n\nsklearn.covariance._elliptic_envelope.EllipticEnvelope.score_samples:\n  Compute the negative Mahalanobis distances.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            The data matrix.\n\n        Returns\n        -------\n        negative_mahal_distances : array-like of shape (n_samples,)\n            Opposite of the Mahalanobis distances.\n\nsklearn.covariance._elliptic_envelope.EllipticEnvelope.predict:\n  Predict labels (1 inlier, -1 outlier) of X according to fitted model.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            The data matrix.\n\n        Returns\n        -------\n        is_inlier : ndarray of shape (n_samples,)\n            Returns -1 for anomalies/outliers and +1 for inliers.\n\nsklearn.covariance._elliptic_envelope.EllipticEnvelope.score:\n  Return the mean accuracy on the given test data and labels.\n\n        In multi-label classification, this is the subset accuracy\n        which is a harsh metric since you require for each sample that\n        each label set be correctly predicted.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            Test samples.\n\n        y : array-like of shape (n_samples,) or (n_samples, n_outputs)\n            True labels for X.\n\n        sample_weight : array-like of shape (n_samples,), default=None\n            Sample weights.\n\n        Returns\n        -------\n        score : float\n            Mean accuracy of self.predict(X) w.r.t. y.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states that `score_samples` returns `-self.mahalanobis(X)` and describes `mahalanobis` as computing squared Mahalanobis distances. The documentation says `score_samples` returns 'Opposite of the Mahalanobis distances' \u2014 the answer's claim that it returns negative *squared* Mahalanobis distances is a code-level detail not directly stated in the doc, but could be seen as slightly inconsistent with the doc's description of 'negative Mahalanobis distances' (not 'negative squared Mahalanobis distances'). However, this is a known subtlety in sklearn's implementation.",
      "The answer includes many specific code-level details (line numbers, inheritance chain, internal implementation details like `reweight_covariance`, `_parameter_constraints`, `_nonrobust_covariance`, `predict` implementation) that go beyond the provided documentation. These are plausible extrapolations from source code knowledge but cannot be verified from the given doc alone.",
      "The answer says `offset_` is calculated as `np.percentile(-self.dist_, 100.0 * self.contamination)` \u2014 the documentation only says the offset depends on the contamination parameter and is defined such that we obtain the expected number of outliers. The specific formula cannot be verified from the doc, but it is not contradicted either.",
      "The answer states that `EllipticEnvelope.score` uses `accuracy_score` rather than log-likelihood. The documentation does describe `score` as returning 'mean accuracy on the given test data and labels', which is consistent with using accuracy_score rather than log-likelihood."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments to remove specific line number references that may not match stripped versions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_32",
    "repo": "scikit_learn",
    "question": "How does `make_column_transformer` auto-generate transformer names and assemble everything into a fitted pipeline?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `make_column_transformer` function works through the following algorithm:\n\n1. **Input parsing**: It receives `*transformers` as variable positional args, each being a `(transformer, columns)` tuple.\n\n2. **Name generation via `_get_transformer_list`** (line 1342): This helper function unpacks the input tuples using `zip(*estimators)` to separate transformers and columns. It then delegates to `_name_estimators` from `sklearn.pipeline` to auto-generate names.\n\n3. **`_name_estimators` algorithm** (in `sklearn/pipeline.py` line 1376): Names are generated using `type(estimator).__name__.lower()` for estimator objects, or kept as-is if the estimator is already a string (like 'drop' or 'passthrough'). For duplicate names, the function counts occurrences via `defaultdict(int)`, removes names that appear only once from the counter, then iterates in **reversed** order appending \"-N\" suffixes (counting down). So if two `StandardScaler`s are passed, they become `'standardscaler-1'` and `'standardscaler-2'`.\n\n4. **ColumnTransformer construction** (line 1479): The function passes the transformer list to `ColumnTransformer.__init__` along with the keyword arguments (remainder, sparse_threshold, n_jobs, verbose, verbose_feature_names_out, force_int_remainder_cols). Notably, `transformer_weights` is NOT passed to the constructor.\n\n5. **During fit**: The `ColumnTransformer.fit` method calls `fit_transform`. In `fit_transform`, it validates transformers (checking they have `fit`/`fit_transform` and `transform` methods), resolves callable columns via `_validate_column_callables`, computes remainder columns via `_validate_remainder`, then dispatches parallel jobs using `joblib.Parallel`.\n\n6. **Passthrough handling**: During `_call_func_on_transformers`, if a transformer is 'passthrough', it's replaced with a `FunctionTransformer(accept_sparse=True, check_inverse=False, feature_names_out='one-to-one')` with its output set to match the ColumnTransformer's output configuration.\n\n7. **Column selection**: Each transformer's columns are indexed from the input using `_safe_indexing(X, columns, axis=1)`, so each transformer only sees its subset of columns.\n\n8. **Sparse output decision**: After fit_transform, if any output is sparse, density is computed as `nnz / total` and `sparse_output_` is set to `True` if density < `sparse_threshold` (default 0.3).\n\n9. **Output stacking**: `_hstack` either uses `scipy.sparse.hstack(...).tocsr()` for sparse output, or for dense output with DataFrame containers, it renames columns to temporary names (`tmp_col_name_{i}_{j}`) before using the container adapter's `hstack`. Otherwise, plain `np.hstack` is used.\n\n10. **Validation constraints**: The `_validate_names` method from `_BaseComposition` checks that names are unique, don't contain double underscores ('__'), and don't conflict with constructor argument names (like 'remainder', 'n_jobs', etc.).",
    "rubric": [
      "Explains that input tuples are unpacked via zip(*estimators) in _get_transformer_list to separate transformers and columns",
      "Describes _name_estimators generating names using type(estimator).__name__.lower() (or keeping strings like 'drop'/'passthrough' as-is)",
      "Explains duplicate name handling: counting occurrences via defaultdict, removing unique names from counter, then iterating in reversed order appending '-N' suffixes that count down",
      "Mentions that transformer_weights is intentionally not passed to ColumnTransformer",
      "Explains that fit delegates to fit_transform, which validates transformers, resolves callable columns, computes remainder columns, then dispatches parallel jobs",
      "Describes passthrough handling: replaced with FunctionTransformer(accept_sparse=True, check_inverse=False, feature_names_out='one-to-one') with output config set",
      "Explains column selection via _safe_indexing(X, columns, axis=1) so each transformer sees only its column subset",
      "Describes sparse output decision: computes density as nnz/total and sets sparse_output_ True if density < sparse_threshold (default 0.3)",
      "Explains _hstack behavior: sparse.hstack().tocsr() for sparse output, or temporary column renaming + adapter.hstack for DataFrames, or np.hstack for plain dense arrays"
    ],
    "key_files": [
      "sklearn/compose/_column_transformer.py",
      "sklearn/pipeline.py",
      "sklearn/utils/metaestimators.py"
    ],
    "source_doc": "[docstring: sklearn/compose/_column_transformer.py] sklearn.compose._column_transformer.make_column_transformer\nConstruct a ColumnTransformer from the given transformers.\n\n    This is a shorthand for the ColumnTransformer constructor; it does not\n    require, and does not permit, naming the transformers. Instead, they will\n    be given names automatically based on their types. It also does not allow\n    weighting with ``transformer_weights``.\n\n    Read more in the :ref:`User Guide <make_column_transformer>`.\n\n    Parameters\n    ----------\n    *transformers : tuples\n        Tuples of the form (transformer, columns) specifying the\n        transformer objects to be applied to subsets of the data.\n\n        transformer : {'drop', 'passthrough'} or estimator\n            Estimator must support :term:`fit` and :term:`transform`.\n            Special-cased strings 'drop' and 'passthrough' are accepted as\n            well, to indicate to drop the columns or to pass them through\n            untransformed, respectively.\n        columns : str,  array-like of str, int, array-like of int, slice,                 array-like of bool or callable\n            Indexes the data on its second axis. Integers are interpreted as\n            positional columns, while strings can reference DataFrame columns\n            by name. A scalar string or int should be used where\n            ``transformer`` expects X to be a 1d array-like (vector),\n            otherwise a 2d array will be passed to the transformer.\n            A callable is passed the input data `X` and can return any of the\n            above. To select multiple columns by name or dtype, you can use\n            :obj:`make_column_selector`.\n\n    remainder : {'drop', 'passthrough'} or estimator, default='drop'\n        By default, only the specified columns in `transformers` are\n        transformed and combined in the output, and the non-specified\n        columns are dropped. (default of ``'drop'``).\n        By specifying ``remainder='passthrough'``, all remaining columns that\n        were not specified in `transformers` will be automatically passed\n        through. This subset of columns is concatenated with the output of\n        the transformers.\n        By setting ``remainder`` to be an estimator, the remaining\n        non-specified columns will use the ``remainder`` estimator. The\n        estimator must support :term:`fit` and :term:`transform`.\n\n    sparse_threshold : float, default=0.3\n        If the transformed output consists of a mix of sparse and dense data,\n        it will be stacked as a sparse matrix if the density is lower than this\n        value. Use ``sparse_threshold=0`` to always return dense.\n        When the transformed output consists of all sparse or all dense data,\n        the stacked result will be sparse or dense, respectively, and this\n        keyword will be ignored.\n\n    n_jobs : int, default=None\n        Number of jobs to run in parallel.\n        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.\n        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`\n        for more details.\n\n    verbose : bool, default=False\n        If True, the time elapsed while fitting each transformer will be\n        printed as it is completed.\n\n    verbose_feature_names_out : bool, default=True\n        If True, :meth:`ColumnTransformer.get_feature_names_out` will prefix\n        all feature names with the name of the transformer that generated that\n        feature.\n        If False, :meth:`ColumnTransformer.get_feature_names_out` will not\n        prefix any feature names and will error if feature names are not\n        unique.\n\n        .. versionadded:: 1.0\n\n    force_int_remainder_cols : bool, default=True\n        This parameter has no effect.\n\n        .. note::\n            If you do not access the list of columns for the remainder columns\n            in the :attr:`ColumnTransformer.transformers_` fitted attribute,\n            you do not need to set this parameter.\n\n        .. versionadded:: 1.5\n\n        .. versionchanged:: 1.7\n           The default value for `force_int_remainder_cols` will change from\n           `True` to `False` in version 1.7.\n\n        .. deprecated:: 1.7\n           `force_int_remainder_cols` is deprecated and will be removed in version 1.9.\n\n    Returns\n    -------\n    ct : ColumnTransformer\n        Returns a :class:`ColumnTransformer` object.\n\n    See Also\n    --------\n    ColumnTransformer : Class that allows combining the\n        outputs of multiple transformer objects used on column subsets\n        of the data into a single feature space.\n\n    Examples\n    --------\n    >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder\n    >>> from sklearn.compose import make_column_transformer\n    >>> make_column_transformer(\n    ...     (StandardScaler(), ['numerical_column']),\n    ...     (OneHotEncoder(), ['categorical_column']))\n    ColumnTransformer(transformers=[('standardscaler', StandardScaler(...),\n                                     ['numerical_column']),\n                                    ('onehotencoder', OneHotEncoder(...),\n                                     ['categorical_column'])])",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims about internal implementation details (line numbers, specific helper functions, internal algorithms) go beyond what the documentation covers and cannot be verified from the provided docstring alone, but none contradict the documentation.",
      "The claim about `_name_estimators` using `type(estimator).__name__.lower()` is plausible and consistent with the example output showing 'standardscaler' and 'onehotencoder', but cannot be verified from the docstring alone.",
      "The description of sparse_threshold behavior is consistent with the documentation.",
      "The statement that transformer_weights is not passed is consistent with the doc stating 'it does not allow weighting with transformer_weights'.",
      "The description of remainder behavior is consistent with the documentation.",
      "The claim about force_int_remainder_cols having 'no effect' is consistent with the documentation."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in rationale comments for why transformer_weights is not passed and why fit calls fit_transform; all other claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_36",
    "repo": "scikit_learn",
    "question": "How does `empirical_covariance` compute the covariance matrix, and how does it integrate with the `EmpiricalCovariance` class and discriminant analysis?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `empirical_covariance` function computes a covariance estimator with the following implementation details:\n\n**Implementation Steps:**\n1. Input is validated via `check_array(X, ensure_2d=False, ensure_all_finite=False)` \u2014 notably it disables 2D enforcement and allows non-finite values.\n2. If the input is 1D, it is reshaped to `(1, -1)`.\n3. If there is only one sample (`X.shape[0] == 1`), a warning is issued: \"Only one sample available. You may want to reshape your data array\".\n4. When `assume_centered=True`, it computes `np.dot(X.T, X) / X.shape[0]` (divides by n_samples, not n_samples-1).\n5. When `assume_centered=False`, it uses `np.cov(X.T, bias=1)` where `bias=1` normalizes by N rather than N-1.\n6. If the result is 0-dimensional (scalar), it wraps it as `np.array([[covariance]])`.\n\n**Parameter Validation:**\nThe function is decorated with `@validate_params` which validates that `X` is \"array-like\" and `assume_centered` is \"boolean\". The decorator (from `sklearn.utils._param_validation`) checks `get_config()[\"skip_parameter_validation\"]` and if True, skips validation entirely. It sets `prefer_skip_nested_validation=True` which causes nested calls to have `skip_parameter_validation` set to True via `config_context`.\n\n**Integration with EmpiricalCovariance class:**\nThe `EmpiricalCovariance.fit()` method calls `empirical_covariance()` and then `_set_covariance()`, which stores the covariance matrix and computes the precision matrix using `linalg.pinvh(covariance, check_finite=False)` if `store_precision=True`.\n\n**Class hierarchy:**\n`EmpiricalCovariance` extends `BaseEstimator`.\n\n**Usage in discriminant analysis:**\nThe `_cov()` helper in `sklearn/discriminant_analysis.py` calls `empirical_covariance(X)` when shrinkage is None or 'empirical', and `shrunk_covariance(empirical_covariance(X), shrinkage)` when shrinkage is a float.\n\n**Log-likelihood computation:**\nThe related `log_likelihood(emp_cov, precision)` function computes: `(-sum(emp_cov * precision) + fast_logdet(precision) - p*log(2\u03c0)) / 2`, where `fast_logdet` uses `xp.linalg.slogdet` and returns `-xp.inf` if the sign is not positive.",
    "rubric": [
      "Explains that input is validated with check_array allowing non-finite values and not enforcing 2D",
      "Mentions 1D inputs are reshaped to (1, -1) and a warning is issued for single-sample inputs",
      "States that when assume_centered=True, covariance is computed as dot(X.T, X) / n_samples (biased, divides by N not N-1)",
      "States that when assume_centered=False, np.cov(X.T, bias=1) is used (normalizing by N)",
      "Mentions 0-dimensional results are wrapped in a 2D array",
      "Explains that EmpiricalCovariance.fit() calls empirical_covariance() then _set_covariance(), which stores covariance and computes precision via linalg.pinvh",
      "Describes how discriminant_analysis._cov() uses empirical_covariance directly or wraps it with shrunk_covariance depending on the shrinkage parameter",
      "Mentions the @validate_params decorator validates X as array-like and assume_centered as boolean"
    ],
    "key_files": [
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/utils/extmath.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/discriminant_analysis.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_empirical_covariance.py] sklearn.covariance._empirical_covariance.empirical_covariance\nCompute the Maximum likelihood covariance estimator.\n\n    Parameters\n    ----------\n    X : ndarray of shape (n_samples, n_features)\n        Data from which to compute the covariance estimate.\n\n    assume_centered : bool, default=False\n        If `True`, data will not be centered before computation.\n        Useful when working with data whose mean is almost, but not exactly\n        zero.\n        If `False`, data will be centered before computation.\n\n    Returns\n    -------\n    covariance : ndarray of shape (n_features, n_features)\n        Empirical covariance (Maximum Likelihood Estimator).\n\n    Examples\n    --------\n    >>> from sklearn.covariance import empirical_covariance\n    >>> X = [[1,1,1],[1,1,1],[1,1,1],\n    ...      [0,0,0],[0,0,0],[0,0,0]]\n    >>> empirical_covariance(X)\n    array([[0.25, 0.25, 0.25],\n           [0.25, 0.25, 0.25],\n           [0.25, 0.25, 0.25]])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed inheritance hierarchy claim not present in provided files, and rewrote documentation-level interpretations like 'MLE' and 'pseudo-inverse for Hermitian matrices'.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_37",
    "repo": "scikit_learn",
    "question": "What is the class hierarchy of scikit-learn's covariance estimators and how does EmpiricalCovariance implement its core methods?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `EmpiricalCovariance` class in `sklearn/covariance/_empirical_covariance.py` is the base class for the covariance estimation hierarchy in scikit-learn's covariance module.\n\n**Class hierarchy:** `EmpiricalCovariance` extends `sklearn.base.BaseEstimator` directly. It is the parent class for:\n- `ShrunkCovariance` (in `_shrunk_covariance.py`)\n- `LedoitWolf` (in `_shrunk_covariance.py`)\n- `OAS` (in `_shrunk_covariance.py`)\n- `MinCovDet` (in `_robust_covariance.py`)\n- `BaseGraphicalLasso` (in `_graph_lasso.py`), which is in turn parent of `GraphicalLasso` and `GraphicalLassoCV`\n- `EllipticEnvelope` inherits from both `OutlierMixin` and `MinCovDet`, making it a grandchild of `EmpiricalCovariance`\n\n**Key implementation details:**\n\n1. **Precision computation uses pseudo-inverse:** `_set_covariance()` computes precision using `scipy.linalg.pinvh(covariance, check_finite=False)` \u2014 a Hermitian pseudo-inverse, not a regular matrix inverse. The same `pinvh` call is used in `get_precision()` when `store_precision=False`.\n\n2. **The `empirical_covariance` function** uses two different computation paths: when `assume_centered=True`, it computes `np.dot(X.T, X) / X.shape[0]`; when `assume_centered=False`, it uses `np.cov(X.T, bias=1)`. It also handles 1D input by reshaping it to `(1, -1)` and issues a warning for single-sample inputs. If the result is 0-dimensional (scalar), it wraps it as `np.array([[covariance]])`.\n\n3. **The `log_likelihood` function** computes: `(-sum(emp_cov * precision) + fast_logdet(precision) - p * log(2*pi)) / 2.0`. The `fast_logdet` utility (from `sklearn.utils.extmath`) uses `numpy.linalg.slogdet` and returns `-inf` if the sign of the determinant is not strictly positive.\n\n4. **The `score()` method** centers test data by subtracting `self.location_` before computing the empirical covariance of the test set with `assume_centered=True`, then calls `log_likelihood` with that test covariance and the model's precision matrix.\n\n5. **The `mahalanobis()` method** uses `sklearn.metrics.pairwise.pairwise_distances` with `metric='mahalanobis'` and passes the precision matrix as the `VI` parameter. It wraps the computation in `config_context(assume_finite=True)`. The result is squared (distances are computed, then squared via `** 2`).\n\n6. **The `error_norm()` method** for `norm='spectral'` computes `np.amax(linalg.svdvals(np.dot(error.T, error)))` \u2014 the maximum singular value of error^T @ error. For `norm='frobenius'`, it uses `np.sum(error**2)`. Any other norm raises `NotImplementedError`.\n\n7. **Metadata routing:** The class has `__metadata_request__score = {\"X_test\": metadata_routing.UNUSED}`, indicating that the `score` method's first parameter is named `X_test` (not the standard `X`), marked as UNUSED for metadata routing.\n\n8. **`BaseGraphicalLasso`** inherits `EmpiricalCovariance._parameter_constraints` but pops `'store_precision'` from its own constraints copy.\n\n9. **`ShrunkCovariance.fit()`** does NOT call the parent `fit()`, instead duplicating the location estimation logic and calling `_set_covariance` directly after applying shrinkage.",
    "rubric": [
      "Identifies EmpiricalCovariance as the base class extending BaseEstimator, and lists its subclasses: ShrunkCovariance, LedoitWolf, OAS, MinCovDet, and BaseGraphicalLasso (parent of GraphicalLasso and GraphicalLassoCV), plus EllipticEnvelope inheriting from both OutlierMixin and MinCovDet",
      "Explains that _set_covariance and get_precision use scipy.linalg.pinvh (pseudo-inverse) rather than a regular matrix inverse",
      "Describes the empirical_covariance function's two computation paths: np.dot(X.T, X)/X.shape[0] when assume_centered=True vs np.cov(X.T, bias=1) when False, plus handling of 1D and single-sample edge cases",
      "Explains the log_likelihood formula: (-sum(emp_cov * precision) + fast_logdet(precision) - p*log(2*pi)) / 2.0",
      "Describes how score() centers test data by subtracting self.location_ then computes empirical covariance with assume_centered=True before calling log_likelihood",
      "Explains that mahalanobis() uses pairwise_distances with metric='mahalanobis' and VI=precision, wrapped in config_context(assume_finite=True), and returns squared distances",
      "Describes error_norm's two norms: frobenius uses np.sum(error**2) and spectral uses np.amax(linalg.svdvals(error.T @ error))",
      "Notes that BaseGraphicalLasso inherits EmpiricalCovariance._parameter_constraints but pops 'store_precision' from its constraints",
      "Notes that ShrunkCovariance.fit() does not call the parent fit(), instead duplicating the location estimation logic to avoid a potential extra matrix inversion"
    ],
    "key_files": [
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/covariance/__init__.py",
      "sklearn/covariance/_shrunk_covariance.py",
      "sklearn/covariance/_graph_lasso.py",
      "sklearn/covariance/_robust_covariance.py",
      "sklearn/covariance/_elliptic_envelope.py",
      "sklearn/utils/extmath.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_empirical_covariance.py] sklearn.covariance._empirical_covariance.EmpiricalCovariance\nsklearn.covariance._empirical_covariance.EmpiricalCovariance:\n  Maximum likelihood covariance estimator.\n\n    Read more in the :ref:`User Guide <covariance>`.\n\n    Parameters\n    ----------\n    store_precision : bool, default=True\n        Specifies if the estimated precision is stored.\n\n    assume_centered : bool, default=False\n        If True, data are not centered before computation.\n        Useful when working with data whose mean is almost, but not exactly\n        zero.\n        If False (default), data are centered before computation.\n\n    Attributes\n    ----------\n    location_ : ndarray of shape (n_features,)\n        Estimated location, i.e. the estimated mean.\n\n    covariance_ : ndarray of shape (n_features, n_features)\n        Estimated covariance matrix.\n\n    precision_ : ndarray of shape (n_features, n_features)\n        Estimated pseudo-inverse matrix.\n        (stored only if store_precision is True)\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    EllipticEnvelope : An object for detecting outliers in\n        a Gaussian distributed dataset.\n    GraphicalLasso : Sparse inverse covariance estimation\n        with an l1-penalized estimator.\n    LedoitWolf : LedoitWolf Estimator.\n    MinCovDet : Minimum Covariance Determinant\n        (robust estimator of covariance).\n    OAS : Oracle Approximating Shrinkage Estimator.\n    ShrunkCovariance : Covariance estimator with shrinkage.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.covariance import EmpiricalCovariance\n    >>> from sklearn.datasets import make_gaussian_quantiles\n    >>> real_cov = np.array([[.8, .3],\n    ...                      [.3, .4]])\n    >>> rng = np.random.RandomState(0)\n    >>> X = rng.multivariate_normal(mean=[0, 0],\n    ...                             cov=real_cov,\n    ...                             size=500)\n    >>> cov = EmpiricalCovariance().fit(X)\n    >>> cov.covariance_\n    array([[0.7569, 0.2818],\n           [0.2818, 0.3928]])\n    >>> cov.location_\n    array([0.0622, 0.0193])\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance._set_covariance:\n  Saves the covariance and precision estimates\n\n        Storage is done accordingly to `self.store_precision`.\n        Precision stored only if invertible.\n\n        Parameters\n        ----------\n        covariance : array-like of shape (n_features, n_features)\n            Estimated covariance matrix to be stored, and from which precision\n            is computed.\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance.get_precision:\n  Getter for the precision matrix.\n\n        Returns\n        -------\n        precision_ : array-like of shape (n_features, n_features)\n            The precision matrix associated to the current covariance object.\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance.fit:\n  Fit the maximum likelihood covariance estimator to X.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n          Training data, where `n_samples` is the number of samples and\n          `n_features` is the number of features.\n\n        y : Ignored\n            Not used, present for API consistency by convention.\n\n        Returns\n        -------\n        self : object\n            Returns the instance itself.\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance.score:\n  Compute the log-likelihood of `X_test` under the estimated Gaussian model.\n\n        The Gaussian model is defined by its mean and covariance matrix which are\n        represented respectively by `self.location_` and `self.covariance_`.\n\n        Parameters\n        ----------\n        X_test : array-like of shape (n_samples, n_features)\n            Test data of which we compute the likelihood, where `n_samples` is\n            the number of samples and `n_features` is the number of features.\n            `X_test` is assumed to be drawn from the same distribution than\n            the data used in fit (including centering).\n\n        y : Ignored\n            Not used, present for API consistency by convention.\n\n        Returns\n        -------\n        res : float\n            The log-likelihood of `X_test` with `self.location_` and `self.covariance_`\n            as estimators of the Gaussian model mean and covariance matrix respectively.\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance.error_norm:\n  Compute the Mean Squared Error between two covariance estimators.\n\n        Parameters\n        ----------\n        comp_cov : array-like of shape (n_features, n_features)\n            The covariance to compare with.\n\n        norm : {\"frobenius\", \"spectral\"}, default=\"frobenius\"\n            The type of norm used to compute the error. Available error types:\n            - 'frobenius' (default): sqrt(tr(A^t.A))\n            - 'spectral': sqrt(max(eigenvalues(A^t.A))\n            where A is the error ``(comp_cov - self.covariance_)``.\n\n        scaling : bool, default=True\n            If True (default), the squared error norm is divided by n_features.\n            If False, the squared error norm is not rescaled.\n\n        squared : bool, default=True\n            Whether to compute the squared error norm or the error norm.\n            If True (default), the squared error norm is returned.\n            If False, the error norm is returned.\n\n        Returns\n        -------\n        result : float\n            The Mean Squared Error (in the sense of the Frobenius norm) between\n            `self` and `comp_cov` covariance estimators.\n\nsklearn.covariance._empirical_covariance.EmpiricalCovariance.mahalanobis:\n  Compute the squared Mahalanobis distances of given observations.\n\n        For a detailed example of how outliers affects the Mahalanobis distance,\n        see :ref:`sphx_glr_auto_examples_covariance_plot_mahalanobis_distances.py`.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            The observations, the Mahalanobis distances of the which we\n            compute. Observations are assumed to be drawn from the same\n            distribution than the data used in fit.\n\n        Returns\n        -------\n        dist : ndarray of shape (n_samples,)\n            Squared Mahalanobis distances of the observations.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The class hierarchy details (ShrunkCovariance, LedoitWolf, OAS, MinCovDet, BaseGraphicalLasso, EllipticEnvelope inheritance) are not in the provided documentation. These are plausible code-level details but cannot be verified from the given doc chunk.",
      "The claim that EmpiricalCovariance extends 'sklearn.base.BaseEstimator directly' is not stated in the documentation. Plausible but unverifiable from the provided text.",
      "Implementation details about pinvh, empirical_covariance function internals, log_likelihood function, score() internals, mahalanobis() using pairwise_distances, error_norm() SVD implementation, metadata routing, BaseGraphicalLasso constraints, and ShrunkCovariance.fit() behavior are all code-level claims not verifiable from the provided documentation.",
      "The documentation says precision_ is the 'Estimated pseudo-inverse matrix', which is consistent with the answer's claim about using pinvh.",
      "The documentation says _set_covariance stores precision 'only if invertible', while the answer says it uses pinvh (pseudo-inverse) which handles singular matrices \u2014 these are not directly contradictory but the nuance differs slightly.",
      "The answer says error_norm returns 'The Mean Squared Error (in the sense of the Frobenius norm)' which matches the documentation's return description, but the answer's detailed implementation claims about svdvals and np.sum(error**2) are unverifiable.",
      "The answer's claim about the score method naming ('X_test should have been called X') is unverifiable from the docs, though the docs do use X_test as the parameter name for score()."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-only rationales and a stripped comment needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_39",
    "repo": "scikit_learn",
    "question": "Where does data flow through the GraphicalLasso fitting process, from input validation to the inner solver and convergence check?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `GraphicalLasso.fit()` method's data flow traces through the following call chain:\n\n1. **Input validation**: `validate_data(self, X, ensure_min_features=2, ensure_min_samples=2)` \u2014 requires at least 2 features and 2 samples.\n\n2. **Covariance computation branch**: If `self.covariance == 'precomputed'`, it copies X directly as `emp_cov` and sets `location_` to zeros. Otherwise, it calls `empirical_covariance(X, assume_centered=self.assume_centered)` from `_empirical_covariance.py`.\n\n3. **Core solver**: `_graphical_lasso()` is called with `cov_init=None` (always None from `GraphicalLasso.fit`, though `graphical_lasso_path` can pass a warm-start value).\n\n4. **Inside `_graphical_lasso`**: \n   - **Alpha=0 early return**: When alpha==0, it computes `linalg.inv(emp_cov)` directly and returns iteration count 0.\n   - **Tikhonov-like initialization**: Off-diagonal coefficients of the covariance starting point are scaled by 0.95 (`covariance_ *= 0.95`), while diagonal entries are restored from `emp_cov`. Initial precision is computed via `linalg.pinvh(covariance_)`.\n   - **CD mode**: Uses `cd_fast.enet_coordinate_descent_gram` with `beta=0`, `random=False`, `positive=False`, `do_screening=True`. The inner max_iter is `max(20, max_iter)` \u2014 coupled with outer solver's max_iter with a minimum floor of 20. Initial coefficients are warm-started as `-(precision_[indices != idx, idx] / (precision_[idx, idx] + 1000 * eps))`.\n   - **LARS mode**: Uses `lars_path_gram` with `alpha_min=alpha / (n_features - 1)` and `method='lars'`, `return_path=False`.\n   - **Convergence**: Checked via `np.abs(d_gap) < tol` where `d_gap` is computed by `_dual_gap()`. A `ConvergenceWarning` is issued if max_iter is exhausted.\n   - **Error handling**: `np.errstate` differs by mode \u2014 CD uses `over='raise', invalid='ignore'`, while LARS uses `invalid='raise'`.\n\n5. **Class hierarchy**: `GraphicalLasso` \u2192 `BaseGraphicalLasso` \u2192 `EmpiricalCovariance` \u2192 `BaseEstimator`. `BaseGraphicalLasso` explicitly pops `store_precision` from `_parameter_constraints`, meaning GraphicalLasso does not expose the `store_precision` parameter despite inheriting from `EmpiricalCovariance`.\n\n6. **GraphicalLassoCV**: During cross-validation, `graphical_lasso_path` is called with `max_iter=int(0.1 * self.max_iter)` \u2014 only 10% of the configured max_iter is used during CV folds.\n\n7. **Objective function** (`_objective`): Computes `-2.0 * log_likelihood(mle, precision_) + p * log(2\u03c0) + alpha * (sum(|precision_|) - sum(|diag(precision_)|))`. The penalty explicitly excludes diagonal elements.\n\n8. **`graphical_lasso` function**: The public functional API internally creates a `GraphicalLasso` instance with `covariance='precomputed'` and `assume_centered=True`, then calls `.fit(emp_cov)`.\n\n9. **Return values**: `_graphical_lasso` returns a 4-tuple: `(covariance_, precision_, costs, i + 1)` where costs is a list of (objective, dual_gap) pairs, and `i + 1` is the iteration count.\n\n10. **`log_likelihood`** uses `fast_logdet` from `sklearn.utils.extmath`, which computes log-determinant via `np.linalg.slogdet` and returns `-inf` if the sign is not positive.",
    "rubric": [
      "Mentions validate_data requiring at least 2 features and 2 samples",
      "Explains the covariance=='precomputed' branch (copies X directly as emp_cov, sets location_ to zeros) vs computing empirical_covariance",
      "Notes that _graphical_lasso is called with cov_init=None from GraphicalLasso.fit",
      "Describes the alpha==0 early return path using linalg.inv with 0 iterations",
      "Explains the 0.95 off-diagonal scaling initialization and pinvh for initial precision",
      "Describes CD mode using enet_coordinate_descent_gram with beta=0, inner max_iter of max(20, max_iter), and warm-started coefficients from precision matrix",
      "Describes LARS mode using lars_path_gram with alpha_min=alpha/(n_features-1) and method='lars'",
      "Explains convergence is checked via np.abs(d_gap) < tol with ConvergenceWarning on exhaustion",
      "Mentions that BaseGraphicalLasso pops store_precision from _parameter_constraints",
      "Notes GraphicalLassoCV uses int(0.1 * self.max_iter) during cross-validation folds",
      "Explains the _objective function penalizes off-diagonal precision elements (excludes diagonal)",
      "Notes the public graphical_lasso function creates a GraphicalLasso with covariance='precomputed' and assume_centered=True",
      "Mentions _graphical_lasso returns a 4-tuple of (covariance_, precision_, costs, i+1)"
    ],
    "key_files": [
      "sklearn/covariance/_graph_lasso.py",
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/utils/extmath.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_graph_lasso.py] sklearn.covariance._graph_lasso.GraphicalLasso\nsklearn.covariance._graph_lasso.GraphicalLasso:\n  Sparse inverse covariance estimation with an l1-penalized estimator.\n\n    For a usage example see\n    :ref:`sphx_glr_auto_examples_applications_plot_stock_market.py`.\n\n    Read more in the :ref:`User Guide <sparse_inverse_covariance>`.\n\n    .. versionchanged:: v0.20\n        GraphLasso has been renamed to GraphicalLasso\n\n    Parameters\n    ----------\n    alpha : float, default=0.01\n        The regularization parameter: the higher alpha, the more\n        regularization, the sparser the inverse covariance.\n        Range is (0, inf].\n\n    mode : {'cd', 'lars'}, default='cd'\n        The Lasso solver to use: coordinate descent or LARS. Use LARS for\n        very sparse underlying graphs, where p > n. Elsewhere prefer cd\n        which is more numerically stable.\n\n    covariance : \"precomputed\", default=None\n        If covariance is \"precomputed\", the input data in `fit` is assumed\n        to be the covariance matrix. If `None`, the empirical covariance\n        is estimated from the data `X`.\n\n        .. versionadded:: 1.3\n\n    tol : float, default=1e-4\n        The tolerance to declare convergence: if the dual gap goes below\n        this value, iterations are stopped. Range is (0, inf].\n\n    enet_tol : float, default=1e-4\n        The tolerance for the elastic net solver used to calculate the descent\n        direction. This parameter controls the accuracy of the search direction\n        for a given column update, not of the overall parameter estimate. Only\n        used for mode='cd'. Range is (0, inf].\n\n    max_iter : int, default=100\n        The maximum number of iterations.\n\n    verbose : bool, default=False\n        If verbose is True, the objective function and dual gap are\n        plotted at each iteration.\n\n    eps : float, default=eps\n        The machine-precision regularization in the computation of the\n        Cholesky diagonal factors. Increase this for very ill-conditioned\n        systems. Default is `np.finfo(np.float64).eps`.\n\n        .. versionadded:: 1.3\n\n    assume_centered : bool, default=False\n        If True, data are not centered before computation.\n        Useful when working with data whose mean is almost, but not exactly\n        zero.\n        If False, data are centered before computation.\n\n    Attributes\n    ----------\n    location_ : ndarray of shape (n_features,)\n        Estimated location, i.e. the estimated mean.\n\n    covariance_ : ndarray of shape (n_features, n_features)\n        Estimated covariance matrix\n\n    precision_ : ndarray of shape (n_features, n_features)\n        Estimated pseudo inverse matrix.\n\n    n_iter_ : int\n        Number of iterations run.\n\n    costs_ : list of (objective, dual_gap) pairs\n        The list of values of the objective function and the dual gap at\n        each iteration. Returned only if return_costs is True.\n\n        .. versionadded:: 1.3\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    graphical_lasso : L1-penalized covariance estimator.\n    GraphicalLassoCV : Sparse inverse covariance with\n        cross-validated choice of the l1 penalty.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.covariance import GraphicalLasso\n    >>> true_cov = np.array([[0.8, 0.0, 0.2, 0.0],\n    ...                      [0.0, 0.4, 0.0, 0.0],\n    ...                      [0.2, 0.0, 0.3, 0.1],\n    ...                      [0.0, 0.0, 0.1, 0.7]])\n    >>> np.random.seed(0)\n    >>> X = np.random.multivariate_normal(mean=[0, 0, 0, 0],\n    ...                                   cov=true_cov,\n    ...                                   size=200)\n    >>> cov = GraphicalLasso().fit(X)\n    >>> np.around(cov.covariance_, decimals=3)\n    array([[0.816, 0.049, 0.218, 0.019],\n           [0.049, 0.364, 0.017, 0.034],\n           [0.218, 0.017, 0.322, 0.093],\n           [0.019, 0.034, 0.093, 0.69 ]])\n    >>> np.around(cov.location_, decimals=3)\n    array([0.073, 0.04 , 0.038, 0.143])\n\nsklearn.covariance._graph_lasso.GraphicalLasso.fit:\n  Fit the GraphicalLasso model to X.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            Data from which to compute the covariance estimate.\n\n        y : Ignored\n            Not used, present for API consistency by convention.\n\n        Returns\n        -------\n        self : object\n            Returns the instance itself.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims in the answer go far beyond what is stated in the provided documentation, describing internal implementation details (e.g., 0.95 scaling factor, cd_fast.enet_coordinate_descent_gram parameters, dual gap computation, error handling specifics, GraphicalLassoCV's 10% max_iter behavior, etc.) that cannot be verified from the given docstring alone.",
      "However, none of these claims directly contradict the provided documentation.",
      "The claims that can be checked against the documentation are consistent: alpha default=0.01, mode options 'cd'/'lars', tol default=1e-4, enet_tol only used for mode='cd', max_iter default=100, covariance='precomputed' behavior, assume_centered behavior, returned attributes (covariance_, precision_, location_, n_iter_, costs_), and the fit method signature."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only minor interpretive parentheticals and a line number reference needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_38",
    "repo": "scikit_learn",
    "question": "Why does the graphical lasso implementation use so many numerical safeguards like off-diagonal scaling, different errstate configs per mode, and the 1000*eps denominator trick?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `graphical_lasso` function and its internal implementation `_graphical_lasso` contain several key design patterns:\n\n1. **Off-diagonal scaling (line 102):** The initial covariance estimate is scaled by 0.95 (`covariance_ *= 0.95`) while restoring the diagonal to the original empirical covariance values.\n\n2. **Different numpy errstate handling for different modes (lines 111-114):** For 'cd' mode, `errors = dict(over='raise', invalid='ignore')` is used, while for 'lars' mode, `errors = dict(invalid='raise')`. This means CD ignores invalid floating point results while raising on overflow, whereas LARS raises on invalid results.\n\n3. **Warm-starting initialization for CD with regularized division (line 136-138):** The initial coefficients for CD are computed as `-(precision_[indices != idx, idx] / (precision_[idx, idx] + 1000 * eps))`. The `1000 * eps` added to the denominator prevents division by zero or near-zero values, serving as regularization.\n\n4. **Coupled inner/outer max_iter with minimum of 20 (line 152):** The code uses `max(20, max_iter)` for the inner CD solver, ensuring at least 20 iterations for the inner coordinate descent regardless of the outer `max_iter` setting.\n\n5. **Diagonal exclusion from penalty (lines 54, 66):** Both `_objective` and `_dual_gap` subtract `np.abs(np.diag(precision_)).sum()` from the L1 penalty term, meaning diagonal coefficients are explicitly not penalized.\n\n6. **Reduced max_iter during cross-validation (line 1035):** `GraphicalLassoCV.fit` passes `int(0.1 * self.max_iter)` during the CV grid search - only 10% of the max iterations. ConvergenceWarnings are suppressed during this phase.\n\n7. **`graphical_lasso` public function delegates to `GraphicalLasso` class (lines 348-365):** The public `graphical_lasso` function internally creates a `GraphicalLasso` instance with `covariance='precomputed'` and `assume_centered=True`, then fits it and extracts results.\n\n8. **`BaseGraphicalLasso` removes `store_precision` from parameter constraints (line 378):** The class explicitly pops `store_precision` from the inherited constraints.\n\n9. **`scipy.linalg.pinvh` for initial precision (line 105):** The pseudo-inverse for Hermitian matrices is used rather than a standard inverse for computing the initial precision from the covariance.\n\n10. **Score sanity check in CV (line 1056):** Scores >= `0.1 / np.finfo(np.float64).eps` are treated as NaN to avoid selecting pathologically large scores from ill-conditioned estimates.",
    "rubric": [
      "Off-diagonal coefficients scaled by 0.95 as Tikhonov-like regularization to handle ill-conditioned covariance initializations during cross-validation",
      "CD mode uses errors dict with over='raise' and invalid='ignore', while LARS uses invalid='raise' because the solvers have different numerical error profiles",
      "The 1000*eps added to the denominator when computing initial CD coefficients prevents division by zero or near-zero precision diagonal values",
      "The inner CD solver uses max(20, max_iter) to ensure at least 20 iterations regardless of outer max_iter setting",
      "Diagonal elements are excluded from the L1 penalty in both _objective and _dual_gap functions",
      "scipy.linalg.pinvh (pseudo-inverse for Hermitian matrices) is used instead of standard inverse for initial precision computation",
      "During cross-validation, only 10% of max_iter is used (int(0.1 * self.max_iter)) with ConvergenceWarnings suppressed",
      "Scores >= 0.1/eps are treated as NaN during CV to avoid selecting pathologically large scores from ill-conditioned estimates"
    ],
    "key_files": [
      "sklearn/covariance/_graph_lasso.py",
      "sklearn/covariance/_empirical_covariance.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_graph_lasso.py] sklearn.covariance._graph_lasso.graphical_lasso\nL1-penalized covariance estimator.\n\n    Read more in the :ref:`User Guide <sparse_inverse_covariance>`.\n\n    .. versionchanged:: v0.20\n        graph_lasso has been renamed to graphical_lasso\n\n    Parameters\n    ----------\n    emp_cov : array-like of shape (n_features, n_features)\n        Empirical covariance from which to compute the covariance estimate.\n\n    alpha : float\n        The regularization parameter: the higher alpha, the more\n        regularization, the sparser the inverse covariance.\n        Range is (0, inf].\n\n    mode : {'cd', 'lars'}, default='cd'\n        The Lasso solver to use: coordinate descent or LARS. Use LARS for\n        very sparse underlying graphs, where p > n. Elsewhere prefer cd\n        which is more numerically stable.\n\n    tol : float, default=1e-4\n        The tolerance to declare convergence: if the dual gap goes below\n        this value, iterations are stopped. Range is (0, inf].\n\n    enet_tol : float, default=1e-4\n        The tolerance for the elastic net solver used to calculate the descent\n        direction. This parameter controls the accuracy of the search direction\n        for a given column update, not of the overall parameter estimate. Only\n        used for mode='cd'. Range is (0, inf].\n\n    max_iter : int, default=100\n        The maximum number of iterations.\n\n    verbose : bool, default=False\n        If verbose is True, the objective function and dual gap are\n        printed at each iteration.\n\n    return_costs : bool, default=False\n        If return_costs is True, the objective function and dual gap\n        at each iteration are returned.\n\n    eps : float, default=eps\n        The machine-precision regularization in the computation of the\n        Cholesky diagonal factors. Increase this for very ill-conditioned\n        systems. Default is `np.finfo(np.float64).eps`.\n\n    return_n_iter : bool, default=False\n        Whether or not to return the number of iterations.\n\n    Returns\n    -------\n    covariance : ndarray of shape (n_features, n_features)\n        The estimated covariance matrix.\n\n    precision : ndarray of shape (n_features, n_features)\n        The estimated (sparse) precision matrix.\n\n    costs : list of (objective, dual_gap) pairs\n        The list of values of the objective function and the dual gap at\n        each iteration. Returned only if return_costs is True.\n\n    n_iter : int\n        Number of iterations. Returned only if `return_n_iter` is set to True.\n\n    See Also\n    --------\n    GraphicalLasso : Sparse inverse covariance estimation\n        with an l1-penalized estimator.\n    GraphicalLassoCV : Sparse inverse covariance with\n        cross-validated choice of the l1 penalty.\n\n    Notes\n    -----\n    The algorithm employed to solve this problem is the GLasso algorithm,\n    from the Friedman 2008 Biostatistics paper. It is the same algorithm\n    as in the R `glasso` package.\n\n    One possible difference with the `glasso` R package is that the\n    diagonal coefficients are not penalized.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.datasets import make_sparse_spd_matrix\n    >>> from sklearn.covariance import empirical_covariance, graphical_lasso\n    >>> true_cov = make_sparse_spd_matrix(n_dim=3,random_state=42)\n    >>> rng = np.random.RandomState(42)\n    >>> X = rng.multivariate_normal(mean=np.zeros(3), cov=true_cov, size=3)\n    >>> emp_cov = empirical_covariance(X, assume_centered=True)\n    >>> emp_cov, _ = graphical_lasso(emp_cov, alpha=0.05)\n    >>> emp_cov\n    array([[ 1.687,  0.212, -0.209],\n           [ 0.212,  0.221, -0.0817],\n           [-0.209, -0.0817, 0.232]])",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer reference specific line numbers and internal implementation details (e.g., scaling by 0.95, numpy errstate handling, warm-starting initialization, coupled inner/outer max_iter, reduced max_iter during CV, score sanity check, scipy.linalg.pinvh usage, BaseGraphicalLasso removing store_precision) that are not present in the provided documentation. These appear to be derived from reading the actual source code rather than the docstring provided.",
      "Claim 5 about diagonal exclusion from penalty is consistent with the documentation's note: 'One possible difference with the glasso R package is that the diagonal coefficients are not penalized.'",
      "Claim 7 states the public graphical_lasso function internally creates a GraphicalLasso instance with covariance='precomputed' and assume_centered=True. This is not mentioned in the documentation and cannot be verified from it, though it is plausible implementation detail.",
      "No claims directly contradict the provided documentation, but the vast majority of the answer's content goes far beyond what the documentation states and cannot be verified from it alone."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims relied on stripped comments (Tikhonov rationale, TODO note text, solver error explanations) but most design patterns are verifiable from code structure alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_41",
    "repo": "scikit_learn",
    "question": "What is the `ShrunkCovariance` class, how does it relate to `EmpiricalCovariance`, and how does its shrinkage regularization work?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `ShrunkCovariance` class is a covariance estimator that applies diagonal shrinkage regularization to the empirical covariance matrix. It inherits from `EmpiricalCovariance` (which itself inherits from `BaseEstimator`), and applies the shrinkage formula: `(1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features)` where `mu = trace(cov) / n_features`.\n\nKey implementation details:\n\n1. **The `fit` method does NOT call the parent class `fit`** \u2014 instead, it directly computes the empirical covariance, applies shrinkage via the `shrunk_covariance()` function, and then calls `self._set_covariance()`.\n\n2. **The `_set_covariance` method** (inherited from `EmpiricalCovariance`) uses `scipy.linalg.pinvh` (pseudo-inverse for Hermitian matrices) with `check_finite=False` to compute the precision matrix. If `store_precision=False`, it sets `self.precision_` to `None`.\n\n3. **The standalone `shrunk_covariance()` function supports n-dimensional (batched) input** \u2014 it uses `check_array(emp_cov, allow_nd=True)`, computes trace using `np.trace(emp_cov, axis1=-2, axis2=-1)`, and uses `np.expand_dims` to handle broadcasting for batched covariance matrices.\n\n4. **Parameter validation**: `ShrunkCovariance._parameter_constraints` extends `EmpiricalCovariance._parameter_constraints` (which validates `store_precision` as boolean and `assume_centered` as boolean) by adding `shrinkage` with constraint `Interval(Real, 0, 1, closed=\"both\")`.\n\n5. **The `_fit_context` decorator** is applied to the `fit` method with `prefer_skip_nested_validation=True`. The `_validate_params` method in `BaseEstimator` calls `validate_parameter_constraints` with the class's `_parameter_constraints`.\n\n6. **Inherited methods**: `ShrunkCovariance` inherits `score()` (computes log-likelihood using `log_likelihood(test_cov, self.get_precision())`), `error_norm()` (supports frobenius and spectral norms), `mahalanobis()` (uses `pairwise_distances` with the mahalanobis metric and precision as `VI`), and `get_precision()` (recomputes via `pinvh` if precision wasn't stored).\n\n7. **The `empirical_covariance` function** uses `np.cov(X.T, bias=1)` when data is not assumed centered, and `np.dot(X.T, X) / X.shape[0]` when it is. It passes `ensure_all_finite=False` to `check_array`.",
    "rubric": [
      "Explains that ShrunkCovariance inherits from EmpiricalCovariance (which inherits from BaseEstimator) and applies diagonal shrinkage regularization",
      "States the shrinkage formula: (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features) where mu = trace(cov) / n_features",
      "Notes that ShrunkCovariance.fit() does NOT call the parent fit method \u2014 it directly computes empirical covariance, applies shrunk_covariance(), then calls _set_covariance()",
      "Describes that _set_covariance uses scipy.linalg.pinvh (with check_finite=False) to compute the precision matrix, or sets precision_ to None if store_precision is False",
      "Mentions that the standalone shrunk_covariance() function supports n-dimensional/batched input via allow_nd=True and axis-aware trace computation",
      "Describes parameter validation: shrinkage is constrained to Interval(Real, 0, 1, closed='both'), extending the parent's boolean constraints for store_precision and assume_centered",
      "Mentions key inherited methods like score() (uses log_likelihood), error_norm() (frobenius/spectral), mahalanobis() (uses pairwise_distances), and get_precision() (recomputes via pinvh if not stored)"
    ],
    "key_files": [
      "sklearn/covariance/_shrunk_covariance.py",
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_shrunk_covariance.py] sklearn.covariance._shrunk_covariance.ShrunkCovariance\nsklearn.covariance._shrunk_covariance.ShrunkCovariance:\n  Covariance estimator with shrinkage.\n\n    Read more in the :ref:`User Guide <shrunk_covariance>`.\n\n    Parameters\n    ----------\n    store_precision : bool, default=True\n        Specify if the estimated precision is stored.\n\n    assume_centered : bool, default=False\n        If True, data will not be centered before computation.\n        Useful when working with data whose mean is almost, but not exactly\n        zero.\n        If False, data will be centered before computation.\n\n    shrinkage : float, default=0.1\n        Coefficient in the convex combination used for the computation\n        of the shrunk estimate. Range is [0, 1].\n\n    Attributes\n    ----------\n    covariance_ : ndarray of shape (n_features, n_features)\n        Estimated covariance matrix\n\n    location_ : ndarray of shape (n_features,)\n        Estimated location, i.e. the estimated mean.\n\n    precision_ : ndarray of shape (n_features, n_features)\n        Estimated pseudo inverse matrix.\n        (stored only if store_precision is True)\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n        .. versionadded:: 0.24\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    EllipticEnvelope : An object for detecting outliers in\n        a Gaussian distributed dataset.\n    EmpiricalCovariance : Maximum likelihood covariance estimator.\n    GraphicalLasso : Sparse inverse covariance estimation\n        with an l1-penalized estimator.\n    GraphicalLassoCV : Sparse inverse covariance with cross-validated\n        choice of the l1 penalty.\n    LedoitWolf : LedoitWolf Estimator.\n    MinCovDet : Minimum Covariance Determinant\n        (robust estimator of covariance).\n    OAS : Oracle Approximating Shrinkage Estimator.\n\n    Notes\n    -----\n    The regularized covariance is given by:\n\n    (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features)\n\n    where mu = trace(cov) / n_features\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.covariance import ShrunkCovariance\n    >>> from sklearn.datasets import make_gaussian_quantiles\n    >>> real_cov = np.array([[.8, .3],\n    ...                      [.3, .4]])\n    >>> rng = np.random.RandomState(0)\n    >>> X = rng.multivariate_normal(mean=[0, 0],\n    ...                                   cov=real_cov,\n    ...                                   size=500)\n    >>> cov = ShrunkCovariance().fit(X)\n    >>> cov.covariance_\n    array([[0.7387, 0.2536],\n           [0.2536, 0.4110]])\n    >>> cov.location_\n    array([0.0622, 0.0193])\n\nsklearn.covariance._shrunk_covariance.ShrunkCovariance.fit:\n  Fit the shrunk covariance model to X.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            Training data, where `n_samples` is the number of samples\n            and `n_features` is the number of features.\n\n        y : Ignored\n            Not used, present for API consistency by convention.\n\n        Returns\n        -------\n        self : object\n            Returns the instance itself.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Claims about internal implementation details (points 1-7) go beyond the provided documentation, but are presented as 'Key implementation details not in the documentation' and appear to be plausible code-level details consistent with scikit-learn's architecture.",
      "The claim that ShrunkCovariance inherits from EmpiricalCovariance is not explicitly stated in the documentation but is consistent with the See Also section listing EmpiricalCovariance and the general sklearn covariance module structure.",
      "The shrinkage formula matches the documentation exactly: '(1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features)' where 'mu = trace(cov) / n_features'.",
      "All parameter descriptions (store_precision, assume_centered, shrinkage) and attribute descriptions are consistent with the documentation.",
      "The shrinkage parameter range [0, 1] with Interval(Real, 0, 1, closed='both') is consistent with the documented 'Range is [0, 1]'."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor leakage from a stripped comment explaining why parent fit is not called, and some details about _fit_context behavior not visible in shown code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_40",
    "repo": "scikit_learn",
    "question": "How does the `shrunk_covariance` function apply shrinkage, and how does its N-D array handling differ from the internal `_ledoit_wolf` and `_oas` helpers?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `shrunk_covariance` function in `sklearn/covariance/_shrunk_covariance.py` implements shrinkage of covariance matrices using the following algorithm:\n\n1. **Input validation**: It calls `check_array(emp_cov, allow_nd=True)` which allows N-dimensional arrays (not just 2D), enabling batch processing of multiple covariance matrices simultaneously.\n\n2. **Core computation**:\n   - Gets `n_features` from `emp_cov.shape[-1]` (last dimension)\n   - Computes `shrunk_cov = (1.0 - shrinkage) * emp_cov` (scales off-diagonal and diagonal)\n   - Computes `mu = np.trace(emp_cov, axis1=-2, axis2=-1) / n_features` using the last two axes for trace computation\n   - Uses `np.expand_dims(mu, axis=tuple(range(mu.ndim, emp_cov.ndim)))` to broadcast `mu` across the identity matrix dimensions\n   - Adds `shrinkage * mu * np.eye(n_features)` to the diagonal\n\n3. **Multi-dimensional support**: Unlike the internal `_ledoit_wolf` and `_oas` functions which use `shrunk_cov.flat[:: n_features + 1] += shrinkage * mu` (flat indexing trick to modify the diagonal in-place), the public `shrunk_covariance` function uses `np.eye(n_features)` with broadcasting to support batches of covariance matrices (N-D input).\n\n4. **Parameter validation**: The `@validate_params` decorator enforces that `shrinkage` is a `Real` number in the interval `[0, 1]` (closed on both sides) using `Interval(Real, 0, 1, closed=\"both\")`.\n\n5. **Integration with ShrunkCovariance class**: The `ShrunkCovariance.fit()` method does NOT call the parent `EmpiricalCovariance.fit()`. It computes the empirical covariance, applies `shrunk_covariance()`, then calls `self._set_covariance(covariance)` which stores the covariance and computes the precision matrix using `linalg.pinvh(covariance, check_finite=False)` only if `store_precision=True`.\n\n6. **empirical_covariance function**: Uses `np.cov(X.T, bias=1)` when data is not assumed centered, or `np.dot(X.T, X) / X.shape[0]` when assumed centered.",
    "rubric": [
      "Explains the shrinkage formula: (1 - shrinkage) * emp_cov + shrinkage * mu * identity, where mu = trace(emp_cov) / n_features",
      "Mentions that shrunk_covariance uses check_array with allow_nd=True to accept N-dimensional (batch) inputs",
      "Describes that n_features comes from emp_cov.shape[-1] and np.trace uses axis1=-2, axis2=-1 for N-D support",
      "Notes the use of np.expand_dims and np.eye(n_features) with broadcasting for adding to the diagonal in the public function",
      "Contrasts with _ledoit_wolf and _oas which use flat indexing (shrunk_cov.flat[::n_features+1] += shrinkage * mu) for in-place diagonal modification, limiting them to 2D",
      "Mentions the @validate_params decorator constraining shrinkage to the [0, 1] interval (Interval(Real, 0, 1, closed='both'))",
      "Explains how ShrunkCovariance.fit() bypasses the parent fit(), computes empirical covariance, applies shrunk_covariance, and stores results via _set_covariance"
    ],
    "key_files": [
      "sklearn/covariance/_shrunk_covariance.py",
      "sklearn/covariance/_empirical_covariance.py",
      "sklearn/covariance/tests/test_covariance.py"
    ],
    "source_doc": "[docstring: sklearn/covariance/_shrunk_covariance.py] sklearn.covariance._shrunk_covariance.shrunk_covariance\nCalculate covariance matrices shrunk on the diagonal.\n\n    Read more in the :ref:`User Guide <shrunk_covariance>`.\n\n    Parameters\n    ----------\n    emp_cov : array-like of shape (..., n_features, n_features)\n        Covariance matrices to be shrunk, at least 2D ndarray.\n\n    shrinkage : float, default=0.1\n        Coefficient in the convex combination used for the computation\n        of the shrunk estimate. Range is [0, 1].\n\n    Returns\n    -------\n    shrunk_cov : ndarray of shape (..., n_features, n_features)\n        Shrunk covariance matrices.\n\n    Notes\n    -----\n    The regularized (shrunk) covariance is given by::\n\n        (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features)\n\n    where `mu = trace(cov) / n_features`.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> from sklearn.datasets import make_gaussian_quantiles\n    >>> from sklearn.covariance import empirical_covariance, shrunk_covariance\n    >>> real_cov = np.array([[.8, .3], [.3, .4]])\n    >>> rng = np.random.RandomState(0)\n    >>> X = rng.multivariate_normal(mean=[0, 0], cov=real_cov, size=500)\n    >>> shrunk_covariance(empirical_covariance(X))\n    array([[0.739, 0.254],\n           [0.254, 0.411]])",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims go well beyond the provided documentation, describing internal implementation details (line numbers, flat indexing tricks, validate_params decorator, ShrunkCovariance class internals, empirical_covariance implementation) that cannot be verified from the given docstring alone.",
      "However, the core formula described in point 2 is consistent with the documentation's Notes section: (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features) where mu = trace(cov) / n_features.",
      "The parameter description (shrinkage in [0,1], emp_cov shape, return shape) is consistent with the documentation.",
      "No claims directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in design rationale ('avoid unnecessary matrix inversion'), function descriptions ('pseudo-inverse for Hermitian matrices', 'MLE estimator dividing by N'), and the label 'diagonal shrinkage'; all core algorithmic claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_43",
    "repo": "scikit_learn",
    "question": "Where is PLSCanonical implemented, and how does its configuration and deflation behavior differ from PLSRegression and CCA?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "PLSCanonical is located in `sklearn/cross_decomposition/_pls.py` and inherits from the abstract base class `_PLS`. Here is where the key functionality lives and flows:\n\n**Class hierarchy and location:**\n- `PLSCanonical` (line 660) extends `_PLS` (line 155), which is an abstract class with metaclass `ABCMeta`\n- `_PLS` inherits from five mixins/bases in this order: `ClassNamePrefixFeaturesOutMixin`, `TransformerMixin`, `RegressorMixin`, `MultiOutputMixin`, `BaseEstimator`\n- PLSCanonical's `__init__` (line 771-790) hardcodes `deflation_mode=\"canonical\"` and `mode=\"A\"` when calling `super().__init__()`, while exposing `algorithm` as a user-configurable parameter (unlike PLSRegression which hardcodes `algorithm=\"nipals\"` and `mode=\"A\"` with `deflation_mode=\"regression\"`)\n- CCA also uses `deflation_mode=\"canonical\"` but sets `mode=\"B\"` and hardcodes `algorithm=\"nipals\"`\n\n**Key implementation details in `_PLS.fit()`:**\n- `_norm_y_weights` is set to `True` when `deflation_mode == \"canonical\"` (line 264), meaning PLSCanonical and CCA normalize y_weights but PLSRegression does not\n- The `rank_upper_bound` for `n_components` differs: for `deflation_mode=\"regression\"` it's `min(n, p)`, but for `deflation_mode=\"canonical\"` (PLSCanonical/CCA) it's `min(n, p, q)` (line 256-257)\n- During canonical deflation (line 331-334), y is deflated by regressing on `y_scores`, whereas in regression mode (line 335-338), y is deflated by regressing on `x_scores`\n- The fit method stores internal `_x_scores` and `_y_scores` as private attributes, but PLSRegression's overridden `fit()` additionally exposes them as public `x_scores_` and `y_scores_` attributes (lines 655-656); PLSCanonical does NOT override `fit()` and does NOT expose these publicly\n- `_predict_1d` flag is set in `_PLS.fit()` (line 242-245): if y is 1D input, predictions will be returned as 1D via `y_pred.ravel()` in `predict()`\n\n**Power method (`_get_first_singular_vectors_power_method`):**\n- For mode \"A\" (PLSCanonical), x_weights are computed as `X.T @ y_score / (y_score @ y_score)` and y_weights as `y.T @ x_score / (x_score.T @ x_score)` (lines 82, 90)\n- For mode \"B\" (CCA), pseudo-inverses of X and y are precomputed using `_pinv2_old()` and used instead (lines 76, 80, 88)\n- Convergence is checked as `np.dot(x_weights_diff, x_weights_diff) < tol` (line 98), and the loop also breaks early when `y.shape[1] == 1` (single target)\n- Initial `y_score` is set to the first column of y whose values are not all near zero (line 63)\n\n**Parameter constraints:**\n- PLSCanonical's `_parameter_constraints` is derived from `_PLS._parameter_constraints` by popping `\"deflation_mode\"` and `\"mode\"` (lines 759-761), leaving `algorithm` available to users\n- PLSRegression and CCA pop `\"deflation_mode\"`, `\"mode\"`, AND `\"algorithm\"` (3 params removed vs 2)\n\n**Tags and testing:**\n- `_PLS.__sklearn_tags__()` sets `tags.regressor_tags.poor_score = True` and `tags.target_tags.required = False` (lines 501-504)\n- `MultiOutputMixin` is included in the base classes of `_PLS`\n- The test `test_one_component_equivalence` verifies that PLSSVD, PLSRegression, and PLSCanonical produce equivalent results when `n_components=1`\n- The rotation matrices are computed as `x_weights @ pinv(x_loadings.T @ x_weights)` (line 354-357)\n- `coef_` is computed as `(x_rotations @ y_loadings.T * _y_std).T / _x_std` (lines 362-363), and `intercept_` is simply `_y_mean` (line 364)\n\n**Module exposure:**\n- `__all__` in `_pls.py` exports `[\"PLSSVD\", \"PLSCanonical\", \"PLSRegression\"]` (notably excludes CCA)\n- `__init__.py` imports and re-exports all four: `CCA`, `PLSSVD`, `PLSCanonical`, `PLSRegression`",
    "rubric": [
      "PLSCanonical is defined in sklearn/cross_decomposition/_pls.py and inherits from the abstract base class _PLS",
      "_PLS inherits from ClassNamePrefixFeaturesOutMixin, TransformerMixin, RegressorMixin, MultiOutputMixin, and BaseEstimator (with ABCMeta metaclass)",
      "PLSCanonical hardcodes deflation_mode='canonical' and mode='A' in its __init__, while exposing algorithm as user-configurable",
      "PLSRegression hardcodes deflation_mode='regression', mode='A', and algorithm='nipals'; CCA hardcodes deflation_mode='canonical', mode='B', and algorithm='nipals'",
      "In canonical deflation mode, y is deflated by regressing on y_scores; in regression mode, y is deflated by regressing on x_scores",
      "_norm_y_weights is True when deflation_mode=='canonical' (PLSCanonical/CCA), False for PLSRegression",
      "The n_components upper bound is min(n, p, q) for canonical deflation vs min(n, p) for regression deflation",
      "PLSRegression overrides fit() to expose x_scores_ and y_scores_ as public attributes; PLSCanonical does not override fit() and only stores them as private (_x_scores, _y_scores)",
      "PLSCanonical's _parameter_constraints pops 'deflation_mode' and 'mode' (2 params), while PLSRegression and CCA also pop 'algorithm' (3 params)"
    ],
    "key_files": [
      "sklearn/cross_decomposition/_pls.py",
      "sklearn/cross_decomposition/__init__.py",
      "sklearn/cross_decomposition/tests/test_pls.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/cross_decomposition/_pls.py] sklearn.cross_decomposition._pls.PLSCanonical\nsklearn.cross_decomposition._pls.PLSCanonical:\n  Partial Least Squares transformer and regressor.\n\n    For a comparison between other cross decomposition algorithms, see\n    :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`.\n\n    Read more in the :ref:`User Guide <cross_decomposition>`.\n\n    .. versionadded:: 0.8\n\n    Parameters\n    ----------\n    n_components : int, default=2\n        Number of components to keep. Should be in `[1, min(n_samples,\n        n_features, n_targets)]`.\n\n    scale : bool, default=True\n        Whether to scale `X` and `y`.\n\n    algorithm : {'nipals', 'svd'}, default='nipals'\n        The algorithm used to estimate the first singular vectors of the\n        cross-covariance matrix. 'nipals' uses the power method while 'svd'\n        will compute the whole SVD.\n\n    max_iter : int, default=500\n        The maximum number of iterations of the power method when\n        `algorithm='nipals'`. Ignored otherwise.\n\n    tol : float, default=1e-06\n        The tolerance used as convergence criteria in the power method: the\n        algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less\n        than `tol`, where `u` corresponds to the left singular vector.\n\n    copy : bool, default=True\n        Whether to copy `X` and `y` in fit before applying centering, and\n        potentially scaling. If False, these operations will be done inplace,\n        modifying both arrays.\n\n    Attributes\n    ----------\n    x_weights_ : ndarray of shape (n_features, n_components)\n        The left singular vectors of the cross-covariance matrices of each\n        iteration.\n\n    y_weights_ : ndarray of shape (n_targets, n_components)\n        The right singular vectors of the cross-covariance matrices of each\n        iteration.\n\n    x_loadings_ : ndarray of shape (n_features, n_components)\n        The loadings of `X`.\n\n    y_loadings_ : ndarray of shape (n_targets, n_components)\n        The loadings of `y`.\n\n    x_rotations_ : ndarray of shape (n_features, n_components)\n        The projection matrix used to transform `X`.\n\n    y_rotations_ : ndarray of shape (n_targets, n_components)\n        The projection matrix used to transform `y`.\n\n    coef_ : ndarray of shape (n_targets, n_features)\n        The coefficients of the linear model such that `y` is approximated as\n        `y = X @ coef_.T + intercept_`.\n\n    intercept_ : ndarray of shape (n_targets,)\n        The intercepts of the linear model such that `y` is approximated as\n        `y = X @ coef_.T + intercept_`.\n\n        .. versionadded:: 1.1\n\n    n_iter_ : list of shape (n_components,)\n        Number of iterations of the power method, for each\n        component. Empty if `algorithm='svd'`.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    CCA : Canonical Correlation Analysis.\n    PLSSVD : Partial Least Square SVD.\n\n    Examples\n    --------\n    >>> from sklearn.cross_decomposition import PLSCanonical\n    >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]]\n    >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]]\n    >>> plsca = PLSCanonical(n_components=2)\n    >>> plsca.fit(X, y)\n    PLSCanonical()\n    >>> X_c, y_c = plsca.transform(X, y)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states convergence is checked as 'np.dot(x_weights_diff, x_weights_diff) < tol', but the documentation says 'the squared norm of u_i - u_{i-1} is less than tol, where u corresponds to the left singular vector.' These are consistent (x_weights is the left singular vector), but the variable naming differs slightly from the doc's notation.",
      "The answer claims line numbers and specific implementation details that cannot be verified from the provided documentation alone - these are plausible code-level details but unverifiable from the doc chunk.",
      "The answer states PLSCanonical 'does NOT override fit() and does NOT expose [x_scores_, y_scores_] publicly' - this is not contradicted by the docs (which don't mention these attributes), but is unverifiable from the provided chunk.",
      "The answer states '__all__ in _pls.py exports [\"PLSSVD\", \"PLSCanonical\", \"PLSRegression\"] (notably excludes CCA)' - this is a code-level claim that cannot be verified from the documentation provided."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the MultiOutputMixin tag behavior claim needed minor rewriting since its implementation wasn't fully visible.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_42",
    "repo": "scikit_learn",
    "question": "Why does PLSRegression differ from PLSCanonical and CCA in its hardcoded parameters, deflation logic, score exposure, rank bounds, and prediction mechanics?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "PLSRegression's design is built around several deliberate decisions in the _PLS base class and its specialized subclass:\n\n1. **Why PLSRegression hardcodes `deflation_mode='regression'`, `mode='A'`, and `algorithm='nipals'`**: PLSRegression removes these three parameters from its `_parameter_constraints` and fixes them internally in its `__init__` by calling `super().__init__` with these fixed values. In `deflation_mode='regression'`, y is deflated using x_scores (as seen in the code: `y_loadings = np.dot(x_scores, yk) / np.dot(x_scores, x_scores)`), unlike canonical mode which deflates y using y_scores. The `mode='A'` means it uses `X.T @ y_score / (y_score.T @ y_score)` for computing weights (not pseudo-inverse as in CCA's mode B). The algorithm is always 'nipals' (power method) \u2014 unlike PLSCanonical which exposes the 'svd' option in its constructor.\n\n2. **Why `predict` only centers X but does not scale it**: In the `predict` method, only `X -= self._x_mean` is applied without dividing by `_x_std`. This is because `coef_` already incorporates the scaling. Specifically, `coef_` is computed as `(x_rotations_ @ y_loadings_.T * _y_std).T / _x_std`, which bakes the x_std division and y_std multiplication into the coefficients.\n\n3. **Why `intercept_` is simply `_y_mean`**: Since X is centered (X -= x_mean) during prediction, and the coefficients incorporate all scaling, the intercept is just the mean of y computed during fitting (`self.intercept_ = self._y_mean`).\n\n4. **Why the power method breaks immediately for single-target problems**: The convergence check contains the condition `y.shape[1] == 1`, causing the loop to break after the first iteration regardless of convergence.\n\n5. **Why `_norm_y_weights` is set to `False` for PLSRegression**: The code sets `self._norm_y_weights = self.deflation_mode == 'canonical'`, so PLSRegression (deflation_mode='regression') does NOT normalize y_weights. This affects how y_scores are computed \u2014 without normalization, `y_ss = np.dot(y_weights, y_weights)` and `y_scores = np.dot(yk, y_weights) / y_ss`, whereas with normalization `y_ss=1`.\n\n6. **Why `x_scores_` and `y_scores_` are only exposed by PLSRegression (not PLSCanonical/CCA)**: PLSRegression overrides `fit()` to explicitly assign `self.x_scores_ = self._x_scores` and `self.y_scores_ = self._y_scores` as public attributes. The parent `_PLS` class stores them as private `_x_scores`/`_y_scores`, and PLSCanonical/CCA do not override `fit()`.\n\n7. **Why the rank bound differs**: For PLSRegression, n_components is bounded by `min(n, p)` (where n=n_samples, p=n_features), while for canonical/CCA modes it's bounded by `min(n, p, q)` (where q=n_targets). This is implemented as: `rank_upper_bound = min(n, p) if self.deflation_mode == 'regression' else min(n, p, q)`.\n\n8. **Why `_pinv2_old` exists**: It is a custom pseudo-inverse function that uses a specific condition threshold of `max(s) * factor[dtype] * eps` where factor is `{'f': 1e3, 'd': 1e6}` (1e3 for float32 and 1e6 for float64). It performs SVD, filters singular values below this threshold, and computes the pseudo-inverse.\n\n9. **Why near-zero columns in yk are zeroed out before NIPALS**: The code applies `yk_mask = np.all(np.abs(yk) < 10 * y_eps, axis=0); yk[:, yk_mask] = 0.0` before the NIPALS power method is called, zeroing out columns where all values are below `10 * y_eps`.",
    "rubric": [
      "PLSRegression hardcodes deflation_mode='regression', mode='A', and algorithm='nipals' by removing them from _parameter_constraints and passing fixed values to super().__init__",
      "In regression deflation mode, y is deflated using x_scores (y_loadings = x_scores.dot(yk) / x_scores.dot(x_scores)), unlike canonical mode which deflates y using y_scores",
      "predict only centers X (X -= _x_mean) without scaling because coef_ already incorporates _x_std and _y_std via the formula (x_rotations_ @ y_loadings_.T * _y_std).T / _x_std",
      "intercept_ is simply _y_mean because X is already centered and all scaling is baked into coef_",
      "The power method breaks after one iteration for single-target problems due to the condition 'y.shape[1] == 1' in the convergence check",
      "_norm_y_weights is False for PLSRegression (set via deflation_mode == 'canonical'), meaning y_weights are not normalized and y_ss = dot(y_weights, y_weights) is used in computing y_scores",
      "PLSRegression overrides fit() to expose x_scores_ and y_scores_ as public attributes (copied from private _x_scores/_y_scores), which PLSCanonical and CCA do not do",
      "For PLSRegression, n_components is bounded by min(n_samples, n_features), while canonical/CCA modes bound it by min(n_samples, n_features, n_targets)",
      "_pinv2_old uses a custom singular value threshold based on dtype-specific factors (1e3 for float32, 1e6 for float64) times machine epsilon"
    ],
    "key_files": [
      "sklearn/cross_decomposition/_pls.py",
      "sklearn/base.py"
    ],
    "source_doc": "[docstring: sklearn/cross_decomposition/_pls.py] sklearn.cross_decomposition._pls.PLSRegression\nsklearn.cross_decomposition._pls.PLSRegression:\n  PLS regression.\n\n    PLSRegression is also known as PLS2 or PLS1, depending on the number of\n    targets.\n\n    For a comparison between other cross decomposition algorithms, see\n    :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`.\n\n    Read more in the :ref:`User Guide <cross_decomposition>`.\n\n    .. versionadded:: 0.8\n\n    Parameters\n    ----------\n    n_components : int, default=2\n        Number of components to keep. Should be in `[1, n_features]`.\n\n    scale : bool, default=True\n        Whether to scale `X` and `y`.\n\n    max_iter : int, default=500\n        The maximum number of iterations of the power method when\n        `algorithm='nipals'`. Ignored otherwise.\n\n    tol : float, default=1e-06\n        The tolerance used as convergence criteria in the power method: the\n        algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less\n        than `tol`, where `u` corresponds to the left singular vector.\n\n    copy : bool, default=True\n        Whether to copy `X` and `y` in :term:`fit` before applying centering,\n        and potentially scaling. If `False`, these operations will be done\n        inplace, modifying both arrays.\n\n    Attributes\n    ----------\n    x_weights_ : ndarray of shape (n_features, n_components)\n        The left singular vectors of the cross-covariance matrices of each\n        iteration.\n\n    y_weights_ : ndarray of shape (n_targets, n_components)\n        The right singular vectors of the cross-covariance matrices of each\n        iteration.\n\n    x_loadings_ : ndarray of shape (n_features, n_components)\n        The loadings of `X`.\n\n    y_loadings_ : ndarray of shape (n_targets, n_components)\n        The loadings of `y`.\n\n    x_scores_ : ndarray of shape (n_samples, n_components)\n        The transformed training samples.\n\n    y_scores_ : ndarray of shape (n_samples, n_components)\n        The transformed training targets.\n\n    x_rotations_ : ndarray of shape (n_features, n_components)\n        The projection matrix used to transform `X`.\n\n    y_rotations_ : ndarray of shape (n_targets, n_components)\n        The projection matrix used to transform `y`.\n\n    coef_ : ndarray of shape (n_target, n_features)\n        The coefficients of the linear model such that `y` is approximated as\n        `y = X @ coef_.T + intercept_`.\n\n    intercept_ : ndarray of shape (n_targets,)\n        The intercepts of the linear model such that `y` is approximated as\n        `y = X @ coef_.T + intercept_`.\n\n        .. versionadded:: 1.1\n\n    n_iter_ : list of shape (n_components,)\n        Number of iterations of the power method, for each\n        component.\n\n    n_features_in_ : int\n        Number of features seen during :term:`fit`.\n\n    feature_names_in_ : ndarray of shape (`n_features_in_`,)\n        Names of features seen during :term:`fit`. Defined only when `X`\n        has feature names that are all strings.\n\n        .. versionadded:: 1.0\n\n    See Also\n    --------\n    PLSCanonical : Partial Least Squares transformer and regressor.\n\n    Examples\n    --------\n    >>> from sklearn.cross_decomposition import PLSRegression\n    >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]]\n    >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]]\n    >>> pls2 = PLSRegression(n_components=2)\n    >>> pls2.fit(X, y)\n    PLSRegression()\n    >>> y_pred = pls2.predict(X)\n\n    For a comparison between PLS Regression and :class:`~sklearn.decomposition.PCA`, see\n    :ref:`sphx_glr_auto_examples_cross_decomposition_plot_pcr_vs_pls.py`.\n\nsklearn.cross_decomposition._pls.PLSRegression.fit:\n  Fit model to data.\n\n        Parameters\n        ----------\n        X : array-like of shape (n_samples, n_features)\n            Training vectors, where `n_samples` is the number of samples and\n            `n_features` is the number of predictors.\n\n        y : array-like of shape (n_samples,) or (n_samples, n_targets)\n            Target vectors, where `n_samples` is the number of samples and\n            `n_targets` is the number of response variables.\n\n        Returns\n        -------\n        self : object\n            Fitted model.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer reference specific line numbers and internal implementation details (e.g., lines 98, 264, 289, 362-363, 475, 612-613, 655-656) that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the given doc chunk.",
      "The answer states coef_ shape incorporates scaling as '(x_rotations_ @ y_loadings_.T * _y_std).T / _x_std' \u2014 this is an implementation detail not in the documentation, though it's plausible.",
      "The documentation describes coef_ shape as (n_target, n_features), and the answer's claim about coef_ incorporating scaling is consistent with the documented formula 'y = X @ coef_.T + intercept_'.",
      "The answer's claim about intercept_ being _y_mean is plausible but not explicitly stated in the documentation \u2014 the doc only says intercept_ is the intercept of the linear model.",
      "The answer says x_scores_ and y_scores_ are 'only exposed by PLSRegression (not PLSCanonical/CCA)' \u2014 the documentation only covers PLSRegression and doesn't make comparative claims about other classes' attribute exposure.",
      "The answer references _pinv2_old and near-zero column zeroing (points 8 and 9) which are pure implementation details not covered by the documentation.",
      "Point 1 mentions algorithm is always 'nipals' for PLSRegression \u2014 the documentation does mention max_iter is used 'when algorithm=\"nipals\"' and says 'Ignored otherwise', which slightly contradicts the claim that algorithm is always nipals (the doc implies there could be other algorithm options, though it doesn't explicitly list them for PLSRegression)."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rationale explanations (performance optimization, mathematical justifications, scipy version history) rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_45",
    "repo": "scikit_learn",
    "question": "What happens internally when `clear_data_home` is called, and what are its dependencies, validation behavior, and side effects?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`clear_data_home` is a two-line function in `sklearn/datasets/_base.py` that first resolves the data home directory path by calling `get_data_home(data_home)`, then deletes it entirely using `shutil.rmtree(data_home)`. Key details:\n\n1. **Dependency on `get_data_home`**: `clear_data_home` delegates path resolution to `get_data_home`, which means if `data_home=None`, it checks the `SCIKIT_LEARN_DATA` environment variable via `environ.get(\"SCIKIT_LEARN_DATA\", join(\"~\", \"scikit_learn_data\"))`. If that env var is also unset, the default path is `~/scikit_learn_data`. The path is expanded via `os.path.expanduser` and the directory is created via `os.makedirs(data_home, exist_ok=True)` before being returned.\n\n2. **Directory is created then immediately destroyed**: Because `clear_data_home` calls `get_data_home` first (which creates the directory with `makedirs`), and then calls `shutil.rmtree`, the directory is momentarily created if it doesn't exist, then immediately removed. After `clear_data_home` completes, the directory does NOT exist (confirmed by the test assertion `assert not os.path.exists(data_home)` in `test_data_home`).\n\n3. **Parameter validation via `@validate_params` decorator**: Both `clear_data_home` and `get_data_home` are decorated with `@validate_params` accepting `data_home` as `[str, os.PathLike, None]` with `prefer_skip_nested_validation=True`. The `validate_params` decorator (in `sklearn/utils/_param_validation.py`) stores the constraints as a `_skl_parameter_constraints` attribute on the function, and at runtime checks `get_config()[\"skip_parameter_validation\"]` to optionally skip validation. When `prefer_skip_nested_validation=True`, the inner call to `get_data_home` from `clear_data_home` will skip its own parameter validation because the decorator wraps the call in `config_context(skip_parameter_validation=True)`.\n\n4. **Widely depended on**: Multiple dataset fetcher modules are imported in `sklearn/datasets/__init__.py` (e.g., `_california_housing.py`, `_covtype.py`, `_openml.py`, `_twenty_newsgroups.py`, etc.), suggesting `clear_data_home` can affect cached data used by these fetchers.\n\n5. **Test coverage**: The test `test_data_home` in `test_base.py` is parametrized with `[None, Path, _DummyPath]` (where `_DummyPath` is a minimal `os.PathLike` implementation with just `__fspath__`) and is marked `@pytest.mark.thread_unsafe`. It verifies the create-delete-recreate cycle: `get_data_home` \u2192 exists, `clear_data_home` \u2192 not exists, `get_data_home` again \u2192 exists.\n\n6. **`shutil` is imported at the module level**, not locally, and `shutil.rmtree` is used directly without any error handling or guards \u2014 if the path doesn't exist as a directory, the call will raise an error (though in practice `get_data_home` ensures it exists first).",
    "rubric": [
      "Explains that clear_data_home calls get_data_home to resolve the data home path, then calls shutil.rmtree to delete it",
      "Describes that get_data_home checks the SCIKIT_LEARN_DATA environment variable (defaulting to ~/scikit_learn_data) and expands the path with expanduser",
      "Notes the create-then-destroy behavior: get_data_home calls makedirs(exist_ok=True) creating the directory, which clear_data_home then immediately removes with rmtree",
      "Mentions the @validate_params decorator on both functions accepting data_home as [str, os.PathLike, None] with prefer_skip_nested_validation=True",
      "Explains that prefer_skip_nested_validation=True causes the nested get_data_home call to skip its own parameter validation",
      "Notes that shutil.rmtree is used without error handling, relying on get_data_home to ensure the directory exists first",
      "Mentions test coverage: test_data_home is parametrized with [None, Path, _DummyPath] and verifies the create-delete-recreate cycle with assert not os.path.exists"
    ],
    "key_files": [
      "sklearn/datasets/_base.py",
      "sklearn/datasets/tests/test_base.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/datasets/__init__.py"
    ],
    "source_doc": "[docstring: sklearn/datasets/_base.py] sklearn.datasets._base.clear_data_home\nDelete all the content of the data home cache.\n\n    Parameters\n    ----------\n    data_home : str or path-like, default=None\n        The path to scikit-learn data directory. If `None`, the default path\n        is `~/scikit_learn_data`.\n\n    Examples\n    --------\n    >>> from sklearn.datasets import clear_data_home\n    >>> clear_data_home()  # doctest: +SKIP",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor line number references and one claim about the exact count of dependent modules needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_44",
    "repo": "scikit_learn",
    "question": "How does `get_data_home` resolve and create the scikit-learn data directory when no argument is provided?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `get_data_home` function in `sklearn/datasets/_base.py` implements a simple 3-step algorithm:\n\n1. **Resolve the path**: If `data_home` is `None`, it reads the `SCIKIT_LEARN_DATA` environment variable via `environ.get('SCIKIT_LEARN_DATA', join('~', 'scikit_learn_data'))`. If the env var is not set, it defaults to `~/scikit_learn_data` (constructed using `os.path.join('~', 'scikit_learn_data')`).\n\n2. **Expand user home**: It calls `os.path.expanduser(data_home)` to resolve the `~` symbol to the actual user home directory. This works whether `data_home` was user-supplied or derived from the environment variable/default.\n\n3. **Create directory**: It calls `os.makedirs(data_home, exist_ok=True)` to create the directory (and any intermediate parent directories) if it doesn't already exist. The `exist_ok=True` flag means no error is raised if the directory already exists.\n\nThe function is decorated with `@validate_params` which enforces that `data_home` must be one of `[str, os.PathLike, None]`. This validation can be globally skipped if the `skip_parameter_validation` config option is set. The `prefer_skip_nested_validation=True` parameter means validation of inner function parameters called by `get_data_home` will be skipped.\n\nThe return type is `str` (as annotated by `-> str`), meaning even if an `os.PathLike` is passed in, `expanduser` converts it to a string.\n\nThe companion function `clear_data_home` calls `get_data_home(data_home)` first (which ensures the directory exists), then immediately calls `shutil.rmtree(data_home)` to delete it entirely \u2014 note it does NOT recreate the directory after deletion.\n\nThe `fetch_file` function uses `get_data_home()` and combines it with a URL-derived folder name to auto-organize downloads (as seen in test code).\n\n`RemoteFileMetadata` is defined as a `namedtuple` with fields `('filename', 'url', 'checksum')` and is used by `_fetch_remote` (as seen from imports of `NamedTemporaryFile` with `delete=False` and `hashlib` for checksum verification).",
    "rubric": [
      "Mentions checking the SCIKIT_LEARN_DATA environment variable via environ.get, with a default of join('~', 'scikit_learn_data')",
      "Mentions calling expanduser to resolve the ~ to the actual user home directory",
      "Mentions calling makedirs with exist_ok=True to create the directory if it doesn't exist",
      "Mentions the @validate_params decorator enforcing data_home must be str, os.PathLike, or None",
      "Mentions clear_data_home calls get_data_home first then shutil.rmtree without recreating the directory"
    ],
    "key_files": [
      "sklearn/datasets/_base.py",
      "sklearn/utils/_param_validation.py",
      "sklearn/datasets/tests/test_base.py"
    ],
    "source_doc": "[docstring: sklearn/datasets/_base.py] sklearn.datasets._base.get_data_home\nReturn the path of the scikit-learn data directory.\n\n    This folder is used by some large dataset loaders to avoid downloading the\n    data several times.\n\n    By default the data directory is set to a folder named 'scikit_learn_data' in the\n    user home folder.\n\n    Alternatively, it can be set by the 'SCIKIT_LEARN_DATA' environment\n    variable or programmatically by giving an explicit folder path. The '~'\n    symbol is expanded to the user home folder.\n\n    If the folder does not already exist, it is automatically created.\n\n    Parameters\n    ----------\n    data_home : str or path-like, default=None\n        The path to scikit-learn data directory. If `None`, the default path\n        is `~/scikit_learn_data`.\n\n    Returns\n    -------\n    data_home: str\n        The path to scikit-learn data directory.\n\n    Examples\n    --------\n    >>> import os\n    >>> from sklearn.datasets import get_data_home\n    >>> data_home_path = get_data_home()\n    >>> os.path.exists(data_home_path)\n    True",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many implementation details (environment variable lookup via environ.get, os.makedirs with exist_ok=True, @validate_params decorator, clear_data_home behavior, RemoteFileMetadata, _fetch_remote, fetch_file, downstream fetchers) that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the given doc chunk.",
      "The claim about expanduser converting PathLike to string is a plausible but unverifiable implementation detail from this documentation alone.",
      "The claim about clear_data_home not recreating the directory after deletion is unverifiable from the provided documentation.",
      "The claim about fetch_california_housing appending '_py3' suffix is a very specific implementation detail that cannot be verified and could be outdated or incorrect.",
      "The claim about fetch_file being 'added in v1.6' is unverifiable from this documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed claims about fetch_openml/fetch_california_housing internals not in provided code, and the 'v1.6' version note.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_47",
    "repo": "scikit_learn",
    "question": "Where is `fetch_covtype` defined and how does it download, parse, cache, and serve the covertype dataset?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `fetch_covtype` function is defined in `sklearn/datasets/_covtype.py` and is publicly exposed via `sklearn/datasets/__init__.py` (`from sklearn.datasets._covtype import fetch_covtype`). Here are the key implementation details:\n\n**Data storage location**: The data is cached in a `covertype` subdirectory under the data home. The samples and targets are stored as separate joblib-pickled files via `_pkl_filepath(covtype_dir, \"samples\")` and `_pkl_filepath(covtype_dir, \"targets\")`.\n\n**Download source**: The archive is downloaded not from UCI directly but from Figshare at `https://ndownloader.figshare.com/files/5976039` with filename `covtype.data.gz` and SHA256 checksum `614360d0257557dd1792834a85a1cdebfadc3c4f30b011d56afee7ffb5b15771`.\n\n**Feature names construction**: The 54 feature names are built from 10 explicit names (Elevation, Aspect, Slope, Horizontal_Distance_To_Hydrology, Vertical_Distance_To_Hydrology, Horizontal_Distance_To_Roadways, Hillshade_9am, Hillshade_Noon, Hillshade_3pm, Horizontal_Distance_To_Fire_Points) plus 4 generated names (`Wilderness_Area_0` through `Wilderness_Area_3`) and 40 generated names (`Soil_Type_0` through `Soil_Type_39`). The target column is named `Cover_Type` (stored in `TARGET_NAMES = [\"Cover_Type\"]`).\n\n**Data parsing**: The raw CSV data inside the gzip file is parsed using `np.genfromtxt(GzipFile(filename=archive_path), delimiter=\",\")`. The last column is the target, extracted via `Xy[:, -1]` and cast to `np.int32` using `copy=False`. Data is compressed with `joblib.dump(X, ..., compress=9)` at the maximum compression level.\n\n**Atomic download pattern**: The function uses a `TemporaryDirectory(dir=covtype_dir)` created as a direct subdirectory of the target `covtype_dir`. Temporary pickle files are written there and then moved to the final location via `os.rename`.\n\n**Shuffle mechanism**: When `shuffle=True`, it generates an index array with `np.arange(X.shape[0])`, uses `check_random_state(random_state)` to create an RNG, shuffles the index, and applies it to both X and y.\n\n**Lazy loading**: If data was already downloaded (both pickle files exist), it loads them with `joblib.load`. The code uses a try/except on `NameError` for `X, y` to decide whether to load from cache\u2014if the variables already exist from a fresh download, loading is skipped.\n\n**`_convert_data_dataframe`**: Located in `sklearn/datasets/_base.py`, when `as_frame=True`, it creates a pandas DataFrame with the feature columns, a separate target DataFrame, concatenates them into a combined frame, and squeezes the target to a Series when there's only one target column (which is the case here with `Cover_Type`).\n\n**`RemoteFileMetadata`**: Defined as a `namedtuple` in `_base.py`: `RemoteFileMetadata = namedtuple(\"RemoteFileMetadata\", [\"filename\", \"url\", \"checksum\"])`.\n\n**`_fetch_remote`**: In `_base.py`, the imports show usage of `urlretrieve`, `URLError`, `NamedTemporaryFile`, and `shutil` suggesting download with retry logic. The full implementation details of `_fetch_remote` require examining the complete function body.\n\n**Test fixture**: In `sklearn/conftest.py`, the `fetch_covtype_fxt` test fixture is created by `_fetch_fixture(fetch_covtype)`, which checks the `SKLEARN_SKIP_NETWORK_TESTS` environment variable to decide whether to allow downloading, and skips the test if data is not available and downloads are disabled.\n\n**Parameter validation**: The `@validate_params` decorator enforces that `n_retries` must be an integer \u2265 1 (using `Interval(Integral, 1, None, closed=\"left\")`) and `delay` must be a float > 0.0 (using `Interval(Real, 0.0, None, closed=\"neither\")`).",
    "rubric": [
      "Defined in sklearn/datasets/_covtype.py and exposed via sklearn/datasets/__init__.py",
      "Downloads from Figshare (https://ndownloader.figshare.com/files/5976039), not UCI directly, with SHA256 checksum verification",
      "Parses the gzip CSV using np.genfromtxt with delimiter=',' and extracts target as the last column cast to np.int32",
      "Caches samples and targets as separate joblib-pickled files in a 'covertype' subdirectory under data_home, compressed at level 9",
      "Uses a TemporaryDirectory inside covtype_dir for atomic writes, then os.rename to final location",
      "Feature names are 10 explicit names plus 4 Wilderness_Area and 40 Soil_Type generated names (54 total), with target named Cover_Type",
      "Shuffle uses check_random_state and shuffles an index array applied to both X and y",
      "Lazy loading via joblib.load when pickle files already exist, using try/except on NameError to decide whether to load from cache",
      "When as_frame=True, uses _convert_data_dataframe to create DataFrame/Series output",
      "@validate_params decorator enforces n_retries >= 1 (Integral) and delay > 0.0 (Real)"
    ],
    "key_files": [
      "sklearn/datasets/_covtype.py",
      "sklearn/datasets/_base.py",
      "sklearn/utils/_bunch.py",
      "sklearn/conftest.py",
      "sklearn/datasets/tests/test_covtype.py"
    ],
    "source_doc": "[docstring: sklearn/datasets/_covtype.py] sklearn.datasets._covtype.fetch_covtype\nLoad the covertype dataset (classification).\n\n    Download it if necessary.\n\n    =================   ============\n    Classes                        7\n    Samples total             581012\n    Dimensionality                54\n    Features                     int\n    =================   ============\n\n    Read more in the :ref:`User Guide <covtype_dataset>`.\n\n    Parameters\n    ----------\n    data_home : str or path-like, default=None\n        Specify another download and cache folder for the datasets. By default\n        all scikit-learn data is stored in '~/scikit_learn_data' subfolders.\n\n    download_if_missing : bool, default=True\n        If False, raise an OSError if the data is not locally available\n        instead of trying to download the data from the source site.\n\n    random_state : int, RandomState instance or None, default=None\n        Determines random number generation for dataset shuffling. Pass an int\n        for reproducible output across multiple function calls.\n        See :term:`Glossary <random_state>`.\n\n    shuffle : bool, default=False\n        Whether to shuffle dataset.\n\n    return_X_y : bool, default=False\n        If True, returns ``(data.data, data.target)`` instead of a Bunch\n        object.\n\n        .. versionadded:: 0.20\n\n    as_frame : bool, default=False\n        If True, the data is a pandas DataFrame including columns with\n        appropriate dtypes (numeric). The target is a pandas DataFrame or\n        Series depending on the number of target columns. If `return_X_y` is\n        True, then (`data`, `target`) will be pandas DataFrames or Series as\n        described below.\n\n        .. versionadded:: 0.24\n\n    n_retries : int, default=3\n        Number of retries when HTTP errors are encountered.\n\n        .. versionadded:: 1.5\n\n    delay : float, default=1.0\n        Number of seconds between retries.\n\n        .. versionadded:: 1.5\n\n    Returns\n    -------\n    dataset : :class:`~sklearn.utils.Bunch`\n        Dictionary-like object, with the following attributes.\n\n        data : ndarray of shape (581012, 54)\n            Each row corresponds to the 54 features in the dataset.\n        target : ndarray of shape (581012,)\n            Each value corresponds to one of\n            the 7 forest covertypes with values\n            ranging between 1 to 7.\n        frame : dataframe of shape (581012, 55)\n            Only present when `as_frame=True`. Contains `data` and `target`.\n        DESCR : str\n            Description of the forest covertype dataset.\n        feature_names : list\n            The names of the dataset columns.\n        target_names: list\n            The names of the target columns.\n\n    (data, target) : tuple if ``return_X_y`` is True\n        A tuple of two ndarray. The first containing a 2D array of\n        shape (n_samples, n_features) with each row representing one\n        sample and each column representing the features. The second\n        ndarray of shape (n_samples,) containing the target samples.\n\n        .. versionadded:: 0.20\n\n    Examples\n    --------\n    >>> from sklearn.datasets import fetch_covtype\n    >>> cov_type = fetch_covtype()\n    >>> cov_type.data.shape\n    (581012, 54)\n    >>> cov_type.target.shape\n    (581012,)\n    >>> # Let's check the 4 first feature names\n    >>> cov_type.feature_names[:4]\n    ['Elevation', 'Aspect', 'Slope', 'Horizontal_Distance_To_Hydrology']",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor issues with specific line numbers, version references, and design rationale comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_46",
    "repo": "scikit_learn",
    "question": "Why does the California housing fetcher cache raw column totals instead of the derived per-household averages it actually returns?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `fetch_california_housing` function employs several performance and reliability design decisions:\n\n**1. Two-level caching with joblib compression (performance optimization):**\nAfter downloading the raw tarball (`cal_housing.tgz`), the function parses the CSV data with `np.loadtxt`, reorders the columns, and then serializes the resulting numpy array using `joblib.dump(cal_housing, filepath, compress=6)`. The raw archive is immediately deleted with `remove(archive_path)`. On subsequent calls, the pre-parsed data is loaded directly via `joblib.load(filepath)`, skipping download, tarball extraction, and CSV parsing entirely. The compression level of 6 is used.\n\n**2. The cached filename uses `_pkl_filepath`:**\nThe filepath is determined by `_pkl_filepath(data_home, 'cal_housing.pkz')`.\n\n**3. Column reordering before caching (data preprocessing optimization):**\nThe raw data file columns are reordered using `columns_index = [8, 7, 2, 3, 4, 5, 6, 1, 0]` before being cached. This reordering happens only once during download, not on every load. The first column (index 0 after reorder) becomes the target (median house value), and columns 1-8 become the features.\n\n**4. Derived features computed on every load, not cached:**\nAverage rooms (`data[:, 2] /= data[:, 5]`), average bedrooms (`data[:, 3] /= data[:, 5]`), and average occupancy (`data[:, 5] = data[:, 4] / data[:, 5]`) are all computed by dividing by the households column every time the function is called \u2014 they are NOT part of the cached pickle. This means the cached data stores raw totals, and derived averages are computed fresh on each call. The order of operations matters: AveRooms and AveBedrms are computed first using the original households value in column 5, then column 5 itself is overwritten with AveOccup (population / households).\n\n**5. Parallel-safe download mechanism:**\nThe `_fetch_remote` function is used for downloading. Based on the imports in `_base.py`, `NamedTemporaryFile` and `shutil.move` are available, suggesting a temp-file-based atomic download approach.\n\n**6. SHA256 checksum:**\nThe expected checksum is `aaa5c9a6afe2225cc2aed2723682ae403280c4a3695a2ddda4ffb5d8215ea681`, defined in the `ARCHIVE` RemoteFileMetadata.\n\n**7. Retry with delay for network resilience:**\nThe `n_retries` and `delay` parameters are passed to `_fetch_remote`. Based on imports of `URLError` and `time` in `_base.py`, retry logic with delays is implemented. The `n_retries` parameter is validated to be \u2265 1 (via `Interval(Integral, 1, None, closed='left')`) and `delay` must be strictly positive (via `Interval(Real, 0.0, None, closed='neither')`).\n\n**8. Lazy pandas import:**\n`_convert_data_dataframe` calls `check_pandas_support()` only when `as_frame=True`, avoiding importing pandas unless needed. The target column name in the DataFrame is `'MedHouseVal'`.",
    "rubric": [
      "Explains that the cached pickle (via joblib.dump with compress=6) stores the column-reordered raw data, not the derived averages",
      "Notes that derived features (AveRooms, AveBedrms, AveOccup) are computed fresh on every call by dividing by the households column",
      "Mentions the column reordering (columns_index = [8, 7, 2, 3, 4, 5, 6, 1, 0]) happens once at download time before caching",
      "Explains the order-of-operations dependency: AveRooms and AveBedrms must be computed before AveOccup overwrites column 5 (households)",
      "Mentions the two-level caching strategy: raw tarball is downloaded, parsed with np.loadtxt, serialized with joblib.dump(compress=6), and the archive is deleted",
      "Notes that subsequent calls load from the .pkz file via joblib.load, skipping download, extraction, and CSV parsing"
    ],
    "key_files": [
      "sklearn/datasets/_california_housing.py",
      "sklearn/datasets/_base.py"
    ],
    "source_doc": "[docstring: sklearn/datasets/_california_housing.py] sklearn.datasets._california_housing.fetch_california_housing\nLoad the California housing dataset (regression).\n\n    ==============   ==============\n    Samples total             20640\n    Dimensionality                8\n    Features                   real\n    Target           real 0.15 - 5.\n    ==============   ==============\n\n    Read more in the :ref:`User Guide <california_housing_dataset>`.\n\n    Parameters\n    ----------\n    data_home : str or path-like, default=None\n        Specify another download and cache folder for the datasets. By default\n        all scikit-learn data is stored in '~/scikit_learn_data' subfolders.\n\n    download_if_missing : bool, default=True\n        If False, raise an OSError if the data is not locally available\n        instead of trying to download the data from the source site.\n\n    return_X_y : bool, default=False\n        If True, returns ``(data.data, data.target)`` instead of a Bunch\n        object.\n\n        .. versionadded:: 0.20\n\n    as_frame : bool, default=False\n        If True, the data is a pandas DataFrame including columns with\n        appropriate dtypes (numeric, string or categorical). The target is\n        a pandas DataFrame or Series depending on the number of target_columns.\n\n        .. versionadded:: 0.23\n\n    n_retries : int, default=3\n        Number of retries when HTTP errors are encountered.\n\n        .. versionadded:: 1.5\n\n    delay : float, default=1.0\n        Number of seconds between retries.\n\n        .. versionadded:: 1.5\n\n    Returns\n    -------\n    dataset : :class:`~sklearn.utils.Bunch`\n        Dictionary-like object, with the following attributes.\n\n        data : ndarray, shape (20640, 8)\n            Each row corresponding to the 8 feature values in order.\n            If ``as_frame`` is True, ``data`` is a pandas object.\n        target : numpy array of shape (20640,)\n            Each value corresponds to the median\n            house value in units of 100,000.\n            If ``as_frame`` is True, ``target`` is a pandas object.\n        feature_names : list of length 8\n            Array of ordered feature names used in the dataset.\n        DESCR : str\n            Description of the California housing dataset.\n        frame : pandas DataFrame\n            Only present when `as_frame=True`. DataFrame with ``data`` and\n            ``target``.\n\n            .. versionadded:: 0.23\n\n    (data, target) : tuple if ``return_X_y`` is True\n        A tuple of two ndarray. The first containing a 2D array of\n        shape (n_samples, n_features) with each row representing one\n        sample and each column representing the features. The second\n        ndarray of shape (n_samples,) containing the target samples.\n\n        .. versionadded:: 0.20\n\n    Notes\n    -----\n\n    This dataset consists of 20,640 samples and 9 features.\n\n    Examples\n    --------\n    >>> from sklearn.datasets import fetch_california_housing\n    >>> housing = fetch_california_housing()\n    >>> print(housing.data.shape, housing.target.shape)\n    (20640, 8) (20640,)\n    >>> print(housing.feature_names[0:6])\n    ['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup']",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (joblib compression levels, column reordering indices, atomic temp file rename, SHA256 checksum values, line numbers in source files, etc.) that are not present in the provided documentation. These appear to be derived from reading the actual source code rather than the docstring provided.",
      "The claim about n_retries being validated to be \u2265 1 via 'Interval(Integral, 1, None, closed=\"left\")' cannot be verified from the documentation, which only states 'n_retries : int, default=3'.",
      "The claim about delay being strictly positive via 'Interval(Real, 0.0, None, closed=\"neither\")' cannot be verified from the documentation, which only states 'delay : float, default=1.0'.",
      "The documentation says 'This dataset consists of 20,640 samples and 9 features' in the Notes section, while the answer's claim about column reordering producing 1 target + 8 features is consistent with this but the specific reordering indices cannot be verified from the doc.",
      "The target column name 'MedHouseVal' mentioned in the answer is not stated in the documentation.",
      "The SHA256 checksum value mentioned is not in the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code, but some rely on truncated function bodies or stripped comments about design rationale.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "scikit_learn_gen_48",
    "repo": "scikit_learn",
    "question": "How does `fetch_lfw_people` download, cache, load, and process images from raw files into the final returned dataset?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `fetch_lfw_people` function in `sklearn/datasets/_lfw.py` implements a multi-step pipeline for loading the LFW dataset:\n\n1. **Download & Cache**: `_check_fetch_lfw()` creates a `lfw_home` directory under the scikit-learn data home (default `~/scikit_learn_data/lfw_home`). It downloads 3 metadata files (pairsDevTrain.txt, pairsDevTest.txt, pairs.txt) from figshare.com URLs, plus either the funneled or non-funneled archive. After extraction, the archive .tgz file is deleted via `os.remove(archive_path)`. The funneled variant stores data in `lfw_funneled/` subfolder, while the non-funneled uses `lfw/`.\n\n2. **Memoization with joblib**: The function uses `Memory(location=lfw_home, compress=6, verbose=0)` to cache the internal `_fetch_lfw_people` function.\n\n3. **Internal loading (`_fetch_lfw_people`)**: It scans the data folder sorted by person name directories (using `sorted(listdir(data_folder_path))`), filters by `min_faces_per_person`, replaces underscores with spaces in names, and uses `np.searchsorted(target_names, person_names)` to create integer target labels. The results are shuffled with `np.random.RandomState(42).shuffle(indices)`.\n\n4. **Image loading (`_load_imgs`)**: Requires PIL/Pillow. Uses `Image.open()` then `pil_img.crop()` using the slice parameter (swapping height/width: `crop(w_slice.start, h_slice.start, w_slice.stop, h_slice.stop)`). If resize is not None, calls `pil_img.resize((w, h))`. Pixel values are divided by 255.0 to scale to [0.0, 1.0] floats. Grayscale conversion is done by averaging across axis=2 (`face.mean(axis=2)`). A default_slice of `(slice(0, 250), slice(0, 250))` is used when slice_ is None. Progress is logged every 1000 faces.\n\n5. **Parameter validation**: Uses `@validate_params` decorator. The `min_faces_per_person` parameter's actual default is `0` in the function signature. The `slice_` parameter accepts `tuple` or `Hidden(None)`. The `resize` parameter is validated as `Interval(Real, 0, None, closed='neither')` meaning it must be strictly positive.\n\n6. **Return value**: `faces.reshape(len(faces), -1)` flattens images to 2D for the `data` attribute, while `images` retains the original shape. The DESCR is loaded from `sklearn/datasets/descr/lfw.rst` using `importlib.resources`.\n\n7. **Remote files**: Both archives and metadata files are `RemoteFileMetadata` namedtuples (defined in `_base.py` as `namedtuple('RemoteFileMetadata', ['filename', 'url', 'checksum'])`). The `checksum` field is present, and `_base.py` imports `URLError`, `TimeoutError`, `NamedTemporaryFile`, and `shutil`, suggesting checksum verification, retry support, and atomic file writing via temporary files.\n\n8. **Tarfile extraction**: Uses `tarfile_extractall` from `sklearn.utils.fixes`, which tries to use `filter='data'` and falls back to plain `extractall` on `TypeError`.",
    "rubric": [
      "Explains that _check_fetch_lfw creates a lfw_home directory and downloads metadata files (pairsDevTrain.txt, pairsDevTest.txt, pairs.txt) plus either the funneled or non-funneled archive, then deletes the .tgz after extraction",
      "Mentions that tarfile extraction uses tarfile_extractall which tries filter='data' and falls back to plain extractall on TypeError",
      "Describes that joblib Memory with compress=6 is used to memoize/cache the internal _fetch_lfw_people function",
      "Explains the internal loading: scans sorted person directories, filters by min_faces_per_person, replaces underscores with spaces in names, uses np.searchsorted for target labels, and shuffles with RandomState(42)",
      "Describes image loading via PIL: crops using the slice parameter, optionally resizes, divides pixel values by 255.0, and converts to grayscale by averaging across axis=2 when color is False",
      "Mentions that faces are reshaped to 2D (flattened) for the data attribute while images retains the original shape in the returned Bunch"
    ],
    "key_files": [
      "sklearn/datasets/_lfw.py",
      "sklearn/datasets/_base.py",
      "sklearn/utils/fixes.py",
      "sklearn/datasets/tests/test_lfw.py"
    ],
    "source_doc": "[docstring: sklearn/datasets/_lfw.py] sklearn.datasets._lfw.fetch_lfw_people\nLoad the Labeled Faces in the Wild (LFW) people dataset (classification).\n\n    Download it if necessary.\n\n    =================   =======================\n    Classes                                5749\n    Samples total                         13233\n    Dimensionality                         5828\n    Features            real, between 0 and 255\n    =================   =======================\n\n    For a usage example of this dataset, see\n    :ref:`sphx_glr_auto_examples_applications_plot_face_recognition.py`.\n\n    Read more in the :ref:`User Guide <labeled_faces_in_the_wild_dataset>`.\n\n    Parameters\n    ----------\n    data_home : str or path-like, default=None\n        Specify another download and cache folder for the datasets. By default\n        all scikit-learn data is stored in '~/scikit_learn_data' subfolders.\n\n    funneled : bool, default=True\n        Download and use the funneled variant of the dataset.\n\n    resize : float or None, default=0.5\n        Ratio used to resize the each face picture. If `None`, no resizing is\n        performed.\n\n    min_faces_per_person : int, default=None\n        The extracted dataset will only retain pictures of people that have at\n        least `min_faces_per_person` different pictures.\n\n    color : bool, default=False\n        Keep the 3 RGB channels instead of averaging them to a single\n        gray level channel. If color is True the shape of the data has\n        one more dimension than the shape with color = False.\n\n    slice_ : tuple of slice, default=(slice(70, 195), slice(78, 172))\n        Provide a custom 2D slice (height, width) to extract the\n        'interesting' part of the jpeg files and avoid use statistical\n        correlation from the background.\n\n    download_if_missing : bool, default=True\n        If False, raise an OSError if the data is not locally available\n        instead of trying to download the data from the source site.\n\n    return_X_y : bool, default=False\n        If True, returns ``(dataset.data, dataset.target)`` instead of a Bunch\n        object. See below for more information about the `dataset.data` and\n        `dataset.target` object.\n\n        .. versionadded:: 0.20\n\n    n_retries : int, default=3\n        Number of retries when HTTP errors are encountered.\n\n        .. versionadded:: 1.5\n\n    delay : float, default=1.0\n        Number of seconds between retries.\n\n        .. versionadded:: 1.5\n\n    Returns\n    -------\n    dataset : :class:`~sklearn.utils.Bunch`\n        Dictionary-like object, with the following attributes.\n\n        data : numpy array of shape (13233, 2914)\n            Each row corresponds to a ravelled face image\n            of original size 62 x 47 pixels.\n            Changing the ``slice_`` or resize parameters will change the\n            shape of the output.\n        images : numpy array of shape (13233, 62, 47)\n            Each row is a face image corresponding to one of the 5749 people in\n            the dataset. Changing the ``slice_``\n            or resize parameters will change the shape of the output.\n        target : numpy array of shape (13233,)\n            Labels associated to each face image.\n            Those labels range from 0-5748 and correspond to the person IDs.\n        target_names : numpy array of shape (5749,)\n            Names of all persons in the dataset.\n            Position in array corresponds to the person ID in the target array.\n        DESCR : str\n            Description of the Labeled Faces in the Wild (LFW) dataset.\n\n    (data, target) : tuple if ``return_X_y`` is True\n        A tuple of two ndarray. The first containing a 2D array of\n        shape (n_samples, n_features) with each row representing one\n        sample and each column representing the features. The second\n        ndarray of shape (n_samples,) containing the target samples.\n\n        .. versionadded:: 0.20\n\n    Examples\n    --------\n    >>> from sklearn.datasets import fetch_lfw_people\n    >>> lfw_people = fetch_lfw_people()\n    >>> lfw_people.data.shape\n    (13233, 2914)\n    >>> lfw_people.target.shape\n    (13233,)\n    >>> for name in lfw_people.target_names[:5]:\n    ...    print(name)\n    AJ Cook\n    AJ Lamas\n    Aaron Eckhart\n    Aaron Guiel\n    Aaron Patterson",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states that pixel values are 'divided by 255.0 to scale to [0.0, 1.0] floats', but the documentation says 'Features: real, between 0 and 255', which contradicts this claim about scaling to [0,1].",
      "The answer claims 'min_faces_per_person parameter's actual default is 0 in the function signature (not None as the docstring says)' - the documentation clearly states 'default=None', and while this might be true at the code level, it contradicts the documented interface.",
      "Many implementation details (joblib memoization with compress=6, RandomState(42) shuffling, crop parameter ordering, progress logging every 1000 faces, default_slice of (0,250), tarfile_extractall with filter='data', etc.) cannot be verified from the provided documentation and may be hallucinations or accurate code details."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in rationale for shuffling (IID/SGD comment), docstring comparison for min_faces_per_person default, and 'security (Python 3.12+)' context for tarfile filtering.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_02",
    "repo": "matplotlib",
    "question": "Why does the PGF backend's PdfPages class accumulate figures into a BytesIO buffer and defer PDF generation to close time, rather than writing PDF incrementally like backend_pdf's PdfPages?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The PGF backend's `PdfPages` class uses a fundamentally different multi-page PDF strategy from the standard `backend_pdf.PdfPages`, and several design decisions reveal why:\n\n1. **Deferred LaTeX compilation (all-at-once approach):** Unlike `backend_pdf.PdfPages` which writes PDF directly and incrementally (wrapping a `PdfFile` object), the PGF `PdfPages` accumulates all figure PGF data into a single in-memory `BytesIO` buffer (`self._file`). LaTeX is only invoked once at `close()` time via `_run_latex()`. The PGF backend relies on an external LaTeX engine to produce PDF \u2014 it can't write PDF directly, as evidenced by the subprocess calls to the configured texsystem.\n\n2. **Lazy header writing keyed to first figure's dimensions:** The `_write_header()` is called only when the first figure is saved (`if self._n_figures == 0`). The paper dimensions in the `\\usepackage[geometry]` are taken from the first figure's `get_size_inches()`. Subsequent pages with different sizes use TeX-level dimension commands (`\\pdfpagewidth`/`\\pdfpageheight` or their LuaTeX equivalents `\\pagewidth`/`\\pageheight`) via `\\ifdefined` conditionals.\n\n3. **Metadata handled via hyperref LaTeX package:** PDF metadata is injected into the LaTeX document via `\\usepackage[pdfinfo={...}]{hyperref}` rather than being written directly into PDF info dict structures. The `_metadata_to_str` helper converts metadata key-value pairs to a format hyperref accepts (e.g., converting `datetime` objects via `_datetime_to_pdf` from `backend_pdf`).\n\n4. **Reuse of `_create_pdf_info_dict` from backend_pdf:** The PGF PdfPages imports and calls `_create_pdf_info_dict('pgf', self._metadata)` from `backend_pdf`, passing `'pgf'` as the backend name. This shared function sets default `Creator` to `f'Matplotlib v{mpl.__version__}'`, `Producer` to `f'Matplotlib pgf backend v{mpl.__version__}'`, and `CreationDate` to either the current time or the time from the `SOURCE_DATE_EPOCH` environment variable.\n\n5. **No empty PDF protection:** In `close()`, if `self._n_figures == 0`, the method skips calling `_run_latex()` entirely (`if self._n_figures > 0`). This avoids running LaTeX on a document with no content. However, `\\end{document}\\n` is always written to the buffer regardless.\n\n6. **Potential bug in close():** The code writes `rb'\\end{document}\\n'` \u2014 because this is a raw byte string, the `\\n` is a literal backslash-n (a LaTeX `\\n` command), not a newline character. This contrasts with the single-page `print_pdf` method which uses `r\"\\end{document}\"` correctly in a `\"\\n\".join([...])` context.\n\n7. **TemporaryDirectory for compilation:** The `_run_latex()` method writes the accumulated BytesIO content to `pdf_pages.tex` in a `TemporaryDirectory`, runs the LaTeX command with flags `-interaction=nonstopmode -halt-on-error -no-shell-escape`, then moves the resulting PDF to the final output filename via `shutil.move`.\n\n8. **Different from backend_pdf PdfPages which uses lazy file opening:** The `backend_pdf.PdfPages` has an `_ensure_file()` pattern that lazily creates the `PdfFile` object on first use, while the PGF version creates its `BytesIO` buffer immediately in `__init__` but defers the LaTeX header until the first `savefig` call.",
    "rubric": [
      "Explains that PGF cannot write PDF directly \u2014 it relies on an external LaTeX engine (subprocess call to texsystem) to compile PGF/TeX into PDF",
      "Describes the deferred/all-at-once approach: all figure PGF data is accumulated into a BytesIO buffer, then LaTeX is invoked once at close() via _run_latex()",
      "Mentions that the header is written lazily on the first figure (when _n_figures == 0), using that figure's dimensions for geometry papersize",
      "Explains how subsequent pages handle different sizes using TeX-level commands (\\pdfpagewidth/\\pdfpageheight or \\pagewidth/\\pageheight via \\ifdefined conditionals)",
      "Notes that metadata is injected via \\usepackage[pdfinfo={...}]{hyperref} rather than written directly into PDF structures",
      "Mentions _run_latex() writes BytesIO content to a temp file, runs LaTeX with nonstopmode/halt-on-error/no-shell-escape flags, then moves the resulting PDF to the output location",
      "Notes that if no figures were added (n_figures == 0), _run_latex() is skipped entirely to avoid running LaTeX on an empty document"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_pgf.py",
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/cbook.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_pgf.py] lib.matplotlib.backends.backend_pgf.PdfPages\nlib.matplotlib.backends.backend_pgf.PdfPages:\n  A multi-page PDF file using the pgf backend\n\n    Examples\n    --------\n    >>> import matplotlib.pyplot as plt\n    >>> # Initialize:\n    >>> with PdfPages('foo.pdf') as pdf:\n    ...     # As many times as you like, create a figure fig and save it:\n    ...     fig = plt.figure()\n    ...     pdf.savefig(fig)\n    ...     # When no figure is specified the current figure is saved\n    ...     pdf.savefig()\n\nlib.matplotlib.backends.backend_pgf.PdfPages.__init__:\n  Create a new PdfPages object.\n\n        Parameters\n        ----------\n        filename : str or path-like\n            Plots using `PdfPages.savefig` will be written to a file at this\n            location. Any older file with the same name is overwritten.\n\n        metadata : dict, optional\n            Information dictionary object (see PDF reference section 10.2.1\n            'Document Information Dictionary'), e.g.:\n            ``{'Creator': 'My software', 'Author': 'Me', 'Title': 'Awesome'}``.\n\n            The standard keys are 'Title', 'Author', 'Subject', 'Keywords',\n            'Creator', 'Producer', 'CreationDate', 'ModDate', and\n            'Trapped'. Values have been predefined for 'Creator', 'Producer'\n            and 'CreationDate'. They can be removed by setting them to `None`.\n\n            Note that some versions of LaTeX engines may ignore the 'Producer'\n            key and set it to themselves.\n\nlib.matplotlib.backends.backend_pgf.PdfPages.close:\n  Finalize this object, running LaTeX in a temporary directory\n        and moving the final pdf file to *filename*.\n\nlib.matplotlib.backends.backend_pgf.PdfPages.savefig:\n  Save a `.Figure` to this file as a new page.\n\n        Any other keyword arguments are passed to `~.Figure.savefig`.\n\n        Parameters\n        ----------\n        figure : `.Figure` or int, default: the active figure\n            The figure, or index of the figure, that is saved to the file.\n\nlib.matplotlib.backends.backend_pgf.PdfPages.get_pagecount:\n  Return the current number of pages in the multipage pdf file.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most of the claims in the answer go far beyond what is documented in the provided documentation chunk. The documentation only describes the public API: class purpose, __init__ parameters (filename, metadata), close(), savefig(), and get_pagecount(). The answer makes extensive claims about internal implementation details (BytesIO buffers, _write_header(), _run_latex(), line numbers, raw byte string bugs, TemporaryDirectory usage, hyperref package usage, etc.) that cannot be verified from the provided documentation alone.",
      "However, the claims that ARE verifiable against the documentation appear consistent: the class is described as 'A multi-page PDF file using the pgf backend', metadata handling matches the documented parameters, the warning about LaTeX engines ignoring 'Producer' is documented, and the predefined values for 'Creator', 'Producer', and 'CreationDate' are mentioned in the docs.",
      "Claim about 'potential bug in close()' with rb'\\end{document}\\n' is an implementation-level claim that cannot be verified from the docs and could be a hallucination or could be accurate - impossible to tell from the provided documentation.",
      "The answer's description of metadata standard keys and their defaults is consistent with the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code structure; a few design rationales and docstring references were removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_04",
    "repo": "matplotlib",
    "question": "How does AnchoredAuxTransformBox use AuxTransformBox internally to position and draw child artists with a custom transform?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `AnchoredAuxTransformBox` class works by composing two key components: an `AuxTransformBox` (as the `drawing_area`) and inheriting from `AnchoredOffsetbox` for positioning within the axes.\n\n**How `AnchoredAuxTransformBox` works internally:**\n\n1. **Initialization**: The `__init__` creates an `AuxTransformBox` instance with the given transform and stores it as `self.drawing_area`. It then calls `AnchoredOffsetbox.__init__` with this as the `child` parameter, passing along `prop=prop` (notably, unlike `AnchoredDrawingArea` which passes `prop=None`).\n\n2. **Transform chain in `AuxTransformBox`**: When artists are added to the `drawing_area`, `AuxTransformBox.add_artist` unconditionally sets their transform to a composite of three transforms chained together: `aux_transform + ref_offset_transform + offset_transform`. The `aux_transform` is the user-provided transform, `ref_offset_transform` is an `Affine2D` that translates the children so their bounding box starts at (0,0), and `offset_transform` is an `Affine2D` that positions the box via `set_offset`.\n\n3. **Dynamic bounding box computation**: Unlike `DrawingArea` which has fixed width/height, `AuxTransformBox.get_bbox(renderer)` dynamically computes its size by: (a) saving the offset_transform matrix, then clearing `ref_offset_transform` and `offset_transform`, (b) computing `get_window_extent` for each child artist, (c) taking the `Bbox.union` of all children's bboxes, (d) setting `ref_offset_transform` to translate by `(-ub.x0, -ub.y0)`, (e) restoring the offset transform from the saved matrix, (f) returning `Bbox.from_bounds(0, 0, ub.width, ub.height)`.\n\n4. **Positioning via `AnchoredOffsetbox`**: The parent's `get_bbox` adds padding by calling `self.get_child().get_bbox(renderer).padded(pad)` where `pad = self.pad * fontsize`. The `get_offset` method uses `_get_anchored_bbox` which maps location codes to anchor positions using `bbox.anchored(c, container=container).p0`.\n\n5. **Drawing**: `AnchoredOffsetbox.draw` draws the frame patch (a `FancyBboxPatch`), then sets the child's offset and calls draw on the child. The `AuxTransformBox.draw` simply iterates over `self._children` and calls `c.draw(renderer)` on each.\n\n**Key design differences from `AnchoredDrawingArea`:**\n- `AuxTransformBox.add_artist` always overrides the artist's transform (`a.set_transform(self.get_transform())`), while `DrawingArea.add_artist` only sets it if not already set (`if not a.is_transform_set()`).\n- `DrawingArea.add_artist` also sets `a.axes` and `a.set_figure(fig)` on added artists; `AuxTransformBox.add_artist` does not.\n- `AuxTransformBox.set_transform` is a no-op (`pass`), so external code cannot change its transform.\n- The `AnchoredOffsetbox` has a `zorder = 5` class attribute.\n- The `_bbox_artist` debug drawing function only runs when `DEBUG = True` (the module-level flag which defaults to `False`), making it a no-op in normal operation.",
    "rubric": [
      "Explains that AnchoredAuxTransformBox creates an AuxTransformBox as self.drawing_area and passes it as the child to AnchoredOffsetbox.__init__",
      "Describes the three-part transform chain in AuxTransformBox.get_transform(): aux_transform + ref_offset_transform + offset_transform",
      "Explains that AuxTransformBox.add_artist unconditionally sets each artist's transform via set_transform(self.get_transform()), unlike DrawingArea which checks is_transform_set() first",
      "Describes how AuxTransformBox.get_bbox dynamically computes bounding box by clearing transforms, computing union of children's window extents, then setting ref_offset_transform to translate by (-ub.x0, -ub.y0)",
      "Explains that AnchoredOffsetbox.draw draws the frame patch, computes the offset via _get_anchored_bbox using the loc code, sets the child's offset, and calls draw on the child",
      "Notes that AuxTransformBox.set_transform is a no-op (pass), preventing external code from overriding the transform"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/anchored_artists.py",
      "lib/matplotlib/offsetbox.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/anchored_artists.py] lib.mpl_toolkits.axes_grid1.anchored_artists.AnchoredAuxTransformBox\nlib.mpl_toolkits.axes_grid1.anchored_artists.AnchoredAuxTransformBox.__init__:\n  An anchored container with transformed coordinates.\n\n        Artists added to the *drawing_area* are scaled according to the\n        coordinates of the transformation used. The dimensions of this artist\n        will scale to contain the artists added.\n\n        Parameters\n        ----------\n        transform : `~matplotlib.transforms.Transform`\n            The transformation object for the coordinate system in use, i.e.,\n            :attr:`!matplotlib.axes.Axes.transData`.\n        loc : str\n            Location of this artist.  Valid locations are\n            'upper left', 'upper center', 'upper right',\n            'center left', 'center', 'center right',\n            'lower left', 'lower center', 'lower right'.\n            For backward compatibility, numeric values are accepted as well.\n            See the parameter *loc* of `.Legend` for details.\n        pad : float, default: 0.4\n            Padding around the child objects, in fraction of the font size.\n        borderpad : float, default: 0.5\n            Border padding, in fraction of the font size.\n        prop : `~matplotlib.font_manager.FontProperties`, optional\n            Font property used as a reference for paddings.\n        frameon : bool, default: True\n            If True, draw a box around this artist.\n        **kwargs\n            Keyword arguments forwarded to `.AnchoredOffsetbox`.\n\n        Attributes\n        ----------\n        drawing_area : `~matplotlib.offsetbox.AuxTransformBox`\n            A container for artists to display.\n\n        Examples\n        --------\n        To display an ellipse in the upper left, with a width of 0.1 and\n        height of 0.4 in data coordinates:\n\n        >>> box = AnchoredAuxTransformBox(ax.transData, loc='upper left')\n        >>> el = Ellipse((0, 0), width=0.1, height=0.4, angle=30)\n        >>> box.drawing_area.add_artist(el)\n        >>> ax.add_artist(box)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (transform chains, bounding box computation, drawing pipeline, etc.) that are not present in the provided documentation. These appear to be based on source code knowledge rather than the given doc chunk.",
      "The claim about `_bbox_artist` only running when `DEBUG = False` seems logically inverted - typically debug drawing would run when DEBUG is True, not False. This could be a hallucination or error.",
      "The claim that `AnchoredAuxTransformBox.__init__` passes `prop=prop` while `AnchoredDrawingArea` passes `prop=None` is not verifiable from the provided documentation.",
      "The claim about `AuxTransformBox.set_transform` being a no-op is not in the documentation and could be a hallucination.",
      "The documentation states `pad` default is 0.4, `borderpad` default is 0.5, and the answer doesn't contradict these values.",
      "The documentation states `drawing_area` is of type `AuxTransformBox`, which the answer correctly states.",
      "The documentation states the class creates 'An anchored container with transformed coordinates' and that 'Artists added to the drawing_area are scaled according to the coordinates of the transformation used' and 'The dimensions of this artist will scale to contain the artists added' - the answer's claims about dynamic bounding box computation are consistent with this."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for purpose descriptions and correcting the DEBUG flag logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_03",
    "repo": "matplotlib",
    "question": "Where does data flow when an AnchoredDrawingArea is rendered, from construction through positioning to drawing its children?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "When `AnchoredDrawingArea` is used, the data flows through the following path:\n\n1. **Construction**: `AnchoredDrawingArea.__init__` creates a `DrawingArea` instance (stored as both `self.da` and `self.drawing_area`) and passes it as the `child` argument to `AnchoredOffsetbox.__init__`. Notably, the `prop` parameter accepted by `AnchoredDrawingArea` is **silently discarded** \u2014 the code hardcodes `prop=None` in the `super().__init__()` call, regardless of what the user passes.\n\n2. **Positioning (AnchoredOffsetbox.get_offset)**: When rendering, `AnchoredOffsetbox.draw()` calls `get_offset()` which computes the border padding using `self.borderpad * fontsize_in_pixels` (where fontsize comes from `self.prop`, which is always None since prop is hardcoded). It calls `_get_anchored_bbox()` which maps loc codes to cardinal directions via a list: `[None, 'NE', 'NW', 'SW', 'SE', 'E', 'W', 'E', 'S', 'N', 'C']`, then calls `bbox.anchored(c, container)` on `matplotlib.transforms.BboxBase` which uses coefficient pairs (e.g., 'NE' \u2192 (1.0, 1.0), 'C' \u2192 (0.5, 0.5)).\n\n3. **Child offset propagation**: `AnchoredOffsetbox.draw()` calls `self.get_child().set_offset((px, py))` which, for `DrawingArea`, updates `self.offset_transform` via `offset_transform.clear()` then `offset_transform.translate(xy[0], xy[1])`.\n\n4. **DrawingArea.draw()**: Computes `dpi_cor = renderer.points_to_pixels(1.)`, scales `self.dpi_transform` by this value. The effective transform for children is `self.dpi_transform + self.offset_transform`. If `_clip_children` is True and a child has no existing clip, a rectangular clip path is created.\n\n5. **Artist transform handling**: `DrawingArea.add_artist()` only sets the transform on added artists if `a.is_transform_set()` returns False. If the artist already has a custom transform set, it's NOT overridden. The `DrawingArea.set_transform()` method is explicitly a no-op (empty body).\n\n6. **Frame drawing**: The `AnchoredOffsetbox` creates a `FancyBboxPatch` with `boxstyle='square,pad=0'`, facecolor='w', edgecolor='k', whose visibility is set to the `frameon` parameter.\n\n7. **Default clip=False**: `AnchoredDrawingArea` creates its `DrawingArea` without passing `clip`, which defaults to `False`, meaning children are NOT clipped to the drawing area bounds by default.\n\n8. **zorder**: `AnchoredOffsetbox` has a class-level `zorder = 5`.\n\n9. **Anchoring to parent Axes**: If no `bbox_to_anchor` is set, `get_bbox_to_anchor()` returns `self.axes.bbox`, anchoring the box relative to the parent Axes' bounding box in display coordinates.",
    "rubric": [
      "Explains that AnchoredDrawingArea.__init__ creates a DrawingArea stored as self.da/self.drawing_area and passes it as child to AnchoredOffsetbox.__init__",
      "Notes that the prop parameter is silently discarded \u2014 hardcoded as prop=None in the super().__init__() call regardless of user input",
      "Describes AnchoredOffsetbox.get_offset computing border padding using self.borderpad * fontsize_in_pixels derived from self.prop",
      "Mentions _get_anchored_bbox mapping loc codes to cardinal direction strings (e.g., 'NE', 'NW') via a list, then calling bbox.anchored(c, container)",
      "Explains that AnchoredOffsetbox.draw() calls self.get_child().set_offset((px, py)), which in DrawingArea updates offset_transform via clear() and translate()",
      "Describes DrawingArea.draw() computing dpi_cor via renderer.points_to_pixels(1.), scaling dpi_transform, with effective child transform being dpi_transform + offset_transform",
      "Notes that DrawingArea.add_artist() only sets the transform on added artists if is_transform_set() returns False (preserving custom transforms)",
      "Mentions DrawingArea.set_transform() is explicitly a no-op",
      "Notes AnchoredOffsetbox creates a FancyBboxPatch with boxstyle='square,pad=0', facecolor='w', edgecolor='k', visibility set by frameon",
      "States that clip defaults to False in DrawingArea, so children are NOT clipped by default",
      "Mentions AnchoredOffsetbox has class-level zorder = 5",
      "Notes that if no bbox_to_anchor is set, get_bbox_to_anchor() returns self.axes.bbox"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/anchored_artists.py",
      "lib/matplotlib/offsetbox.py",
      "lib/matplotlib/transforms.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/anchored_artists.py] lib.mpl_toolkits.axes_grid1.anchored_artists.AnchoredDrawingArea\nlib.mpl_toolkits.axes_grid1.anchored_artists.AnchoredDrawingArea.__init__:\n  An anchored container with a fixed size and fillable `.DrawingArea`.\n\n        Artists added to the *drawing_area* will have their coordinates\n        interpreted as pixels. Any transformations set on the artists will be\n        overridden.\n\n        Parameters\n        ----------\n        width, height : float\n            Width and height of the container, in pixels.\n        xdescent, ydescent : float\n            Descent of the container in the x- and y- direction, in pixels.\n        loc : str\n            Location of this artist.  Valid locations are\n            'upper left', 'upper center', 'upper right',\n            'center left', 'center', 'center right',\n            'lower left', 'lower center', 'lower right'.\n            For backward compatibility, numeric values are accepted as well.\n            See the parameter *loc* of `.Legend` for details.\n        pad : float, default: 0.4\n            Padding around the child objects, in fraction of the font size.\n        borderpad : float, default: 0.5\n            Border padding, in fraction of the font size.\n        prop : `~matplotlib.font_manager.FontProperties`, optional\n            Font property used as a reference for paddings.\n        frameon : bool, default: True\n            If True, draw a box around this artist.\n        **kwargs\n            Keyword arguments forwarded to `.AnchoredOffsetbox`.\n\n        Attributes\n        ----------\n        drawing_area : `~matplotlib.offsetbox.DrawingArea`\n            A container for artists to display.\n\n        Examples\n        --------\n        To display blue and red circles of different sizes in the upper right\n        of an Axes *ax*:\n\n        >>> ada = AnchoredDrawingArea(20, 20, 0, 0,\n        ...                           loc='upper right', frameon=False)\n        >>> ada.drawing_area.add_artist(Circle((10, 10), 10, fc=\"b\"))\n        >>> ada.drawing_area.add_artist(Circle((30, 10), 5, fc=\"r\"))\n        >>> ax.add_artist(ada)",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 5 claims that DrawingArea.add_artist() does NOT override transforms if the artist already has a custom transform set, stating this contradicts the docstring. The docstring explicitly says 'Any transformations set on the artists will be overridden.' This is a claim about implementation behavior that contradicts the documented behavior, though it may reflect actual source code behavior not shown in the provided documentation.",
      "Point 1 claims that the 'prop' parameter is 'silently discarded' and hardcoded to None. This is a specific implementation claim not verifiable from the provided documentation, which lists 'prop' as an optional parameter with no mention of it being discarded.",
      "Point 7 claims DrawingArea defaults clip to False, meaning children are NOT clipped. This is an implementation detail not covered by the documentation.",
      "Many claims (points 2-9) describe internal implementation details about AnchoredOffsetbox, DrawingArea, transforms, zorder, etc. that go well beyond the provided documentation. These cannot be verified from the given doc chunk but appear to be plausible code-level details."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor documentation-referencing parenthetical about docstring contradiction needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_01",
    "repo": "matplotlib",
    "question": "What is the architecture of `PdfPages` and how does it interact with `PdfFile` and `FigureCanvasPdf.print_pdf` to support multi-page PDF output?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `PdfPages` class in `lib/matplotlib/backends/backend_pdf.py` is a thin wrapper around the `PdfFile` class that implements lazy file initialization. Here's how the architecture works:\n\n1. **Lazy initialization pattern**: `PdfPages.__init__` stores only `_filename`, `_metadata`, and sets `_file = None`. The actual `PdfFile` object is not created until `_ensure_file()` is called, which happens on the first `savefig()`, `get_pagecount()`, `infodict()`, or `attach_note()` call.\n\n2. **Interaction with `FigureCanvasPdf.print_pdf`**: When `PdfPages.savefig()` is called, it passes `self` (the PdfPages instance) as the filename to `figure.savefig()` with `format='pdf'` and `backend='pdf'`. Inside `FigureCanvasPdf.print_pdf`, it checks `isinstance(filename, PdfPages)` \u2014 if True, it calls `filename._ensure_file()` to get the underlying `PdfFile`. For multi-page usage, `file.finalize()` is NOT called after each page (only `file.endStream()` is called in the finally block), while for standalone PDF files, both `finalize()` and `close()` are called.\n\n3. **`PdfFile` internals**: The `PdfFile` constructor writes PDF version 1.4 header (`%PDF-1.4\\n`). It uses `itertools.count(1)` for object IDs via `_object_seq`, initializes an `xrefTable` with a zero object entry `[0, 65535, 'the zero object']`, and creates multiple reserved objects (root, pages, fonts, extended graphics states, tiling patterns, Gouraud triangles, external objects, resources).\n\n4. **Page count tracking**: `get_pagecount()` returns `len(self._ensure_file().pageList)` \u2014 `pageList` is a list of `Reference` objects, one appended per `newPage()` call.\n\n5. **Metadata handling**: The `_create_pdf_info_dict` function creates default values for 'Creator' (includes matplotlib version and URL), 'Producer' (includes backend name and version), and 'CreationDate' (respects `SOURCE_DATE_EPOCH` environment variable). It validates metadata keys against a known set and warns on unknown keys.\n\n6. **`keep_empty` parameter**: Deprecated in version 3.10 via `@_api.delete_parameter(\"3.10\", \"keep_empty\", addendum=\"This parameter does nothing.\")`. The parameter defaults to `None` and has no effect.\n\n7. **Context manager**: `__enter__` returns `self`, `__exit__` calls `close()`. The `close()` method calls `self._file.finalize()` followed by `self._file.close()` only if `_file is not None` (i.e., if any page was actually written).\n\n8. **Annotations**: `attach_note` calls `self._ensure_file().newTextnote(text, positionRect)` which creates a text annotation dict with type 'Annot', subtype 'Text', appended to `self.pageAnnotations`. These annotations are associated with the next page when `newPage()` is called.\n\n9. **`RendererPdf`** extends `_backend_pdf_ps.RendererPDFPSBase` (from `lib/matplotlib/backends/_backend_pdf_ps.py`), with `_afm_font_dir` pointing to `fonts/pdfcorefonts` and `_use_afm_rc_name = \"pdf.use14corefonts\"`.\n\n10. **There's also a separate `PdfPages` in `backend_pgf.py`** that uses LaTeX/PGF to compile pages, writing to a BytesIO buffer and running LaTeX on close.",
    "rubric": [
      "Explains PdfPages uses lazy initialization: __init__ stores _filename, _metadata, and sets _file=None; actual PdfFile is created only when _ensure_file() is called",
      "Identifies that _ensure_file() is triggered by savefig(), get_pagecount(), infodict(), or attach_note()",
      "Describes how savefig() passes the PdfPages instance itself as the filename to figure.savefig() with format='pdf' and backend='pdf'",
      "Explains that print_pdf checks isinstance(filename, PdfPages) and calls filename._ensure_file() to get the PdfFile if True, otherwise creates a new PdfFile",
      "Notes that for PdfPages (multi-page), finalize() is NOT called after each page and only endStream() is called in the finally block, whereas standalone PDFs call both finalize() and close()",
      "Describes PdfFile constructor details: writes PDF-1.4 header, uses itertools.count(1) for object IDs, initializes xrefTable with zero object, and creates reserved objects (root, pages, fonts, etc.)",
      "Explains that get_pagecount() returns len(pageList) from the PdfFile, where pageList is populated by newPage() calls",
      "Describes the close() method: calls _file.finalize() then _file.close() only if _file is not None, and supports context manager via __enter__/__exit__",
      "Mentions keep_empty is deprecated via @_api.delete_parameter('3.10', 'keep_empty') and has no effect",
      "Describes metadata handling via _create_pdf_info_dict: sets defaults for Creator (with matplotlib version), Producer (with backend name), and CreationDate (respects SOURCE_DATE_EPOCH)"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/_pylab_helpers.py",
      "lib/matplotlib/backends/_backend_pdf_ps.py",
      "lib/matplotlib/backends/backend_mixed.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_pdf.py] lib.matplotlib.backends.backend_pdf.PdfPages\nlib.matplotlib.backends.backend_pdf.PdfPages:\n  A multi-page PDF file.\n\n    Examples\n    --------\n    >>> import matplotlib.pyplot as plt\n    >>> # Initialize:\n    >>> with PdfPages('foo.pdf') as pdf:\n    ...     # As many times as you like, create a figure fig and save it:\n    ...     fig = plt.figure()\n    ...     pdf.savefig(fig)\n    ...     # When no figure is specified the current figure is saved\n    ...     pdf.savefig()\n\n    Notes\n    -----\n    In reality `PdfPages` is a thin wrapper around `PdfFile`, in order to avoid\n    confusion when using `~.pyplot.savefig` and forgetting the format argument.\n\nlib.matplotlib.backends.backend_pdf.PdfPages.__init__:\n  Create a new PdfPages object.\n\n        Parameters\n        ----------\n        filename : str or path-like or file-like\n            Plots using `PdfPages.savefig` will be written to a file at this location.\n            The file is opened when a figure is saved for the first time (overwriting\n            any older file with the same name).\n\n        metadata : dict, optional\n            Information dictionary object (see PDF reference section 10.2.1\n            'Document Information Dictionary'), e.g.:\n            ``{'Creator': 'My software', 'Author': 'Me', 'Title': 'Awesome'}``.\n\n            The standard keys are 'Title', 'Author', 'Subject', 'Keywords',\n            'Creator', 'Producer', 'CreationDate', 'ModDate', and\n            'Trapped'. Values have been predefined for 'Creator', 'Producer'\n            and 'CreationDate'. They can be removed by setting them to `None`.\n\nlib.matplotlib.backends.backend_pdf.PdfPages.close:\n  Finalize this object, making the underlying file a complete\n        PDF file.\n\nlib.matplotlib.backends.backend_pdf.PdfPages.infodict:\n  Return a modifiable information dictionary object\n        (see PDF reference section 10.2.1 'Document Information\n        Dictionary').\n\nlib.matplotlib.backends.backend_pdf.PdfPages.savefig:\n  Save a `.Figure` to this file as a new page.\n\n        Any other keyword arguments are passed to `~.Figure.savefig`.\n\n        Parameters\n        ----------\n        figure : `.Figure` or int, default: the active figure\n            The figure, or index of the figure, that is saved to the file.\n\nlib.matplotlib.backends.backend_pdf.PdfPages.get_pagecount:\n  Return the current number of pages in the multipage pdf file.\n\nlib.matplotlib.backends.backend_pdf.PdfPages.attach_note:\n  Add a new text note to the page to be saved next. The optional\n        positionRect specifies the position of the new note on the\n        page. It is outside the page per default to make sure it is\n        invisible on printouts.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 1 (lazy initialization): The documentation states 'The file is opened when a figure is saved for the first time', which supports the lazy initialization concept. However, the specific implementation details (_ensure_file, _file=None, etc.) are not in the documentation \u2014 they are plausible code-level extrapolations.",
      "Claim 2 (FigureCanvasPdf.print_pdf interaction): The documentation does not describe FigureCanvasPdf.print_pdf or how isinstance checks work internally. This is plausible but unsupported by the provided docs.",
      "Claim 3 (PdfFile internals - PDF 1.4, itertools.count, xrefTable): None of these details appear in the documentation. They are plausible code-level details but entirely unsupported.",
      "Claim 4 (pageList tracking): The documentation says get_pagecount returns 'the current number of pages in the multipage pdf file' but doesn't mention pageList. Plausible extrapolation.",
      "Claim 5 (metadata handling - SOURCE_DATE_EPOCH, _create_pdf_info_dict): The documentation mentions predefined values for Creator, Producer, CreationDate but doesn't describe SOURCE_DATE_EPOCH or the specific function name. Plausible but unsupported.",
      "Claim 6 (keep_empty parameter deprecated in 3.10): The documentation for __init__ does not mention keep_empty at all. This is an unsupported claim that cannot be verified from the provided docs.",
      "Claim 7 (Context manager behavior): The documentation shows context manager usage in the example and mentions close() finalizes the file. The specific implementation details (finalize + close only if _file is not None) are plausible extrapolations.",
      "Claim 8 (attach_note internals - newTextnote, pageAnnotations): The documentation describes attach_note's purpose but not the internal implementation. Plausible extrapolation.",
      "Claim 9 (RendererPdf details): Not mentioned in the documentation at all. Unsupported.",
      "Claim 10 (Separate PdfPages in backend_pgf.py): Not mentioned in the documentation. Unsupported."
    ],
    "strip_verify_leakage": 0.09,
    "strip_verify_summary": "Minor doc leakage: rationale about PDF 1.4 supporting alpha transparency and 'reproducible builds' purpose of SOURCE_DATE_EPOCH are from comments/docs only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_05",
    "repo": "matplotlib",
    "question": "What is the class hierarchy and component architecture of the AxisArtist system, including its child artists, helper classes, and how ticks and labels are managed?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `AxisArtist` class in `lib/mpl_toolkits/axisartist/axis_artist.py` is a custom artist that draws axis lines, ticks, tick labels, and axis labels as an alternative to Matplotlib's built-in `Axis` class, primarily motivated by the need to support curvilinear grids and angled ticks.\n\n**Class Hierarchy and Architecture:**\n- `AxisArtist` inherits from `matplotlib.artist.Artist` (not from `matplotlib.axis.Axis`), and has a `zorder` of 2.5 (compared to the base `Artist.zorder` of 0).\n- Its child components are separate artist objects: `self.line` (a `PathPatch` or arrow-style artist), `self.major_ticks` and `self.minor_ticks` (instances of `Ticks`, which extends both `AttributeCopier` and `Line2D`), `self.major_ticklabels` and `self.minor_ticklabels` (instances of `TickLabels`), `self.label` (an `AxisLabel`), and `self.offsetText` (an `mtext.Annotation`).\n- `TickLabels` inherits from `AxisLabel` (not directly from `LabelBase` or `mtext.Text`), which means it inherits pad management, `get_ref_artist`, and `set_axis_direction` from `AxisLabel`, but overrides `_default_alignments` and `_default_angles` with different values. For instance, `TickLabels._default_alignments['left']` is `('center', 'right')` while `AxisLabel._default_alignments['left']` is `('bottom', 'center')`.\n- `LabelBase` extends `mtext.Text` and sets `rotation_mode` to `\"anchor\"` in its `__init__`. Its `_text_ref_angle` property returns `self._ref_angle + 90` when `_text_follow_ref_angle` is True.\n\n**Helper System:**\n- The `__init__` method stores the helper as `self._axis_artist_helper` and determines the underlying Matplotlib axis: `axes.yaxis` for \"left\"/\"right\" directions, `axes.xaxis` for \"bottom\"/\"top\".\n- The helper class hierarchy starts with `_AxisArtistHelperBase` (which defines `nth_coord` and a `_to_xy` method for creating coordinate pairs), then `_FixedAxisArtistHelperBase` (for axes-boundary axes) and `_FloatingAxisArtistHelperBase` (for axes at arbitrary data coordinates).\n- `FixedAxisArtistHelperRectilinear.get_tick_iterators` yields tuples of `(loc, angle_normal, angle_tangent, label)` where `angle_normal` and `angle_tangent` are `{0: (90, 0), 1: (0, 90)}[nth_coord]`.\n\n**Tick Direction and Drawing:**\n- The `Ticks` class defines three tick path shapes in `_tick_paths`: `\"in\"` draws from `[0,0]` to `[+1,0]`, `\"out\"` from `[-1,0]` to `[0,0]`, and `\"inout\"` from `[-1/2,0]` to `[+1/2,0]`.\n- In `_get_tick_info`, the tick angle is flipped by 180\u00b0 depending on the relationship between the label angle and the normal angle: `angle_tick = angle_normal if 90 <= (angle_label - angle_normal) % 360 <= 270 else angle_normal + 180`. This ensures ticks point away from the labels.\n- The `_update_ticks` method calculates `_external_pad` for ticklabels based on the major tick size, tick direction (\"in\" \u2192 0, \"inout\" \u2192 0.5, \"out\" \u2192 1), and tick visibility (0 if invisible).\n\n**Axis Line Styling:**\n- `set_axisline_style` uses the `AxislineStyle` class (which inherits from `matplotlib.patches._Style`). The `_Style.__new__` method parses style strings like `\"->,size=1.5\"` by splitting on commas and converting values to floats.\n- Two registered styles exist: `\"->\"` (SimpleArrow using FancyArrowPatch with `\"->\"`  arrow style) and `\"-|>\"` (FilledArrow with `\"-|>\"` arrow style). When no style is set, the axis line is a plain `PathPatch` using rcParams for edge color, linewidth, capstyle, and joinstyle.\n\n**Label Initialization:**\n- The axis label is initialized with text `\"__from_axes__\"` and color `\"auto\"`. When `get_text()` is called and the text is `\"__from_axes__\"`, it delegates to `self._axis.label.get_text()`, effectively mirroring the underlying Matplotlib axis label. Similarly, `\"auto\"` color delegates to the ref artist via `AttributeCopier.get_attribute_from_ref_artist`.\n\n**Offset Transform:**\n- The constructor creates an `offset_transform` using `ScaledTranslation` with a scale of `1/72` (points to inches) combined with the figure's `dpi_scale_trans`, allowing offsets specified in points.\n\n**Integration with Axes:**\n- The `axislines.Axes` class creates four `AxisArtist` instances (bottom, top, left, right) during `clear()`, storing them in an `AxisDict`. The top and right axes have their labels and ticklabels hidden by default. `AxesZero` additionally creates \"xzero\" and \"yzero\" floating axes at coordinate 0, but sets them invisible by default.",
    "rubric": [
      "AxisArtist inherits from matplotlib.artist.Artist (not matplotlib.axis.Axis) with zorder=2.5",
      "Child components include self.line (PathPatch or arrow-style artist), major_ticks/minor_ticks (Ticks class inheriting from AttributeCopier and Line2D), major_ticklabels/minor_ticklabels (TickLabels), self.label (AxisLabel), and self.offsetText (mtext.Annotation)",
      "TickLabels inherits from AxisLabel (not directly from LabelBase or mtext.Text), and overrides _default_alignments and _default_angles with different values than AxisLabel",
      "LabelBase extends mtext.Text with rotation_mode='anchor', and its _text_ref_angle returns self._ref_angle + 90 when _text_follow_ref_angle is True",
      "Helper hierarchy: _AxisArtistHelperBase (defines nth_coord and _to_xy), _FixedAxisArtistHelperBase (for axes-boundary axes), and _FloatingAxisArtistHelperBase (for floating axes at arbitrary coordinates)",
      "FixedAxisArtistHelperRectilinear.get_tick_iterators yields (loc, angle_normal, angle_tangent, label) where angles are {0: (90, 0), 1: (0, 90)}[nth_coord]",
      "Ticks._tick_paths defines 'in' as [0,0]->[+1,0], 'out' as [-1,0]->[0,0], and 'inout' as [-1/2,0]->[+1/2,0]",
      "Tick angle logic: angle_tick = angle_normal if 90 <= (angle_label - angle_normal) % 360 <= 270 else angle_normal + 180",
      "_update_ticks computes _external_pad based on tick size multiplied by direction factor ('in'\u21920, 'inout'\u21920.5, 'out'\u21921) and tick visibility",
      "The axis label is initialized with text '__from_axes__' and color 'auto', delegating to the underlying Matplotlib axis label and ref artist respectively",
      "The axislines.Axes class creates four AxisArtist instances in clear() stored in an AxisDict, with top/right having labels and ticklabels hidden by default; AxesZero adds 'xzero'/'yzero' floating axes set invisible by default"
    ],
    "key_files": [
      "lib/mpl_toolkits/axisartist/axis_artist.py",
      "lib/mpl_toolkits/axisartist/axislines.py",
      "lib/mpl_toolkits/axisartist/axisline_style.py",
      "lib/mpl_toolkits/axes_grid1/mpl_axes.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axisartist/axis_artist.py] lib.mpl_toolkits.axisartist.axis_artist.AxisArtist\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist:\n  An artist which draws axis (a line along which the n-th axes coord\n    is constant) line, ticks, tick labels, and axis label.\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.__init__:\n  Parameters\n        ----------\n        axes : `mpl_toolkits.axisartist.axislines.Axes`\n        helper : `~mpl_toolkits.axisartist.axislines.AxisArtistHelper`\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.set_axis_direction:\n  Adjust the direction, text angle, and text alignment of tick labels\n        and axis labels following the Matplotlib convention for the rectangle\n        axes.\n\n        The *axis_direction* must be one of [left, right, bottom, top].\n\n        =====================    ========== ========= ========== ==========\n        Property                 left       bottom    right      top\n        =====================    ========== ========= ========== ==========\n        ticklabel direction      \"-\"        \"+\"       \"+\"        \"-\"\n        axislabel direction      \"-\"        \"+\"       \"+\"        \"-\"\n        ticklabel angle          90         0         -90        180\n        ticklabel va             center     baseline  center     baseline\n        ticklabel ha             right      center    right      center\n        axislabel angle          180        0         0          180\n        axislabel va             center     top       center     bottom\n        axislabel ha             right      center    right      center\n        =====================    ========== ========= ========== ==========\n\n        Note that the direction \"+\" and \"-\" are relative to the direction of\n        the increasing coordinate. Also, the text angles are actually\n        relative to (90 + angle of the direction to the ticklabel),\n        which gives 0 for bottom axis.\n\n        Parameters\n        ----------\n        axis_direction : {\"left\", \"bottom\", \"right\", \"top\"}\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.set_ticklabel_direction:\n  Adjust the direction of the tick labels.\n\n        Note that the *tick_direction*\\s '+' and '-' are relative to the\n        direction of the increasing coordinate.\n\n        Parameters\n        ----------\n        tick_direction : {\"+\", \"-\"}\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.set_axislabel_direction:\n  Adjust the direction of the axis label.\n\n        Note that the *label_direction*\\s '+' and '-' are relative to the\n        direction of the increasing coordinate.\n\n        Parameters\n        ----------\n        label_direction : {\"+\", \"-\"}\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.get_helper:\n  Return axis artist helper instance.\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.set_axisline_style:\n  Set the axisline style.\n\n        The new style is completely defined by the passed attributes. Existing\n        style attributes are forgotten.\n\n        Parameters\n        ----------\n        axisline_style : str or None\n            The line style, e.g. '->', optionally followed by a comma-separated\n            list of attributes. Alternatively, the attributes can be provided\n            as keywords.\n\n            If *None* this returns a string containing the available styles.\n\n        Examples\n        --------\n        The following two commands are equal:\n\n        >>> set_axisline_style(\"->,size=1.5\")\n        >>> set_axisline_style(\"->\", size=1.5)\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.get_axisline_style:\n  Return the current axisline style.\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist._init_line:\n  Initialize the *line* artist that is responsible to draw the axis line.\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist._get_tick_info:\n  Return a pair of:\n\n        - list of locs and angles for ticks\n        - list of locs, angles and labels for ticklabels.\n\nlib.mpl_toolkits.axisartist.axis_artist.AxisArtist.toggle:\n  Toggle visibility of ticks, ticklabels, and (axis) label.\n        To turn all off, ::\n\n          axis.toggle(all=False)\n\n        To turn all off but ticks on ::\n\n          axis.toggle(all=False, ticks=True)\n\n        To turn all on but (axis) label off ::\n\n          axis.toggle(all=True, label=False)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that 'TickLabels inherits from AxisLabel' is not verifiable from the provided documentation and may be a hallucination based on source code knowledge not present in the doc.",
      "The claim about TickLabels._default_alignments['left'] being ('center', 'right') versus AxisLabel._default_alignments['left'] being ('bottom', 'center') is not in the documentation - the doc shows ticklabel ha for 'left' is 'right' and va is 'center', and axislabel ha for 'left' is 'right' and va is 'center', which partially contradicts this claim.",
      "Many architectural details (zorder of 2.5, PathPatch, ScaledTranslation with 1/72, AxisDict, AxesZero, _tick_paths definitions, etc.) are not present in the provided documentation and represent extrapolations from source code knowledge.",
      "The claim about axislabel va for 'left' being 'bottom' contradicts the documentation which shows axislabel va for 'left' is 'center'.",
      "The set_axis_direction table in the doc shows ticklabel ha for 'left' is 'right' and for 'right' is also 'right', which is correctly reflected in the answer's discussion."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_08",
    "repo": "matplotlib",
    "question": "How does `check_isinstance` in Matplotlib's internal API validate types and format error messages when a check fails?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `check_isinstance` function in `lib/matplotlib/_api/__init__.py` implements a type-checking utility with the following algorithm:\n\n1. **Type normalization:** The `types` parameter is normalized into a tuple using a three-way conditional expression:\n   - If `types` is already a single type (checked via `isinstance(types, type)`), it wraps it in a single-element tuple `(types,)`\n   - If `types` is `None` itself (not inside a tuple), it becomes `(type(None),)`\n   - Otherwise (when it's an iterable like a tuple/list), it converts any `None` entries inside to `type(None)` while leaving other types unchanged\n\n2. **Inner `type_name` helper:** Formats type names for error messages using three cases:\n   - Returns `\"None\"` if the type is `NoneType`\n   - Returns `tp.__qualname__` if the type is from the `builtins` module (e.g., `str`, `int`)\n   - Otherwise returns `f\"{tp.__module__}.{tp.__qualname__}\"` (fully qualified name)\n\n3. **Iteration over kwargs:** For each key-value pair:\n   - Uses Python's built-in `isinstance(v, types)` for the actual check\n   - If the check fails, builds a human-readable error message:\n     - Collects type names via `map(type_name, types)`\n     - If `\"None\"` is among the names, it's moved to the end of the list\n     - For multiple types, formats as `\"TypeA, TypeB or TypeC\"` (comma-separated with 'or' before the last)\n     - For a single type, just uses that type's name\n     - Raises `TypeError` with format: `\"'key' must be an instance of <types>, not a <actual_type>\"`\n\n4. **Positional-only parameter:** The `types` parameter uses Python 3.8+ positional-only syntax (`/`), which means `types` can be used as a keyword argument name in `**kwargs` to be checked.\n\nThe function supports checking multiple values against the same type set in a single call via `**kwargs` and the iteration loop.",
    "rubric": [
      "Explains type normalization: single type wrapped in tuple, None converted to type(None), iterable entries with None replaced by type(None)",
      "Describes the inner type_name helper: returns 'None' for NoneType, __qualname__ for builtins, module.qualname for others",
      "Explains the error message formatting: 'None' is moved to end of the type list, multiple types joined with commas and 'or' before last",
      "Mentions the TypeError format: \"'key' must be an instance of <types>, not a <actual_type>\"",
      "Notes the positional-only parameter (/) allowing 'types' to be used as a kwarg key in the check",
      "Explains iteration over kwargs to check multiple values against the same type set"
    ],
    "key_files": [
      "lib/matplotlib/_api/__init__.py",
      "lib/matplotlib/_api/__init__.pyi"
    ],
    "source_doc": "[docstring: lib/matplotlib/_api/__init__.py] lib.matplotlib._api.check_isinstance\nFor each *key, value* pair in *kwargs*, check that *value* is an instance\n    of one of *types*; if not, raise an appropriate TypeError.\n\n    As a special case, a ``None`` entry in *types* is treated as NoneType.\n\n    Examples\n    --------\n    >>> _api.check_isinstance((SomeClass, None), arg=arg)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only usage examples from other files and minor rationale statements needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_07",
    "repo": "matplotlib",
    "question": "Where are colormap normalization classes defined and how does the norm-to-color pipeline connect them to colormaps?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The colormap normalization classes in matplotlib are located in `lib/matplotlib/colors.py`. The normalization hierarchy starts with an abstract base class `Norm`, which is subclassed by `Normalize`. All concrete norm classes inherit from `Normalize`. The `Norm` ABC defines abstract properties (`vmin`, `vmax`, `clip`, `n_components`) and abstract methods (`__call__`, `autoscale`, `autoscale_None`, `scaled`). It uses a `CallbackRegistry` with a 'changed' signal to notify listeners of updates.\n\n`LogNorm` is NOT defined as a regular class - it is dynamically generated using `make_norm_from_scale(functools.partial(scale.LogScale, nonpositive='mask'))(Normalize)`. The `make_norm_from_scale` decorator creates an inner class that overrides `__call__` to use the scale's transform, applying the transform then linearly mapping the transformed vmin/vmax range to [0,1].\n\nThe `LogTransform` (in `lib/matplotlib/scale.py`) caches optimized log functions in `_log_funcs = {np.e: np.log, 2: np.log2, 10: np.log10}` for performance. For non-standard bases, it falls back to `np.log(values) / np.log(self.base)`. When `nonpositive='mask'` (the mode used by LogNorm), non-positive values produce NaN/inf since no clipping is applied (self._clip is False).\n\nThe normalization is used in the data-to-color pipeline via `Colorizer.to_rgba()` (in `lib/matplotlib/colorizer.py`), which first normalizes data to [0,1] with `self.norm(x)`, then maps to colors with `self.cmap(x)`. The `Colormap.__call__` method (in colors.py) multiplies the [0,1] normalized values by `self.N` (default 256) to get lookup table indices.\n\nOther norm classes include: `TwoSlopeNorm` (uses `np.interp` with points [vmin, vcenter, vmax] \u2192 [0, 0.5, 1]), `CenteredNorm` (keeps vmin/vmax symmetrical around vcenter), `PowerNorm` (applies power transformation with a gamma parameter), `BoundaryNorm` (maps data to discrete boundaries), `SymLogNorm` (generated via `make_norm_from_scale` from `scale.SymmetricalLogScale`), `AsinhNorm` (generated from `scale.AsinhScale` with `linear_width` parameter), `FuncNorm` (generated from `scale.FuncScale` with user-provided forward/inverse functions), `NoNorm` (passes values through unchanged), and `MultiNorm` (wraps multiple scalar norms for multivariate colormaps).",
    "rubric": [
      "Identifies that normalization classes are defined in lib/matplotlib/colors.py",
      "Mentions the Norm ABC as the root abstract class with abstract properties (vmin, vmax, clip, n_components) and abstract methods (__call__, autoscale, autoscale_None, scaled)",
      "Explains that Normalize subclasses Norm and serves as the base for concrete norm classes",
      "Describes that LogNorm is dynamically generated via make_norm_from_scale using functools.partial(scale.LogScale, nonpositive='mask')",
      "Explains how make_norm_from_scale creates an inner class whose __call__ applies the scale's transform then linearly maps transformed vmin/vmax to [0,1]",
      "Mentions LogTransform in scale.py caches log functions in _log_funcs dict ({np.e: np.log, 2: np.log2, 10: np.log10}) for common bases",
      "Describes the pipeline in Colorizer.to_rgba (in colorizer.py): normalizes data with self.norm(x), then maps to colors with self.cmap(x)",
      "Lists other norm classes: TwoSlopeNorm, CenteredNorm, PowerNorm, BoundaryNorm, SymLogNorm, AsinhNorm, FuncNorm, NoNorm, and/or MultiNorm"
    ],
    "key_files": [
      "lib/matplotlib/colors.py",
      "lib/matplotlib/scale.py",
      "lib/matplotlib/colorizer.py",
      "galleries/users_explain/colors/colormapnorms.py"
    ],
    "source_doc": "[docstring: galleries/users_explain/colors/colormapnorms.py] galleries.users_explain.colors.colormapnorms\n.. redirect-from:: /tutorials/colors/colormapnorms\n\n.. _colormapnorms:\n\nColormap normalization\n======================\n\nObjects that use colormaps by default linearly map the colors in the\ncolormap from data values *vmin* to *vmax*.  For example::\n\n    pcm = ax.pcolormesh(x, y, Z, vmin=-1., vmax=1., cmap='RdBu_r')\n\nwill map the data in *Z* linearly from -1 to +1, so *Z=0* will\ngive a color at the center of the colormap *RdBu_r* (white in this\ncase).\n\nMatplotlib does this mapping in two steps, with a normalization from\nthe input data to [0, 1] occurring first, and then mapping onto the\nindices in the colormap.  Normalizations are classes defined in the\n:func:`matplotlib.colors` module.  The default, linear normalization\nis :func:`matplotlib.colors.Normalize`.\n\nArtists that map data to color pass the arguments *vmin* and *vmax* to\nconstruct a :func:`matplotlib.colors.Normalize` instance, then call it:\n\n.. code-block:: pycon\n\n   >>> import matplotlib as mpl\n   >>> norm = mpl.colors.Normalize(vmin=-1, vmax=1)\n   >>> norm(0)\n   0.5\n\nHowever, there are sometimes cases where it is useful to map data to\ncolormaps in a non-linear fashion.\n\nLogarithmic\n-----------\n\nOne of the most common transformations is to plot data by taking its logarithm\n(to the base-10).  This transformation is useful to display changes across\ndisparate scales.  Using `.colors.LogNorm` normalizes the data via\n:math:`log_{10}`.  In the example below, there are two bumps, one much smaller\nthan the other. Using `.colors.LogNorm`, the shape and location of each bump\ncan clearly be seen:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation refers to `matplotlib.colors` as a module (`:func:`matplotlib.colors``), while the answer refers to it as `lib/matplotlib/colors.py` - this is consistent but the doc uses `:func:` role which is technically incorrect (it's a module, not a function), and the answer doesn't note this.",
      "The answer contains extensive implementation details (line numbers, class hierarchies, method implementations) that go far beyond what the documentation covers. These cannot be verified from the documentation alone, but they are plausible code-level details.",
      "The documentation says 'Normalizations are classes defined in the :func:`matplotlib.colors` module' - the answer's claim about an abstract base class `Norm` at line 2304 and `Normalize` at line 2395 cannot be verified from this doc but is plausible.",
      "The documentation says LogNorm 'normalizes the data via log_{10}' - the answer's claim that LogNorm is dynamically generated using `make_norm_from_scale` with `LogScale` is a code-level detail that's plausible but unverifiable from the doc.",
      "The answer's description of LogNorm behavior (masking non-positive values) is consistent with the documentation's description of LogNorm being used to display logarithmic data."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for line numbers and implementation details only visible in truncated/documented sections.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_10",
    "repo": "matplotlib",
    "question": "Why does the `delete_parameter` decorator check `len(inner_args) <= name_idx` before calling `signature.bind()`, and why is `math.inf` used for keyword-only parameters?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `warn_deprecated` function and its related deprecation decorator infrastructure contain several explicit performance optimizations:\n\n1. **Early return fast path in `delete_parameter` wrapper:** The most significant performance optimization is in the `delete_parameter` decorator's wrapper function. Before calling the `signature.bind()` method (which parses arguments according to the function signature), the wrapper first performs a cheap check: `if len(inner_args) <= name_idx and name not in inner_kwargs`. If the deprecated parameter was not passed (which is the common, non-deprecated case), the function returns immediately by calling `func(*inner_args, **inner_kwargs)` without ever calling `signature.bind()`. This avoids the overhead of `signature.bind()` for the common case where deprecated parameters are not used.\n\n2. **Use of `math.inf` for `name_idx` to guarantee the fast path for keyword-only parameters:** When the deprecated parameter is `KEYWORD_ONLY` or when the parameter is not in the signature at all (passed via `**kwargs`), `name_idx` is set to `math.inf`. Since `len(inner_args) <= math.inf` is always true, this means the fast path only needs to check `name not in inner_kwargs`\u2014a single dictionary lookup\u2014rather than doing full signature binding.\n\n3. **Avoidance of `signature.bind()` in `make_keyword_only` wrapper:** The `make_keyword_only` decorator does not use `signature.bind()`. Instead, it simply checks `len(args) > name_idx` to determine if the parameter was passed positionally.\n\n4. **`warn_external` uses `skip_file_prefixes` on Python \u2265 3.12 instead of manual frame walking:** On Python 3.12+, instead of manually walking the call stack frame-by-frame with `sys._getframe()` and regex matching each frame's `__name__`, the function uses Python 3.12's `skip_file_prefixes` parameter to `warnings.warn`. For older Python, it manually walks frames using `sys._getframe()` and `frame.f_back`, and explicitly `del frame` afterward.\n\n5. **`DECORATORS` dict stores decorator references keyed by wrapper function:** The module-level `DECORATORS` dictionary stores decorator references keyed by wrapper function, rather than storing as attributes on wrappers. This is used by pyplot's `_copy_docstring_and_deprecators` to re-apply deprecation decorators to pyplot wrappers, as seen in the code where `_api.deprecation.DECORATORS.get(method)` is accessed while traversing `__wrapped__` chains.",
    "rubric": [
      "Explains that the check provides an early return fast path to avoid calling the expensive signature.bind() in the common case where the deprecated parameter is not passed",
      "Explains that math.inf ensures len(inner_args) <= name_idx is always true for keyword-only parameters, reducing the fast-path check to just a dictionary lookup (name not in inner_kwargs)",
      "Notes that the comment in the code explicitly states this is 'much faster than calling bind()'",
      "Mentions that make_keyword_only similarly avoids signature.bind() by using a simple len(args) > name_idx check",
      "Mentions that warn_external uses skip_file_prefixes on Python \u2265 3.12 instead of manual frame walking as another performance optimization in the deprecation infrastructure"
    ],
    "key_files": [
      "lib/matplotlib/_api/deprecation.py",
      "lib/matplotlib/_api/__init__.py",
      "lib/matplotlib/pyplot.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/_api/deprecation.py] lib.matplotlib._api.deprecation.warn_deprecated\nDisplay a standardized deprecation.\n\n    Parameters\n    ----------\n    since : str\n        The release at which this API became deprecated.\n    message : str, optional\n        Override the default deprecation message.  The ``%(since)s``,\n        ``%(name)s``, ``%(alternative)s``, ``%(obj_type)s``, ``%(addendum)s``,\n        and ``%(removal)s`` format specifiers will be replaced by the values\n        of the respective arguments passed to this function.\n    name : str, optional\n        The name of the deprecated object.\n    alternative : str, optional\n        An alternative API that the user may use in place of the deprecated\n        API.  The deprecation warning will tell the user about this alternative\n        if provided.\n    pending : bool, optional\n        If True, uses a PendingDeprecationWarning instead of a\n        DeprecationWarning.  Cannot be used together with *removal*.\n    obj_type : str, optional\n        The object type being deprecated.\n    addendum : str, optional\n        Additional text appended directly to the final message.\n    removal : str, optional\n        The expected removal version.  With the default (an empty string), a\n        removal version is automatically computed from *since*.  Set to other\n        Falsy values to not schedule a removal date.  Cannot be used together\n        with *pending*.\n\n    Examples\n    --------\n    ::\n\n        # To warn of the deprecation of \"matplotlib.name_of_module\"\n        warn_deprecated('1.4.0', name='matplotlib.name_of_module',\n                        obj_type='module')",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses extensive implementation details (line numbers, internal optimizations, specific code patterns) that are entirely absent from the provided documentation, which only covers the function's public API (parameters, their types, and usage examples).",
      "None of the 6 numbered claims about performance optimizations can be verified from the provided documentation chunk, which is solely a docstring describing parameters of `warn_deprecated`.",
      "The answer appears to be derived from reading the full source code file rather than the provided documentation chunk, making it impossible to verify the claims against the given doc.",
      "The answer does not actually address what `warn_deprecated` does according to its documentation (display a standardized deprecation warning with the described parameters), instead focusing entirely on implementation internals of related functions."
    ],
    "strip_verify_leakage": 0.3,
    "strip_verify_summary": "Several claims relied on stripped comments for rationale (e.g., 'much faster than bind()', reference cycle breaking, select_matching_signature performance note); code-verifiable structural facts were retained.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_06",
    "repo": "matplotlib",
    "question": "Why does `GridSpec.update()` use a special `_UNSET` sentinel instead of `None` for its default parameter values?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The Matplotlib axes arrangement system is designed around a layered architecture where `GridSpec` and `SubplotSpec` are the fundamental positioning primitives, and higher-level functions like `Figure.subplots()` and `Figure.subplot_mosaic()` provide convenient APIs on top. Several key design decisions explain why the code is structured this way:\n\n1. **Why `GridSpec.update()` uses the `_UNSET` sentinel instead of `None`**: `GridSpec.update()` uses `_UNSET` (imported as `_api.UNSET`) as default parameter values rather than `None`. This is because `None` has semantic meaning \u2014 setting a parameter to `None` results in rcParams defaults being used (as seen in `get_subplot_params()` which falls back to `mpl.rcParams[\"figure.subplot.\"+k]` when values are falsy). The `_UNSET` sentinel (defined in `_api/__init__.py` as a singleton `_Unset` class instance) allows the method to distinguish between \"parameter not provided\" and \"explicitly reset to default\".\n\n2. **Why `GridSpec.update()` iterates all figure managers**: After updating its parameters, `GridSpec.update()` iterates through `_pylab_helpers.Gcf.figs.values()` to find and reposition all axes across all open pyplot figures that reference this GridSpec. This allows a single `update()` call to propagate layout changes to all affected axes, since the code checks `if ss.get_gridspec() == self` and repositions matching axes across all figures.\n\n3. **Why `subplot_mosaic` only accepts strict `bool` for sharex/sharey** (unlike `subplots` which accepts 'all', 'row', 'col', 'none'): The `subplots` method in GridSpecBase accepts string values like \"all\", \"row\", \"col\", \"none\" for sharex/sharey as visible in the code.\n\n4. **Why `SubplotSpec.colspan` sorts column indices**: The `colspan` property explicitly sorts the column indices with `sorted([self.num1 % ncols, self.num2 % ncols])`. This handles cases where num2 refers to a column index that is less than num1's column index.\n\n5. **Why `subfigures()` sets `left=0, right=1, bottom=0, top=1`**: When creating subfigures, a GridSpec is constructed with margins explicitly set to fill the entire figure space (left=0, right=1, bottom=0, top=1).\n\n6. **Why `SubFigure` shares the parent's `_axstack`**: SubFigure.__init__ assigns `self._axstack = parent._axstack`, meaning all axes in subfigures are tracked in the parent's axes stack.\n\n7. **Why `_identify_keys_and_nested` uses `_OrderedSet`**: `cbook._OrderedSet` is imported in the axes module (visible in axes/_base.py as `from matplotlib.cbook import _OrderedSet`), providing an ordered set data structure that preserves insertion order while preventing duplicates.",
    "rubric": [
      "Explains that `None` has semantic meaning in this context \u2014 setting a parameter to `None` resets it to rcParams defaults",
      "Explains that `_UNSET` allows distinguishing between 'parameter not provided' (leave unchanged) and 'explicitly set to None' (reset to default)",
      "Mentions that `get_subplot_params()` falls back to `mpl.rcParams['figure.subplot.'+k]` when GridSpec attributes are falsy/None",
      "Notes that `_UNSET` is imported from `matplotlib._api` (defined as a singleton instance of an `_Unset` class)"
    ],
    "key_files": [
      "lib/matplotlib/gridspec.py",
      "lib/matplotlib/figure.py",
      "lib/matplotlib/_api/__init__.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/cbook.py"
    ],
    "source_doc": "[docstring: galleries/users_explain/axes/arranging_axes.py] galleries.users_explain.axes.arranging_axes\n.. redirect-from:: /tutorials/intermediate/gridspec\n.. redirect-from:: /tutorials/intermediate/arranging_axes\n\n.. _arranging_axes:\n\n===================================\nArranging multiple Axes in a Figure\n===================================\n\nOften more than one Axes is wanted on a figure at a time, usually\norganized into a regular grid.  Matplotlib has a variety of tools for\nworking with grids of Axes that have evolved over the history of the library.\nHere we will discuss the tools we think users should use most often, the tools\nthat underpin how Axes are organized, and mention some of the older tools.\n\n.. note::\n\n    Matplotlib uses *Axes* to refer to the drawing area that contains\n    data, x- and y-axis, ticks, labels, title, etc. See :ref:`figure_parts`\n    for more details.  Another term that is often used is \"subplot\", which\n    refers to an Axes that is in a grid with other Axes objects.\n\nOverview\n========\n\nCreate grid-shaped combinations of Axes\n---------------------------------------\n\n`~matplotlib.pyplot.subplots`\n    The primary function used to create figures and a grid of Axes.  It\n    creates and places all Axes on the figure at once, and returns an\n    object array with handles for the Axes in the grid.  See\n    `.Figure.subplots`.\n\nor\n\n`~matplotlib.pyplot.subplot_mosaic`\n    A simple way to create figures and a grid of Axes, with the added\n    flexibility that Axes can also span rows or columns. The Axes are returned\n    in a labelled dictionary instead of an array.  See also\n    `.Figure.subplot_mosaic` and\n    :ref:`mosaic`.\n\nSometimes it is natural to have more than one distinct group of Axes grids,\nin which case Matplotlib has the concept of `.SubFigure`:\n\n`~matplotlib.figure.SubFigure`\n    A virtual figure within a figure.\n\nUnderlying tools\n----------------\n\nUnderlying these are the concept of a `~.gridspec.GridSpec` and\na `~.SubplotSpec`:\n\n`~matplotlib.gridspec.GridSpec`\n    Specifies the geometry of the grid that a subplot will be\n    placed. The number of rows and number of columns of the grid\n    need to be set. Optionally, the subplot layout parameters\n    (e.g., left, right, etc.) can be tuned.\n\n`~matplotlib.gridspec.SubplotSpec`\n    Specifies the location of the subplot in the given `.GridSpec`.\n\n.. _fixed_size_axes:\n\nAdding single Axes at a time\n----------------------------\n\nThe above functions create all Axes in a single function call.  It is also\npossible to add Axes one at a time, and this was originally how Matplotlib\nused to work.  Doing so is generally less elegant and flexible, though\nsometimes useful for interactive work or to place an Axes in a custom\nlocation:\n\n`~matplotlib.figure.Figure.add_axes`\n    Adds a single Axes at a location specified by\n    ``[left, bottom, width, height]`` in fractions of figure width or height.\n\n`~matplotlib.pyplot.subplot` or `.Figure.add_subplot`\n    Adds a single subplot on a figure, with 1-based indexing (inherited from\n    Matlab).  Columns and rows can be spanned by specifying a range of grid\n    cells.\n\n`~matplotlib.pyplot.subplot2grid`\n    Similar to `.pyplot.subplot`, but uses 0-based indexing and two-d python\n    slicing to choose cells.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation-level claims (line numbers, sentinel values, internal method behaviors, specific code patterns) that go far beyond what the documentation covers. The documentation is a high-level overview of arranging axes, not source code documentation.",
      "Claim about `subplot_mosaic` only accepting strict `bool` for sharex/sharey is not mentioned in the documentation - it's plausible code-level detail but unverifiable from this doc.",
      "Claim about `SubplotSpec.colspan` sorting column indices with specific line numbers is not verifiable from this documentation.",
      "Claim about `SubFigure` sharing parent's `_axstack` is not mentioned in the documentation.",
      "Claim about `_identify_keys_and_nested` using `_OrderedSet` is not mentioned in the documentation.",
      "Claim about `GridSpec.update()` using `_UNSET` sentinel is not mentioned in the documentation.",
      "Claim about `GridSpec.update()` iterating all figure managers is not mentioned in the documentation.",
      "The answer's core framing as 'design decisions' with specific line numbers suggests it was derived from source code rather than this documentation chunk, but the claims don't directly contradict the documentation.",
      "The documentation does confirm the high-level claims: GridSpec and SubplotSpec are underlying tools, subplots and subplot_mosaic are higher-level functions, SubFigure is a virtual figure within a figure, and subplot_mosaic returns a labelled dictionary."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims rely on comments/docstrings (design rationales about MATLAB compatibility, future API expansion, z-order management) that would be stripped, while most code-structural claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_09",
    "repo": "matplotlib",
    "question": "What does `select_matching_signature` do, how is it implemented, and what are all its call sites across the codebase?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`select_matching_signature` is a utility function in `lib/matplotlib/_api/__init__.py` that implements signature overloading by trial-and-error function calling. Its implementation is extremely concise: it iterates over `funcs` using `enumerate`, tries calling each with `*args, **kwargs`, catches `TypeError`, and re-raises only if the current function is the last one (`if i == len(funcs) - 1: raise`).\n\nThe function lives in the `matplotlib._api` module alongside related API helpers (`check_isinstance`, `check_in_list`, `check_shape`, `nargs_error`, `kwarg_error`, etc.) and re-exports from `matplotlib._api.deprecation` (including `warn_deprecated`, `deprecated`, `rename_parameter`, `delete_parameter`, `make_keyword_only`, `deprecate_method_override`, etc.).\n\nThe type stub in `__init__.pyi` types it as `def select_matching_signature(funcs: list[Callable], *args: Any, **kwargs: Any) -> Any`.\n\n**Callers in the codebase (7 call sites):**\n\n1. **`colorbar.Colorbar.add_lines`** \u2014 Dispatches between a `(self, CS, erase=True)` signature (accepting a `ContourSet`) and `(self, levels, colors, linewidths, erase=True)` signature. When `CS` is in params, it recursively calls `self.add_lines()` again with the expanded signature.\n\n2. **`lines.AxLine.set_xy1`** \u2014 Dispatches between old `(self, x, y)` and new `(self, xy1)` signatures. The old signature issues a deprecation warning for version 3.10.\n\n3. **`lines.AxLine.set_xy2`** \u2014 Same pattern as `set_xy1`, dispatching between `(self, x, y)` and `(self, xy2)` with a 3.10 deprecation.\n\n4. **`offsetbox._compat_get_offset`** \u2014 A *decorator* wrapping `get_offset` methods. Dispatches between old `(self, width, height, xdescent, ydescent, renderer)` and new `(self, bbox, renderer)`. Applied to `OffsetBox.get_offset` and `AnchoredOffsetbox.get_offset`. When old signature matches, it constructs a `Bbox.from_bounds(-xdescent, -ydescent, width, height)` before calling the wrapped method.\n\n5. **`widgets.MultiCursor.__init__`** \u2014 Dispatches between `(axes)` and `(canvas, axes)`. The canvas parameter is deprecated since 3.11. Before calling `select_matching_signature`, it manually pops `canvas` and `axes` from `lineprops` into a separate `kwargs` dict.\n\n6. **`mpl_toolkits.axisartist.grid_finder.GridFinder.get_grid_info`** \u2014 Dispatches between `(x1, y1, x2, y2)` (deprecated since 3.11) and `(bbox)` signatures.\n\n7. **`mpl_toolkits.mplot3d.axis3d.Axis.__init__`** \u2014 Uniquely uses *instance methods* (`self._old_init`, `self._new_init`) as the func list rather than lambdas. The old signature `(adir, v_intervalx, d_intervalx, axes, ...)` is deprecated since 3.6 (with `pending=True`). The new signature is `(axes, *, rotate_label=None, **kwargs)`. The `Axis` class inherits from `maxis.XAxis`.\n\nAll callers follow the same pattern: lambdas (or methods) returning `locals()`, then checking for old-signature keys in the returned dict, issuing deprecation warnings via `_api.warn_deprecated` when old signatures match, and placing the new/preferred signature last so that TypeErrors from bad arguments reflect the new API.",
    "rubric": [
      "Explains that select_matching_signature implements signature overloading by trying each function in order, catching TypeError, and re-raising only from the last function",
      "Describes the implementation: iterates with enumerate, tries calling each func with *args/**kwargs, catches TypeError, re-raises if i == len(funcs) - 1",
      "Mentions it lives in the _api module alongside helpers like check_isinstance, check_in_list, nargs_error, kwarg_error, etc.",
      "Mentions re-exports from deprecation submodule (warn_deprecated, deprecated, rename_parameter, etc.)",
      "Mentions the type stub signature: funcs is list[Callable], returns Any",
      "Identifies colorbar.Colorbar.add_lines as a call site dispatching between ContourSet and (levels, colors, linewidths) signatures",
      "Identifies lines.AxLine.set_xy1 and set_xy2 as call sites dispatching between old (x, y) and new (xy1/xy2) signatures with 3.10 deprecation",
      "Identifies offsetbox._compat_get_offset as a decorator-based call site dispatching between old (width, height, xdescent, ydescent, renderer) and new (bbox, renderer) signatures",
      "Identifies widgets.MultiCursor.__init__ as a call site dispatching between (axes) and (canvas, axes) with canvas deprecated since 3.11",
      "Identifies grid_finder.GridFinder.get_grid_info as a call site dispatching between (x1, y1, x2, y2) and (bbox)",
      "Identifies mplot3d.axis3d.Axis.__init__ as a call site that uniquely uses instance methods (_old_init, _new_init) instead of lambdas",
      "Notes the common pattern: lambdas returning locals(), checking for old-signature keys, issuing deprecation warnings, with the new/preferred signature placed last"
    ],
    "key_files": [
      "lib/matplotlib/_api/__init__.py",
      "lib/matplotlib/_api/__init__.pyi",
      "lib/matplotlib/colorbar.py",
      "lib/matplotlib/lines.py",
      "lib/matplotlib/offsetbox.py",
      "lib/matplotlib/widgets.py",
      "lib/mpl_toolkits/mplot3d/axis3d.py",
      "lib/mpl_toolkits/axisartist/grid_finder.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/_api/__init__.py] lib.matplotlib._api.select_matching_signature\nSelect and call the function that accepts ``*args, **kwargs``.\n\n    *funcs* is a list of functions which should not raise any exception (other\n    than `TypeError` if the arguments passed do not match their signature).\n\n    `select_matching_signature` tries to call each of the functions in *funcs*\n    with ``*args, **kwargs`` (in the order in which they are given).  Calls\n    that fail with a `TypeError` are silently skipped.  As soon as a call\n    succeeds, `select_matching_signature` returns its return value.  If no\n    function accepts ``*args, **kwargs``, then the `TypeError` raised by the\n    last failing call is re-raised.\n\n    Callers should normally make sure that any ``*args, **kwargs`` can only\n    bind a single *func* (to avoid any ambiguity), although this is not checked\n    by `select_matching_signature`.\n\n    Notes\n    -----\n    `select_matching_signature` is intended to help implementing\n    signature-overloaded functions.  In general, such functions should be\n    avoided, except for back-compatibility concerns.  A typical use pattern is\n    ::\n\n        def my_func(*args, **kwargs):\n            params = select_matching_signature(\n                [lambda old1, old2: locals(), lambda new: locals()],\n                *args, **kwargs)\n            if \"old1\" in params:\n                warn_deprecated(...)\n                old1, old2 = params.values()  # note that locals() is ordered.\n            else:\n                new, = params.values()\n            # do things with params\n\n    which allows *my_func* to be called either with two parameters (*old1* and\n    *old2*) or a single one (*new*).  Note that the new signature is given\n    last, so that callers get a `TypeError` corresponding to the new signature\n    if the arguments they passed in do not match any signature.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Two claims removed: line number references and an internal comment about performance rationale (locals() vs inspect.signature.bind) that exists only in comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_11",
    "repo": "matplotlib",
    "question": "Where is the `deprecated` decorator defined, how does it flow through the codebase, and where are its warning stack levels resolved?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `deprecated` decorator is defined in `lib/matplotlib/_api/deprecation.py` and is re-exported through `lib/matplotlib/_api/__init__.py` via `from .deprecation import deprecated`. The decorator's inner `deprecate` function handles three branches depending on what it decorates: (1) classes (wraps `__init__`), (2) properties/classproperties (creates a `_deprecated_property` subclass), (3) regular functions (wraps with `functools.wraps`). The `warn_deprecated` function it calls internally uses `warn_external` (imported from the `_api` package's `__init__.py`), which dynamically calculates a `stacklevel` by walking frames until it finds one outside the `matplotlib` or `mpl_toolkits` namespaces (on Python < 3.12), or uses `skip_file_prefixes` on Python >= 3.12. The module-level `DECORATORS` dictionary stores mappings from wrapper functions to their decorator partials, and this is consumed by `_copy_docstring_and_deprecators` in `lib/matplotlib/pyplot.py` to propagate `rename_parameter` and `make_keyword_only` decorators to pyplot wrappers. The removal version is auto-calculated by `_generate_deprecation_warning` as `'{macro}.{int(meso) + 2}'` from the `since` version string.",
    "rubric": [
      "Identifies that `deprecated` is defined in `lib/matplotlib/_api/deprecation.py`",
      "Notes that it is re-exported through `lib/matplotlib/_api/__init__.py` via `from .deprecation import deprecated`",
      "Explains the three branches in the inner `deprecate` function: classes (wraps `__init__`), properties/classproperties (creates `_deprecated_property` subclass), and regular functions (uses `functools.wraps`)",
      "Explains that `warn_deprecated` calls `warn_external` from `_api.__init__`, which on Python >= 3.12 uses `skip_file_prefixes` and on older versions walks stack frames to find one outside matplotlib/mpl_toolkits",
      "Mentions the module-level `DECORATORS` dictionary that maps wrappers to decorator partials",
      "Explains that `_copy_docstring_and_deprecators` in `pyplot.py` consumes `DECORATORS` to propagate `rename_parameter`/`make_keyword_only` decorators to pyplot wrappers",
      "Notes that the removal version is auto-calculated in `_generate_deprecation_warning` as `'{macro}.{int(meso) + 2}'` from the `since` version string"
    ],
    "key_files": [
      "lib/matplotlib/_api/deprecation.py",
      "lib/matplotlib/_api/__init__.py",
      "lib/matplotlib/pyplot.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/_api/deprecation.py] lib.matplotlib._api.deprecation.deprecated\nDecorator to mark a function, a class, or a property as deprecated.\n\n    When deprecating a classmethod, a staticmethod, or a property, the\n    ``@deprecated`` decorator should go *under* ``@classmethod`` and\n    ``@staticmethod`` (i.e., `deprecated` should directly decorate the\n    underlying callable), but *over* ``@property``.\n\n    When deprecating a class ``C`` intended to be used as a base class in a\n    multiple inheritance hierarchy, ``C`` *must* define an ``__init__`` method\n    (if ``C`` instead inherited its ``__init__`` from its own base class, then\n    ``@deprecated`` would mess up ``__init__`` inheritance when installing its\n    own (deprecation-emitting) ``C.__init__``).\n\n    Parameters are the same as for `warn_deprecated`, except that *obj_type*\n    defaults to 'class' if decorating a class, 'attribute' if decorating a\n    property, and 'function' otherwise.\n\n    Examples\n    --------\n    ::\n\n        @deprecated('1.4.0')\n        def the_function_to_deprecate():\n            pass",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer go far beyond what the provided documentation states \u2014 such as specific line numbers, internal implementation details about `warn_external`, `DECORATORS` dictionary, `_copy_docstring_and_deprecators`, `_generate_deprecation_warning`, and removal version auto-calculation. These cannot be verified from the given documentation chunk alone.",
      "However, none of the claims that overlap with the documentation appear to contradict it. The answer correctly identifies it as a decorator for marking functions, classes, or properties as deprecated, and correctly notes the three branches (classes, properties, functions).",
      "The claim about `_generate_deprecation_warning` computing removal version as `'{macro}.{int(meso) + 2}'` is a very specific implementation detail that cannot be verified from the doc and could be a hallucination or could be accurate \u2014 it's unverifiable from this snippet."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only exact line numbers and a reference to an unprovided file needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_13",
    "repo": "matplotlib",
    "question": "What is the architecture of the rasterization-by-zorder system, including how artists get split, drawn via `_draw_rasterized`, and composited back by the mixed-mode renderer?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `_draw_rasterized` function (defined in `lib/matplotlib/axes/_base.py`) is a module-level helper function used for rasterizing artists below a given zorder in Axes. Here's how it works architecturally:\n\n**Internal Shim Class `_MinimalArtist`**: The function defines an inner class `_MinimalArtist` (not a subclass of `Artist`) that implements the interface used by the `allow_rasterization` decorator. It has four methods/attributes:\n1. `get_rasterized()` \u2014 always returns `True`\n2. `get_agg_filter()` \u2014 always returns `None`\n3. `get_figure(root=False)` \u2014 delegates to the passed figure, with `root=True` support\n4. `draw(renderer)` \u2014 decorated with `@martist.allow_rasterization`, iterates over and draws each artist in the list\n\nThe function instantiates `_MinimalArtist` and immediately calls `.draw(renderer)`, returning the result.\n\n**Integration with `Axes.draw`**: The function is called from `_AxesBase.draw()` when `self._rasterization_zorder` is set (via `set_rasterization_zorder()`). The splitting logic uses `np.searchsorted` on the sorted artist zorders with `side='right'` to partition artists into `artists_rasterized` (below the threshold) and the rest. If `self.axison and self._frameon`, the `self.patch` is prepended to whichever list is used \u2014 rasterized artists get it if there are rasterized artists, otherwise non-rasterized artists get it. `_rasterization_zorder` is initialized to `None` in `_AxesBase.__init__`.\n\n**The `allow_rasterization` decorator**: Defined in `lib/matplotlib/artist.py`, this decorator manages `renderer._raster_depth` (an integer counter) and `renderer._rasterizing` (a boolean flag), both initialized in `RendererBase.__init__` (in `backend_bases.py`). When `get_rasterized()` returns `True` and `_raster_depth` is 0 and not already rasterizing, it calls `renderer.start_rasterizing()`. In the `finally` block, it decrements `_raster_depth` and, if `figure.suppressComposite` is truthy, it stops and immediately restarts rasterizing. The decorated draw_wrapper has `_supports_rasterization = True`.\n\n**Mixed-Mode Rendering**: `MixedModeRenderer` (in `backends/backend_mixed.py`) implements the actual `start_rasterizing()`/`stop_rasterizing()`. `start_rasterizing` creates a `RendererAgg` instance at the mixed-mode DPI and switches `self._renderer` to it. `stop_rasterizing` extracts the raster buffer via `buffer_rgba()`, crops it using `cbook._get_nonzero_slices` on the alpha channel, and draws it as an image on the vector renderer via `draw_image`, scaling coordinates by `self._figdpi / self.dpi`. The figure's DPI is temporarily changed during rasterization and restored afterward.\n\n**Companion decorator `_finalize_rasterization`**: This decorator checks after drawing whether `renderer._rasterizing` is still `True`, and if so, calls `stop_rasterizing()` and sets `_rasterizing` to `False`.\n\n**Companion function `_prevent_rasterization`**: Applied automatically in `Artist.__init_subclass__` to draw methods that don't have `_supports_rasterization` set. If rasterizing is active at depth 0, it stops rasterization before drawing the artist. The decorated wrapper has `_supports_rasterization = False`.",
    "rubric": [
      "Explains that `_draw_rasterized` defines an inner `_MinimalArtist` shim class (not an Artist subclass) with `get_rasterized()` returning True, `get_agg_filter()` returning None, `get_figure()` delegating to the passed figure, and a `draw()` method decorated with `allow_rasterization` that iterates over artists",
      "Describes the splitting logic in `_AxesBase.draw()`: `np.searchsorted` with `side='right'` partitions sorted artists into rasterized (below zorder threshold) and non-rasterized lists, and `self.patch` is prepended to whichever group applies based on `axison` and `_frameon`",
      "Explains the `allow_rasterization` decorator's management of `renderer._raster_depth` and `renderer._rasterizing`: starts rasterizing when depth is 0 and not already rasterizing, increments depth, and in the finally block decrements depth and handles `suppressComposite` by stopping and restarting",
      "Describes `MixedModeRenderer.start_rasterizing()` creating a `RendererAgg` at mixed-mode DPI and `stop_rasterizing()` extracting the buffer, cropping via alpha channel with `_get_nonzero_slices`, and drawing it as an image on the vector renderer with coordinate scaling by `_figdpi / self.dpi`",
      "Mentions `_prevent_rasterization` decorator applied automatically via `Artist.__init_subclass__` to draw methods lacking `_supports_rasterization`, which stops rasterization at depth 0 before drawing",
      "Mentions `_finalize_rasterization` decorator that checks if `renderer._rasterizing` is still True after drawing and stops it if so"
    ],
    "key_files": [
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/artist.py",
      "lib/matplotlib/backends/backend_mixed.py",
      "lib/matplotlib/backend_bases.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/axes/_base.py] lib.matplotlib.axes._base._draw_rasterized\nA helper function for rasterizing the list of artists.\n\n    The bookkeeping to track if we are or are not in rasterizing mode\n    with the mixed-mode backends is relatively complicated and is now\n    handled in the matplotlib.artist.allow_rasterization decorator.\n\n    This helper defines the absolute minimum methods and attributes on a\n    shim class to be compatible with that decorator and then uses it to\n    rasterize the list of artists.\n\n    This is maybe too-clever, but allows us to reuse the same code that is\n    used on normal artists to participate in the \"are we rasterizing\"\n    accounting.\n\n    Please do not use this outside of the \"rasterize below a given zorder\"\n    functionality of Axes.\n\n    Parameters\n    ----------\n    figure : matplotlib.figure.Figure\n        The figure all of the artists belong to (not checked).  We need this\n        because we can at the figure level suppress composition and insert each\n        rasterized artist as its own image.\n\n    artists : List[matplotlib.artist.Artist]\n        The list of Artists to be rasterized.  These are assumed to all\n        be in the same Figure.\n\n    renderer : matplotlib.backendbases.RendererBase\n        The currently active renderer\n\n    Returns\n    -------\n    None",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many specific implementation details (line numbers, method names, internal class structure, etc.) go far beyond what the documentation states and cannot be verified from the provided docstring alone",
      "The claim about `np.searchsorted` with `side='right'` is plausible but not verifiable from the docstring",
      "The claim about `_MinimalArtist` having exactly four methods/attributes with those exact behaviors is not verifiable from the docstring",
      "Line numbers (4962, 5022, 3288, 735, 44, 158-159) cannot be verified from the provided documentation",
      "Details about `MixedModeRenderer`, `_finalize_rasterization`, `_prevent_rasterization`, `buffer_rgba()`, `cbook._get_nonzero_slices`, etc. are all beyond the scope of the docstring"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor purpose/rationale phrases from documentation were removed or adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_15",
    "repo": "matplotlib",
    "question": "Where is `get_glyphs_subset` called from, and how does the data flow into it from character tracking?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `get_glyphs_subset` function in `lib/matplotlib/backends/_backend_pdf_ps.py` is called from two main locations in the data flow:\n\n1. **PDF backend** (`backend_pdf.py`): Called inside `embedTTFType42()`, which is a nested function inside `embedTTF()`. The call chain is: `writeFonts()` \u2192 `embedTTF()` \u2192 `embedTTFType42()` \u2192 `get_glyphs_subset()`. The `writeFonts()` method iterates over `self._fontNames` and retrieves charmaps from `self._character_tracker.used[filename][subset]`, passing `charmap.values()` (the glyph indices) to `embedTTFType42`, which then calls `get_glyphs_subset(filename, charmap.values())`. The returned TTFont is used as a context manager, and `font_as_file(subset)` converts it to a BytesIO.\n\n2. **PS backend** (`backend_ps.py`): Called inside `_font_to_ps_type42()`. The call chain involves iterating over `ps_renderer._character_tracker.used` and calling `_font_to_ps_type42(font, subset, charmap.values(), fh)` for fonttype 42 fonts. Inside that function, `get_glyphs_subset` is called alongside `fontTools.ttLib.TTFont` (for the original font), both as context managers. The result flows to `_serialize_type42()`, which generates PostScript Type-42 format output.\n\nThe data that feeds into `get_glyphs_subset` originates from `CharacterTracker`, which is initialized with a `subset_size` from `_FONT_MAX_GLYPH` (256 for Type 3, 65536 for Type 42). The `track_glyph()` method records font usage using `font_manager.FontPath` objects as keys, and stores glyph indices in charmap dictionaries.\n\nThe `fontfile` parameter is actually a `font_manager.FontPath` instance (a `str` subclass with a `face_index` property), which stores the face index as `_face_index`. The function accesses `fontfile.face_index` to set `options.font_number`.\n\nThe function creates `fontTools.subset.Options` with `glyph_names=True`, `recommended_glyphs=True`, and `retain_gids=True`, appends 20 additional tables to `drop_tables`, then uses `subset.load_font()`, `subset.Subsetter()`, `subsetter.populate(gids=glyphs)`, and `subsetter.subset(font)` to perform the subsetting.",
    "rubric": [
      "Identifies that get_glyphs_subset is called from the PDF backend (backend_pdf.py) inside embedTTFType42, which is nested inside embedTTF",
      "Identifies that get_glyphs_subset is called from the PS backend (backend_ps.py) inside _font_to_ps_type42",
      "Explains that CharacterTracker tracks glyph usage and its used dictionary provides the charmap/glyph indices passed to get_glyphs_subset",
      "Notes that the fontfile parameter is a FontPath instance (str subclass) with a face_index property used to set options.font_number",
      "Describes the subsetting process: Options are created with glyph_names=True, recommended_glyphs=True, retain_gids=True, extra tables dropped, then load_font/Subsetter/populate/subset are called",
      "Mentions that in the PDF backend the call chain goes through writeFonts -> embedTTF -> embedTTFType42 -> get_glyphs_subset",
      "Mentions that CharacterTracker uses a subset_size parameter related to font type (e.g., 256 for Type 3, 65536 for Type 42)"
    ],
    "key_files": [
      "lib/matplotlib/backends/_backend_pdf_ps.py",
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/backends/backend_ps.py",
      "lib/matplotlib/font_manager.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/_backend_pdf_ps.py] lib.matplotlib.backends._backend_pdf_ps.get_glyphs_subset\nSubset a TTF font.\n\n    Reads the named fontfile and restricts the font to the glyphs.\n\n    Parameters\n    ----------\n    fontfile : FontPath\n        Path to the font file\n    glyphs : set[GlyphIndexType]\n        Set of glyph indices to include in subset.\n\n    Returns\n    -------\n    fontTools.ttLib.ttFont.TTFont\n        An open font object representing the subset, which needs to\n        be closed by the caller.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor line number references and one table count error needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_12",
    "repo": "matplotlib",
    "question": "How does `_process_plot_var_args` parse variable-length arguments and produce artists for `plot()` and `fill()`?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `_process_plot_var_args` class handles the parsing and processing of variable-length arguments used by `Axes.plot()` and `Axes.fill()`. Here's how it works in detail:\n\n**Initialization and Output Modes:**\nThe class constructor (`__init__`) accepts an `output` parameter that must be one of `'Line2D'`, `'Polygon'`, or `'coordinates'` (validated via `_api.check_in_list`). In `_AxesBase.__init__`, two instances are created: `self._get_lines = _process_plot_var_args()` (defaulting to `'Line2D'`) and `self._get_patches_for_fill = _process_plot_var_args('Polygon')`. The `'coordinates'` output mode returns raw `(x, y)` tuples instead of Artists, as seen in the `_make_coordinates` method.\n\n**The `__call__` method (the main entry point):**\n1. It first calls `axes._process_unit_info(kwargs=kwargs)` to handle unit conversion.\n2. It explicitly raises a `TypeError` if `'x'` or `'y'` are passed as keyword arguments (using `_api.kwarg_error` with `inspect.stack()[1].function` to get the caller's name).\n3. If `data` is provided, it uses `mpl._replacer(data, arg)` to look up string keys in the data object. When exactly 2 args are given, it determines whether the second arg is a format string or a data key by calling `_process_plot_format` on it \u2014 if the parse succeeds AND a data substitution was made, it emits a `RuntimeWarning` about ambiguity.\n4. When `data` is provided and there are more than 3 args, it raises a `ValueError` because arbitrary long args with data is not supported.\n5. The `__call__` method is a generator that yields artists. It repeatedly slices off 2 elements from `args`, checks if the next element is a string (format spec), and delegates to `_plot_args`.\n\n**Format string parsing (`_process_plot_format`):**\nThis standalone function converts format strings to `(linestyle, marker, color)` tuples. Key details:\n- It first attempts to parse the entire `fmt` as a color using `mcolors.to_rgba(fmt)`, but specifically excludes the strings `\"0\"` and `\"1\"`.\n- Two-character line styles (like `'--'`, `'-.'`) are checked before single characters.\n- `CN` color notation (e.g., `'C2'`) is handled via `re.match(r\"C\\d+\", fmt[i:])` and converted to RGBA.\n- If neither linestyle nor marker is found in the format string, the linestyle defaults to `mpl.rcParams['lines.linestyle']`. If only one of linestyle/marker is absent, the absent one is set to `'None'`.\n\n**Property cycling (`_PropCycle`):**\nThe `_PropCycle` class expands a `Cycler` into an explicit indexed list (`self._cycler_items = [*cycler]`). The `get_next_color` method advances the cycle index only if `\"color\"` exists in the current entry; if the cycle has no color, it returns the fallback `\"k\"` (black) without advancing. The `getdefaults` method only advances the cycle if at least one of the cycle's keys (excluding those in an `ignore` set) is absent or None in the provided `kw` dict.\n\n**2D data handling in `_plot_args`:**\nAfter parsing the format string, `_plot_args` handles multi-column data: 1D arrays are reshaped to column vectors (`x[:, np.newaxis]`). The number of datasets (`n_datasets`) is `max(ncx, ncy)`, and column indexing uses modular arithmetic (`x[:, j % ncx]`) to broadcast a single x column across multiple y columns. If both x and y have more than 1 column and they differ, a `ValueError` is raised.\n\n**Polygon-specific behavior in `_make_polygon`:**\nWhen output is `'Polygon'`, several marker-related properties (`'marker'`, `'markersize'`, `'markeredgecolor'`, `'markerfacecolor'`, `'markeredgewidth'`) are explicitly ignored from cycle defaults. The `'color'` from the cycle is translated to `facecolor` rather than being passed directly. The polygon is created using `np.column_stack((x, y))`, and the `'closed'` kwarg defaults to `True`.\n\n**Integration with `Axes.plot()`:**\nIn `Axes.plot()` (in `_axes.py`), kwargs are first normalized via `cbook.normalize_kwargs(kwargs, mlines.Line2D)`, then `self._get_lines(self, *args, data=data, **kwargs)` is called (which invokes `__call__`). The returned lines are added via `self.add_line(line)`, and autoscaling is requested afterward.",
    "rubric": [
      "Mentions the class supports multiple output modes ('Line2D', 'Polygon', 'coordinates') set at initialization and validated with check_in_list",
      "Explains that __call__ is a generator that repeatedly slices off (x, y) or (x, y, fmt) groups from args and delegates to _plot_args",
      "Describes the data keyword handling: string args are looked up in the data object via _replacer, with ambiguity detection for 2-arg case emitting a RuntimeWarning",
      "Explains _process_plot_format parses format strings into (linestyle, marker, color) tuples, handling CN colors, two-char line styles before single chars, and defaulting to rcParams linestyle when neither linestyle nor marker is found",
      "Describes how _plot_args handles 2D data by reshaping 1D arrays to column vectors and using modular column indexing (j % ncx) to broadcast",
      "Explains _PropCycle's get_next_color returns 'k' without advancing if the cycle has no color entry, and getdefaults only advances if needed keys are missing/None",
      "Describes _make_polygon's behavior: ignoring marker-related properties, translating 'color' to facecolor, and defaulting 'closed' to True"
    ],
    "key_files": [
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/axes/_axes.py",
      "lib/matplotlib/cbook.py",
      "lib/matplotlib/__init__.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/axes/_base.py] lib.matplotlib.axes._base._process_plot_var_args\nlib.matplotlib.axes._base._process_plot_var_args:\n  Process variable length arguments to `~.Axes.plot`, to support ::\n\n      plot(t, s)\n      plot(t1, s1, t2, s2)\n      plot(t1, s1, 'ko', t2, s2)\n      plot(t1, s1, 'ko', t2, s2, 'r--', t3, e3)\n\n    an arbitrary number of *x*, *y*, *fmt* are allowed\n\nlib.matplotlib.axes._base._process_plot_var_args.get_next_color:\n  Return the next color in the cycle.\n\nlib.matplotlib.axes._base._process_plot_var_args._setdefaults:\n  Add to the dict *kw* the entries in the dict *default* that are absent\n        or set to None in *kw*.\n\nlib.matplotlib.axes._base._process_plot_var_args._plot_args:\n  Process the arguments of ``plot([x], y, [fmt], **kwargs)`` calls.\n\n        This processes a single set of ([x], y, [fmt]) parameters; i.e. for\n        ``plot(x, y, x2, y2)`` it will be called twice. Once for (x, y) and\n        once for (x2, y2).\n\n        x and y may be 2D and thus can still represent multiple datasets.\n\n        For multiple datasets, if the keyword argument *label* is a list, this\n        will unpack the list and assign the individual labels to the datasets.\n\n        Parameters\n        ----------\n        tup : tuple\n            A tuple of the positional parameters. This can be one of\n\n            - (y,)\n            - (x, y)\n            - (y, fmt)\n            - (x, y, fmt)\n\n        kwargs : dict\n            The keyword arguments passed to ``plot()``.\n\n        return_kwargs : bool\n            Whether to also return the effective keyword arguments after label\n            unpacking as well.\n\n        ambiguous_fmt_datakey : bool\n            Whether the format string in *tup* could also have been a\n            misspelled data key.\n\n        Returns\n        -------\n        result\n            If *return_kwargs* is false, a list of Artists representing the\n            dataset(s).\n            If *return_kwargs* is true, a list of (Artist, effective_kwargs)\n            representing the dataset(s). See *return_kwargs*.\n            The Artist is either `.Line2D` (if called from ``plot()``) or\n            `.Polygon` otherwise.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes `_process_plot_var_args` as processing variable length arguments for `Axes.plot()`, but the answer extensively describes `Axes.fill()` integration, `'Polygon'` and `'coordinates'` output modes, `_PropCycle` internals, `_process_plot_format` details, `_make_polygon` behavior, and many other implementation details that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the given doc chunk.",
      "The answer states the Artist is 'either `.Line2D` (if called from ``plot()``) or `.Polygon` otherwise', which is consistent with the documentation's Returns section. However, it also claims a `'coordinates'` output mode that returns raw `(x, y)` tuples \u2014 this is not mentioned in the documentation at all.",
      "The answer claims `get_next_color` returns fallback `'k'` (black) when the cycle has no color \u2014 the documentation only says it 'Return[s] the next color in the cycle' without specifying fallback behavior.",
      "The answer's description of `_plot_args` parameters and return values is consistent with the documentation, including the tuple forms `(y,)`, `(x, y)`, `(y, fmt)`, `(x, y, fmt)` and the return behavior based on `return_kwargs`.",
      "The answer's description of `_setdefaults` behavior is consistent with the documentation: 'Add to the dict *kw* the entries in the dict *default* that are absent or set to None in *kw*'."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; minor rewrites needed for documentation-only explanations like 'MATLAB-style' and rationale for marker property exclusion.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_14",
    "repo": "matplotlib",
    "question": "Why does FigureManagerBase use two separate toolbar systems and how does it decide between them at initialization?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `FigureManagerBase` class is designed as a backend-independent adapter layer between pyplot and GUI frameworks, with several key design decisions evident in the source code:\n\n**1. Dual toolbar system via rcParams-driven branching:**\nThe `__init__` method contains a three-way conditional split based on `rcParams['toolbar']`. When the toolbar setting is NOT `'toolmanager'`, it connects three event handlers (`key_press_handler`, `button_press_handler`, `scroll_handler`) directly. When it IS `'toolmanager'`, it creates a `ToolManager` instance instead and delegates tool registration to `tools.add_tools_to_manager()` and `tools.add_tools_to_container()`. This is why the key/button/scroll handler ids are set to `None` when using `toolmanager` mode \u2014 the toolmanager handles its own event routing. The class attributes `_toolbar2_class = None` and `_toolmanager_toolbar_class = None` allow each GUI backend to opt into one or both toolbar systems by setting these on their subclass.\n\n**2. `scroll_handler_id` attribute:**\nThe `__init__` registers a `scroll_handler` for `'scroll_event'` and stores it in `self.scroll_handler_id`. The `scroll_handler` implements ctrl/x/y + scroll-wheel zoom on rectilinear axes, using a scale factor of `0.85 ** event.step` and requiring a toolbar to be present for the zoom to work.\n\n**3. `_window_title` design choice in `set_window_title`:**\nGUI backends like Tk override `set_window_title` to call `self.window.wm_title(title)` and `get_window_title` to call `self.window.wm_title()`, never using the `_window_title` attribute.\n\n**4. Headless detection in `show()`:**\nThe base `show()` silently returns (instead of raising `NonGuiException`) only when `sys.platform == 'linux'` AND `os.environ.get('DISPLAY')` is falsy.\n\n**5. Interactive backend detection via method identity comparison:**\n`_Backend.draw_if_interactive` determines whether a backend is interactive by checking if the manager class has overridden `start_main_loop` or `pyplot_show` compared to `FigureManagerBase`'s versions: `manager_class.start_main_loop != FigureManagerBase.start_main_loop`. This means simply subclassing without overriding either method means the backend is treated as non-interactive.\n\n**6. `destroy()` calls `_set_base_canvas()` to restore original canvas:**\nThe base `destroy()` method calls `self.canvas.figure._set_base_canvas()`. The `_set_base_canvas` method in `figure.py` creates a new `FigureCanvasBase(self)` (which sets `self.canvas` as a side effect) and resets DPI to `_original_dpi`.\n\n**7. `Gcf._set_new_active_manager` adds a hidden `_cidgcf` button_press callback:**\nWhen a manager is adopted by pyplot via `Gcf._set_new_active_manager`, it connects a `button_press_event` callback that calls `Gcf.set_active(manager)` \u2014 making any figure become the \"active\" figure when clicked. This `_cidgcf` attribute is later disconnected in `Gcf.destroy()` and `Gcf.destroy_all()`.\n\n**8. Axes observer pattern for toolbar updates:**\nThe `__init__` registers an axes observer via `self.canvas.figure.add_axobserver` that calls `self.toolbar.update()` whenever axes change \u2014 but only when NOT using the toolmanager (`if self.toolmanager is None and self.toolbar is not None`). The toolmanager handles its own axes change notifications.\n\n**9. `create_with_canvas` and `new_manager` relationship:**\n`FigureCanvasBase.new_manager(figure, num)` delegates to `cls.manager_class.create_with_canvas(cls, figure, num)`. The Tk backend's override of `create_with_canvas` shows the customization point: it needs to create a `tk.Tk()` window, set up DPI awareness, load icon images, and conditionally call `manager.show()` if interactive.\n\n**10. `pyplot_show` IPython hack:**\nThe `block=None` logic in `pyplot_show` checks for IPython's `%pylab` mode by inspecting whether `matplotlib.pyplot.show` has a `_needmain` attribute (which IPython tacks on). If that attribute exists or interactive mode is on, blocking is disabled.",
    "rubric": [
      "Explains that rcParams['toolbar'] drives a conditional split between 'toolbar2' and 'toolmanager' modes",
      "Notes that in non-toolmanager mode, key_press_handler, button_press_handler, and scroll_handler are connected directly to canvas events",
      "Notes that in toolmanager mode, a ToolManager instance is created and tools are registered via add_tools_to_manager/add_tools_to_container",
      "Explains that handler IDs are set to None in toolmanager mode because the toolmanager handles its own event routing",
      "Mentions the _toolbar2_class and _toolmanager_toolbar_class class attributes that allow GUI backends to opt into one or both systems",
      "Mentions the axobserver that calls toolbar.update() only when NOT using toolmanager mode",
      "Notes that draw_if_interactive determines interactivity by checking if start_main_loop or pyplot_show has been overridden from FigureManagerBase"
    ],
    "key_files": [
      "lib/matplotlib/backend_bases.py",
      "lib/matplotlib/_pylab_helpers.py",
      "lib/matplotlib/figure.py",
      "lib/matplotlib/backends/_backend_tk.py",
      "lib/matplotlib/backend_tools.py",
      "lib/matplotlib/pyplot.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backend_bases.py] lib.matplotlib.backend_bases.FigureManagerBase\nlib.matplotlib.backend_bases.FigureManagerBase:\n  A backend-independent abstraction of a figure container and controller.\n\n    The figure manager is used by pyplot to interact with the window in a\n    backend-independent way. It's an adapter for the real (GUI) framework that\n    represents the visual figure on screen.\n\n    The figure manager is connected to a specific canvas instance, which in turn\n    is connected to a specific figure instance. To access a figure manager for\n    a given figure in user code, you typically use ``fig.canvas.manager``.\n\n    GUI backends derive from this class to translate common operations such\n    as *show* or *resize* to the GUI-specific code. Non-GUI backends do not\n    support these operations and can just use the base class.\n\n    This following basic operations are accessible:\n\n    **Window operations**\n\n    - `~.FigureManagerBase.show`\n    - `~.FigureManagerBase.destroy`\n    - `~.FigureManagerBase.full_screen_toggle`\n    - `~.FigureManagerBase.resize`\n    - `~.FigureManagerBase.get_window_title`\n    - `~.FigureManagerBase.set_window_title`\n\n    **Key and mouse button press handling**\n\n    The figure manager sets up default key and mouse button press handling by\n    hooking up the `.key_press_handler` to the matplotlib event system. This\n    ensures the same shortcuts and mouse actions across backends.\n\n    **Other operations**\n\n    Subclasses will have additional attributes and functions to access\n    additional functionality. This is of course backend-specific. For example,\n    most GUI backends have ``window`` and ``toolbar`` attributes that give\n    access to the native GUI widgets of the respective framework.\n\n    Attributes\n    ----------\n    canvas : `FigureCanvasBase`\n        The backend-specific canvas instance.\n\n    num : int or str\n        The figure number.\n\n    key_press_handler_id : int\n        The default key handler cid, when using the toolmanager.\n        To disable the default key press handling use::\n\n            figure.canvas.mpl_disconnect(\n                figure.canvas.manager.key_press_handler_id)\n\n    button_press_handler_id : int\n        The default mouse button handler cid, when using the toolmanager.\n        To disable the default button press handling use::\n\n            figure.canvas.mpl_disconnect(\n                figure.canvas.manager.button_press_handler_id)\n\nlib.matplotlib.backend_bases.FigureManagerBase.create_with_canvas:\n  Create a manager for a given *figure* using a specific *canvas_class*.\n\n        Backends should override this method if they have specific needs for\n        setting up the canvas or the manager.\n\nlib.matplotlib.backend_bases.FigureManagerBase.start_main_loop:\n  Start the main event loop.\n\n        This method is called by `.FigureManagerBase.pyplot_show`, which is the\n        implementation of `.pyplot.show`.  To customize the behavior of\n        `.pyplot.show`, interactive backends should usually override\n        `~.FigureManagerBase.start_main_loop`; if more customized logic is\n        necessary, `~.FigureManagerBase.pyplot_show` can also be overridden.\n\nlib.matplotlib.backend_bases.FigureManagerBase.pyplot_show:\n  Show all figures.  This method is the implementation of `.pyplot.show`.\n\n        To customize the behavior of `.pyplot.show`, interactive backends\n        should usually override `~.FigureManagerBase.start_main_loop`; if more\n        customized logic is necessary, `~.FigureManagerBase.pyplot_show` can\n        also be overridden.\n\n        Parameters\n        ----------\n        block : bool, optional\n            Whether to block by calling ``start_main_loop``.  The default,\n            None, means to block if we are neither in IPython's ``%pylab`` mode\n            nor in ``interactive`` mode.\n\nlib.matplotlib.backend_bases.FigureManagerBase.show:\n  For GUI backends, show the figure window and redraw.\n        For non-GUI backends, raise an exception, unless running headless (i.e.\n        on Linux with an unset DISPLAY); this exception is converted to a\n        warning in `.Figure.show`.\n\nlib.matplotlib.backend_bases.FigureManagerBase.resize:\n  For GUI backends, resize the window (in physical pixels).\n\nlib.matplotlib.backend_bases.FigureManagerBase.get_window_title:\n  Return the title text of the window containing the figure.\n\nlib.matplotlib.backend_bases.FigureManagerBase.set_window_title:\n  Set the title text of the window containing the figure.\n\n        Examples\n        --------\n        >>> fig = plt.figure()\n        >>> fig.canvas.manager.set_window_title('My figure')",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 2 mentions 'scroll_handler_id' as undocumented - this is plausible but not verifiable from the provided documentation",
      "Point 3 claims specific implementation details about '_window_title' and comments in source code that cannot be verified from the documentation",
      "Point 4 claims specific platform detection logic in show() - the documentation only says 'unless running headless (i.e. on Linux with an unset DISPLAY)' which is consistent but the specific code comment about _get_running_interactive_framework is unverifiable",
      "Point 5 claims interactive backend detection via method identity comparison - not verifiable from documentation",
      "Point 6 claims destroy() calls _set_base_canvas() - not verifiable from documentation",
      "Point 7 claims about Gcf._set_new_active_manager and _cidgcf - not verifiable from documentation",
      "Point 8 claims about axes observer pattern - not verifiable from documentation",
      "Point 9 claims about create_with_canvas and new_manager relationship - partially supported by documentation mentioning 'Backends should override this method if they have specific needs'",
      "Point 10 claims pyplot_show checks for IPython's %pylab mode via '_needmain' attribute - the documentation mentions 'IPython's %pylab mode' in the block parameter description but doesn't specify the mechanism",
      "The documentation states key_press_handler_id and button_press_handler_id are attributes 'when using the toolmanager' but Point 1 claims they are set to None when using toolmanager - this may contradict the documentation which presents them as active handler IDs"
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rely on comments/docstrings explaining design rationale that would be stripped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_16",
    "repo": "matplotlib",
    "question": "How does `font_as_file` work, and how is it used by the PDF and PS backends to bridge fontTools subsetting with font embedding?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `font_as_file` function in `lib/matplotlib/backends/_backend_pdf_ps.py` converts a fontTools `TTFont` object into a `BytesIO` file-like object by calling `font.save(fh, reorderTables=False)` on a newly created `BytesIO` instance and returning it. The `reorderTables=False` parameter is passed to fontTools during serialization.\n\n**Implementation details:**\n1. Creates a new `BytesIO()` instance.\n2. Calls `font.save(fh, reorderTables=False)` to serialize the TTFont into the BytesIO buffer.\n3. Returns the BytesIO object (notably without seeking back to position 0).\n\n**How it fits into the system:**\n\n`font_as_file` is the bridge between fontTools' `TTFont` objects (produced by `get_glyphs_subset`) and matplotlib's own font handling (FT2Font) and output streams. It is used in two backend pipelines:\n\n- **PDF backend (`backend_pdf.py`)**: The returned `BytesIO` is passed to `FT2Font(fontdata)` to reload the subsetted font, and `.getvalue()` provides the raw bytes for the PDF output.\n\n- **PS backend (`backend_ps.py`, `_font_to_ps_type42`)**: `.getvalue()` is immediately called on the returned BytesIO to get raw bytes, which are then passed to `_serialize_type42()` to produce a PostScript Type 42 font representation, where the raw font data is split into sfnts strings of at most 65534 bytes.\n\nThe input `TTFont` typically comes from `get_glyphs_subset`, which uses `fontTools.subset` to create a subsetted font. That function sets `glyph_names=True`, `recommended_glyphs=True`, and `retain_gids=True` in the subset options, and drops extra tables (like FFTM, PfEd, BDF, meta, MERG, TSIV, Zapf, bdat, bloc, cidg, fdsc, feat, fmtx, fond, just, kerx, ltag, morx, trak, xref). The `get_glyphs_subset` return value is used as a context manager (TTFont supports `__enter__`/`__exit__`).\n\nIn tests (`test_backend_pdf.py::test_glyphs_subset`), the BytesIO returned by `font_as_file` is passed directly to `FT2Font()` constructor, confirming that FT2Font can accept file-like objects, not just file paths.",
    "rubric": [
      "Explains that font_as_file creates a BytesIO instance, calls font.save(fh, reorderTables=False), and returns the BytesIO object",
      "Notes that the BytesIO is not seeked back to position 0 before returning",
      "Describes PDF backend usage: the returned BytesIO is passed to FT2Font() constructor to reload the subsetted font, and .getvalue()/.getbuffer() is used for raw bytes",
      "Describes PS backend usage: .getvalue() is called on the returned BytesIO to get raw bytes for Type 42 font serialization",
      "Mentions that the input TTFont typically comes from get_glyphs_subset which performs font subsetting with options like retain_gids=True and drops extra tables",
      "Notes that get_glyphs_subset is used as a context manager (with statement) around the font_as_file call"
    ],
    "key_files": [
      "lib/matplotlib/backends/_backend_pdf_ps.py",
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/backends/backend_ps.py",
      "lib/matplotlib/tests/test_backend_pdf.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/_backend_pdf_ps.py] lib.matplotlib.backends._backend_pdf_ps.font_as_file\nConvert a TTFont object into a file-like object.\n\n    Parameters\n    ----------\n    font : fontTools.ttLib.ttFont.TTFont\n        A font object\n\n    Returns\n    -------\n    BytesIO\n        A file object with the font saved into it",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in explaining the purpose of reorderTables=False and the rationale for dropping tables; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_19",
    "repo": "matplotlib",
    "question": "Where does the PostScript backend handle Type 42 font embedding, and what helper functions does it call to serialize the font data?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `_serialize_type42` function is located in `lib/matplotlib/backends/backend_ps.py`. It is called from `_font_to_ps_type42`, which is called during PostScript output generation when the fonttype is not 3 (i.e., Type 42 font embedding).\n\nThe call chain flows as follows:\n1. The PS backend's page printing logic iterates over fonts/subsets tracked by `ps_renderer._character_tracker.used`\n2. For fonttype 42, it calls `_font_to_ps_type42`\n3. `_font_to_ps_type42` opens the original font via `fontTools.ttLib.TTFont` and creates a subset via `_backend_pdf_ps.get_glyphs_subset`, then gets raw font bytes via `_backend_pdf_ps.font_as_file(subset).getvalue()`\n4. It passes both the original `font` and `subset` objects, plus `fontdata` bytes, to `_serialize_type42`\n\n`_serialize_type42` calls these helper functions:\n- `_version_and_breakpoints(font.get('loca'), fontdata)` \u2014 parses the TrueType font header using `struct.unpack('>3h', fontdata[:6])` to get version and numTables, reads table offsets from the table directory (each entry is 16 bytes starting at offset 12), and combines table offsets with glyph offsets from the loca table to produce breakpoints\n- `_generate_charstrings(subset)` \u2014 creates a PostScript `/CharStrings` dictionary mapping glyph names to indices from the subset font's glyph order\n- `_generate_sfnts(fontdata, subset, breakpoints)` \u2014 splits the raw font data into hex-encoded strings of at most 65534 bytes (using `bisect.bisect_left`), each NUL-terminated with `00`, and wraps lines at 100 characters\n- `_bounds(font)` \u2014 uses `fontTools.pens.boundsPen.BoundsPen` to compute the combined bounding box of all glyphs, returning `(0, 0, 0, 0)` if `pen.bounds` is None\n\nThe output PostScript defines a 10-element dictionary with FontType 42, FontMatrix as identity `[1 0 0 1 0 0]`, FontName derived from name table ID 6 suffixed with `-{subset_index}`, PaintType 0, CIDMap 0, and StandardEncoding. The `CharacterTracker` used by the PS renderer is configured with `_FONT_MAX_GLYPH[42] = 65536` max glyphs per subset for Type 42 fonts.",
    "rubric": [
      "Identifies that `_serialize_type42` is located in `lib/matplotlib/backends/backend_ps.py`",
      "Explains the call chain: page printing logic iterates over `ps_renderer._character_tracker.used`, calls `_font_to_ps_type42`, which calls `_serialize_type42`",
      "Describes that `_font_to_ps_type42` opens the font via `fontTools.ttLib.TTFont` and creates a subset via `_backend_pdf_ps.get_glyphs_subset`, then gets raw bytes via `_backend_pdf_ps.font_as_file`",
      "Identifies helper `_version_and_breakpoints` which parses TrueType header via `struct.unpack('>3h', fontdata[:6])` and combines table offsets with loca glyph offsets into breakpoints",
      "Identifies helper `_generate_charstrings` which maps glyph names to indices from the subset font's glyph order",
      "Identifies helper `_generate_sfnts` which splits font data into hex-encoded strings of at most 65534 bytes using `bisect.bisect_left`, each NUL-terminated",
      "Identifies helper `_bounds` which uses `BoundsPen` to compute the combined bounding box of all glyphs",
      "Notes the output PostScript defines a 10-element dictionary with FontType 42, FontMatrix identity, FontName from name table ID 6 suffixed with subset index, PaintType 0, CIDMap 0, and StandardEncoding"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_ps.py",
      "lib/matplotlib/backends/_backend_pdf_ps.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_ps.py] lib.matplotlib.backends.backend_ps._serialize_type42\nOutput a PostScript Type-42 format representation of font\n\n    Parameters\n    ----------\n    font : fontTools.ttLib.ttFont.TTFont\n        The original font object\n    subset_index : int\n        The subset of the above font to be created.\n    subset : fontTools.ttLib.ttFont.TTFont\n        The subset font object\n    fontdata : bytes\n        The raw font data in TTF format\n\n    Returns\n    -------\n    str\n        The Type-42 formatted font",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor line number references and truncated call-site details needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_20",
    "repo": "matplotlib",
    "question": "How does the PostScript backend split TrueType font data into sfnts substrings, and how are valid breakpoints computed?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `_version_and_breakpoints` function in `lib/matplotlib/backends/backend_ps.py` implements the following algorithm:\n\n1. **Parse the font header**: It reads the first 6 bytes of `fontdata` using `struct.unpack('>3h', fontdata[:6])` to extract the major version (`v1`), minor version (`v2`), and `numTables` as big-endian signed shorts.\n\n2. **Build a table offset dictionary**: It iterates `numTables` times, reading 16-byte table directory entries starting at byte offset 12. Each entry is unpacked with format `'>4sIII'` (4-byte tag string, then 3 unsigned ints: checksum, offset, length). Only the tag and offset are kept; the tag is decoded from ASCII and used as the dictionary key.\n\n3. **Compute glyph breakpoints**: If `loca` is not None, the function computes glyph-level breakpoints within the `glyf` table by adding the `glyf` table's offset to each location in `loca.locations[:-1]` (all entries except the last one). This creates a set of absolute byte offsets into `fontdata` where individual glyphs begin within the glyf table. If `loca` is None, this set is empty.\n\n4. **Merge and sort all breakpoints**: It combines three sets: all table offsets (the values of the `tables` dict), the glyph breakpoints, and `len(fontdata)` (the end of the font data). These are sorted into a list.\n\n5. **Return**: Returns `(version, breakpoints)` where version is `(v1, v2)` tuple.\n\n**How breakpoints are consumed** (`_generate_sfnts`): The caller `_generate_sfnts` uses `bisect.bisect_left(breakpoints, pos + 65534)` to find the nearest valid breakpoint that keeps each substring under 65534 bytes, then takes `breakpoints[i-1]` as the split point. If that would not advance (`newpos <= pos`), it falls back to `breakpoints[-1]` (the end of the font). Each substring is hex-encoded and NUL-terminated with `'00'` appended. The final string is wrapped at 100 characters per line.\n\n**Call chain**: `_font_to_ps_type42` opens both the original TTFont and a subset via `_backend_pdf_ps.get_glyphs_subset`, saves the subset via `_backend_pdf_ps.font_as_file` (which calls `font.save(fh, reorderTables=False)`), then calls `_serialize_type42` which calls `_version_and_breakpoints(font.get('loca'), fontdata)` \u2014 notably passing the *original* font's `loca` table, not the subset's. The resulting breakpoints and version are used to build a complete PostScript Type 42 font definition that includes FontType 42, a FontMatrix of `[1 0 0 1 0 0]`, the CharStrings dictionary, and the sfnts array.",
    "rubric": [
      "Explains that _version_and_breakpoints parses the first 6 bytes of fontdata as three big-endian signed shorts (version major, minor, numTables)",
      "Explains that table directory entries are read in 16-byte chunks starting at offset 12, extracting tag and offset into a dictionary",
      "Explains that glyph breakpoints are computed by adding the glyf table offset to each entry in loca.locations[:-1] (excluding the last location)",
      "Explains that all breakpoints are the union of table offsets, glyph breakpoints, and len(fontdata), sorted into a list",
      "Explains that _generate_sfnts uses bisect_left(breakpoints, pos + 65534) to find a split point that keeps substrings under 65534 bytes",
      "Explains the fallback: if newpos <= pos (no advancement), it uses breakpoints[-1] (end of font data)",
      "Mentions that each substring is hex-encoded and NUL-terminated with '00' appended",
      "Notes that _font_to_ps_type42 passes the original font's loca table but uses the subset's fontdata"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_ps.py",
      "lib/matplotlib/backends/_backend_pdf_ps.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_ps.py] lib.matplotlib.backends.backend_ps._version_and_breakpoints\nRead the version number of the font and determine sfnts breakpoints.\n\n    When a TrueType font file is written as a Type 42 font, it has to be\n    broken into substrings of at most 65535 bytes. These substrings must\n    begin at font table boundaries or glyph boundaries in the glyf table.\n    This function determines all possible breakpoints and it is the caller's\n    responsibility to do the splitting.\n\n    Helper function for _font_to_ps_type42.\n\n    Parameters\n    ----------\n    loca : fontTools.ttLib._l_o_c_a.table__l_o_c_a or None\n        The loca table of the font if available\n    fontdata : bytes\n        The raw data of the font\n\n    Returns\n    -------\n    version : tuple[int, int]\n        A 2-tuple of the major version number and minor version number.\n    breakpoints : list[int]\n        The breakpoints is a sorted list of offsets into fontdata; if loca is not\n        available, just the table boundaries.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states breakpoints include 'table boundaries or glyph boundaries in the glyf table', and the answer's description of computing glyph breakpoints from loca.locations[:-1] is plausible but not explicitly stated in the doc.",
      "The answer claims the substring limit is 65534 bytes rather than the documented 65535 bytes - this could be a minor discrepancy (the doc says 'at most 65535 bytes' while the answer says bisect uses 'pos + 65534').",
      "The answer claims that _font_to_ps_type42 passes the *original* font's loca table rather than the subset's - this is an implementation detail not verifiable from the documentation and could be a hallucination.",
      "The answer's claim about FontMatrix being [1 0 0 1 0 0] is an implementation detail not in the documentation.",
      "Many implementation details (struct formats, _generate_sfnts behavior, _serialize_type42, etc.) go well beyond what the documentation states and cannot be verified from it."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are directly verifiable from the stripped source code; no documentation leakage detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_18",
    "repo": "matplotlib",
    "question": "Why is `_create_pdf_info_dict` designed the way it is \u2014 what design goals does its structure serve across backends and user customization?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `_create_pdf_info_dict` function in `backend_pdf.py` performs the following:\n\n**1. SOURCE_DATE_EPOCH support:** The function reads the `SOURCE_DATE_EPOCH` environment variable. When set, the epoch value is converted to a UTC datetime using `datetime.fromtimestamp(int(source_date_epoch), timezone.utc)` and then reassigned with `source_date.replace(tzinfo=UTC)` where `UTC` is imported from `matplotlib.dates` (which is `datetime.timezone.utc`). When not set, it falls back to `datetime.today()`.\n\n**2. Cross-backend reuse:** The function is shared between two backends \u2014 the native PDF backend (`backend_pdf.py`) and the PGF/LaTeX backend (`backend_pgf.py`). The `backend` parameter controls the Producer string format: `'Matplotlib {backend} backend v{mpl.__version__}'`. In `backend_pdf.py`, it's called as `_create_pdf_info_dict('pdf', metadata or {})` in the `PdfFile.__init__` constructor. In `backend_pgf.py`, it's called as `_create_pdf_info_dict('pgf', metadata or {})`. The PGF backend also imports the companion `_datetime_to_pdf` function from `backend_pdf` to convert datetime objects to PDF date strings.\n\n**3. Defensive validation with informative warnings:** The function defines three inline validator functions (`is_string_like`, `is_date`, `check_trapped`), each with a `text_for_warning` attribute attached to provide human-readable error descriptions. The nine recognized PDF info dictionary keys are: `Title`, `Author`, `Subject`, `Keywords`, `Creator`, `Producer`, `CreationDate`, `ModDate`, and `Trapped`. Unknown keys and type mismatches trigger warnings via `_api.warn_external`, which is a `warnings.warn` wrapper that automatically adjusts `stacklevel` to point outside Matplotlib's own code.\n\n**4. Trapped key special handling:** The `Trapped` key has a special validator `check_trapped` that accepts either a `Name` object with bytes `b'True'`, `b'False'`, or `b'Unknown'`, or plain strings `'True'`, `'False'`, `'Unknown'`. After validation, if `Trapped` is present, it's always converted to a `Name` object via `info['Trapped'] = Name(info['Trapped'])`. The `Name` class uses `__slots__ = ('name',)` and translates non-printable characters to hex-escaped PDF name format.\n\n**5. None-value removal for opt-out:** After merging defaults with user metadata via `{**defaults, **metadata}`, the function filters out any keys with `None` values using a dictionary comprehension: `info = {k: v for (k, v) in info.items() if v is not None}`. This allows users to suppress the default `Creator`, `Producer`, or `CreationDate` entries by explicitly setting them to `None`.\n\n**6. Downstream serialization:** The returned dict is stored as `self.infoDict` on the `PdfFile` object. The `pdfRepr` function handles serializing `datetime` objects by calling `_datetime_to_pdf(d)` which formats dates as `D:YYYYMMDDHHMMSStz` with timezone offset in PDF format (e.g., `Z` for UTC). In the PGF backend, the `_metadata_to_str` function handles converting metadata values to string format for LaTeX output.",
    "rubric": [
      "Explains that the function is shared between the PDF and PGF backends, using a `backend` parameter to customize the Producer string",
      "Explains SOURCE_DATE_EPOCH support for reproducible builds, converting the epoch to a UTC datetime",
      "Explains that setting metadata values to None allows users to opt out of default Creator/Producer/CreationDate entries (filtered via dict comprehension)",
      "Explains the validation system with inline validators that have `text_for_warning` attributes, warning on unknown keys or type mismatches via `_api.warn_external`",
      "Explains special handling of the Trapped key (accepts Name or string, converts to Name object after validation)",
      "Mentions the recognized PDF info dictionary keys (Title, Author, Subject, Keywords, Creator, Producer, CreationDate, ModDate, Trapped)"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/backends/backend_pgf.py",
      "lib/matplotlib/tests/test_backend_pdf.py",
      "lib/matplotlib/_api/__init__.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_pdf.py] lib.matplotlib.backends.backend_pdf._create_pdf_info_dict\nCreate a PDF infoDict based on user-supplied metadata.\n\n    A default ``Creator``, ``Producer``, and ``CreationDate`` are added, though\n    the user metadata may override it. The date may be the current time, or a\n    time set by the ``SOURCE_DATE_EPOCH`` environment variable.\n\n    Metadata is verified to have the correct keys and their expected types. Any\n    unknown keys/types will raise a warning.\n\n    Parameters\n    ----------\n    backend : str\n        The name of the backend to use in the Producer value.\n\n    metadata : dict[str, Union[str, datetime, Name]]\n        A dictionary of metadata supplied by the user with information\n        following the PDF specification, also defined in\n        `~.backend_pdf.PdfPages` below.\n\n        If any value is *None*, then the key will be removed. This can be used\n        to remove any pre-defined values.\n\n    Returns\n    -------\n    dict[str, Union[str, datetime, Name]]\n        A validated dictionary of metadata.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from design rationale statements ('reproducible builds specification') and line number references; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_17",
    "repo": "matplotlib",
    "question": "What is the architecture of the AGG renderer, and how does it delegate drawing operations between Python and C++?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `RendererAgg` class in `lib/matplotlib/backends/backend_agg.py` is a concrete renderer that inherits from `RendererBase` (defined in `lib/matplotlib/backend_bases.py`). It wraps a C++ `_RendererAgg` object (from the pybind11 extension module `matplotlib.backends._backend_agg`) which uses the AGG library (as evidenced by `agg_*` header includes) for rasterization.\n\n**Key architectural details:**\n\n1. **Delegation Pattern via `_update_methods`**: The `_update_methods()` method directly assigns C-level methods from `self._renderer` (the C++ `_RendererAgg` object) to instance attributes: `draw_gouraud_triangles`, `draw_image`, `draw_markers`, `draw_path_collection`, `draw_quad_mesh`, and `copy_from_bbox`. This replaces the Python method lookup with direct calls to the C++ implementation.\n\n2. **C++ Backend Internals**: The C++ `RendererAgg` (in `src/_backend_agg.cpp`) allocates a pixel buffer (`pixBuffer`) of size `width * height * 4` bytes (RGBA). It initializes with a fill color of `agg::rgba(1, 1, 1, 0)` (white, fully transparent). The rasterizer is initialized with a cell block limit of 32768. The maximum image dimension is limited to `2^23` pixels in each direction. The hatch buffer size equals `int(dpi)` squared pixels.\n\n3. **MathTextParser Usage**: `RendererAgg` instantiates `MathTextParser('path')` which uses output type `\"vector\"`. When `parse()` is called, it returns a `VectorParse` namedtuple containing `width`, `height`, `depth`, `glyphs` (list of `(font, fontsize, char_code, glyph_index, dx, dy)` tuples), and `rects` (list of `(x, y, w, h)` tuples). The `draw_mathtext` method passes slant=0 and extend=1 for all glyphs.\n\n4. **`_prepare_font` Method**: Calls `get_font(_fontManager._find_fonts_by_props(font_prop))` which returns an `FT2Font` instance. After getting the font, it calls `font.clear()` then `font.set_size(size, self.dpi)`.\n\n5. **`start_filter`/`stop_filter` Pattern**: `start_filter()` saves the current `_renderer` to `_filter_renderers` list (a stack) and creates a new `_RendererAgg` instance. `stop_filter()` uses `cbook._get_nonzero_slices()` to crop the alpha channel to the smallest non-zero bounding rectangle, divides the cropped image by 255 before passing to `post_processing`, and if the returned image dtype is floating point, multiplies by 255 and casts to `np.uint8`.\n\n6. **Buffer Protocol**: The C++ `_RendererAgg` exposes its buffer via Python's buffer protocol with shape `(height, width, 4)` and strides `(width*4, 4, 1)`, directly accessing `renderer->pixBuffer`.\n\n7. **Serialization**: `__getstate__` only preserves `width`, `height`, and `dpi`; `__setstate__` calls `__init__` to reconstruct everything else.\n\n8. **Parent Class `RendererBase`**: Initializes `_texmanager = None`, `_text2path = text.TextToPath()`, `_raster_depth = 0`, and `_rasterizing = False`. It requires subclasses to implement `draw_path`, `draw_image`, and `draw_gouraud_triangles` at minimum.",
    "rubric": [
      "Explains that RendererAgg inherits from RendererBase and wraps a C++ _RendererAgg object from the pybind11 extension module",
      "Describes the _update_methods() delegation pattern where C-level methods (draw_gouraud_triangles, draw_image, draw_markers, draw_path_collection, draw_quad_mesh, copy_from_bbox) are assigned directly to instance attributes",
      "Notes that the C++ RendererAgg allocates a pixel buffer of width*height*4 bytes (RGBA) and initializes with agg::rgba(1,1,1,0) (white, fully transparent)",
      "Mentions the rasterizer cell block limit of 32768 and the maximum image dimension limit of 2^23 pixels",
      "Describes the start_filter/stop_filter pattern using a _filter_renderers stack, cropping via _get_nonzero_slices on the alpha channel, dividing by 255, and converting back to uint8",
      "Explains the buffer protocol exposing shape (height, width, 4) with strides (width*4, 4, 1) pointing to pixBuffer",
      "Mentions serialization via __getstate__/__setstate__ preserving only width, height, and dpi",
      "Notes RendererBase initializes _texmanager=None, _text2path=TextToPath(), _raster_depth=0, _rasterizing=False and requires draw_path, draw_image, draw_gouraud_triangles as minimum implementations"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_agg.py",
      "src/_backend_agg.cpp",
      "src/_backend_agg_wrapper.cpp",
      "src/_backend_agg.h",
      "lib/matplotlib/backend_bases.py",
      "lib/matplotlib/mathtext.py",
      "lib/matplotlib/_mathtext.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_agg.py] lib.matplotlib.backends.backend_agg.RendererAgg\nlib.matplotlib.backends.backend_agg.RendererAgg:\n  The renderer handles all the drawing primitives using a graphics\n    context instance that controls the colors/styles\n\nlib.matplotlib.backends.backend_agg.RendererAgg.draw_mathtext:\n  Draw mathtext using :mod:`matplotlib.mathtext`.\n\nlib.matplotlib.backends.backend_agg.RendererAgg._prepare_font:\n  Get the `.FT2Font` for *font_prop*, clear its buffer, and set its size.\n\nlib.matplotlib.backends.backend_agg.RendererAgg.restore_region:\n  Restore the saved region. If bbox (instance of BboxBase, or\n        its extents) is given, only the region specified by the bbox\n        will be restored. *xy* (a pair of floats) optionally\n        specifies the new position (the LLC of the original region,\n        not the LLC of the bbox) where the region will be restored.\n\n        >>> region = renderer.copy_from_bbox()\n        >>> x1, y1, x2, y2 = region.get_extents()\n        >>> renderer.restore_region(region, bbox=(x1+dx, y1, x2, y2),\n        ...                         xy=(x1-dx, y1))\n\nlib.matplotlib.backends.backend_agg.RendererAgg.start_filter:\n  Start filtering. It simply creates a new canvas (the old one is saved).\n\nlib.matplotlib.backends.backend_agg.RendererAgg.stop_filter:\n  Save the current canvas as an image and apply post processing.\n\n        The *post_processing* function::\n\n           def post_processing(image, dpi):\n             # ny, nx, depth = image.shape\n             # image (numpy array) has RGBA channels and has a depth of 4.\n             ...\n             # create a new_image (numpy array of 4 channels, size can be\n             # different). The resulting image may have offsets from\n             # lower-left corner of the original image\n             return new_image, offset_x, offset_y\n\n        The saved renderer is restored and the returned image from\n        post_processing is plotted (using draw_image) on it.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (C++ internals, buffer sizes, pixel limits, MathTextParser specifics, serialization behavior, etc.) that go far beyond what the documentation provides. These cannot be verified from the given documentation alone.",
      "The claim about `_prepare_font` calling `get_font(_fontManager._find_fonts_by_props(font_prop))` is not verifiable from the doc, which only says it gets the FT2Font for font_prop, clears its buffer, and sets its size. The general description is consistent but the specific implementation details are unverifiable.",
      "The description of `restore_region` in the answer is not explicitly stated but the documentation's description is consistent with what's implied.",
      "The `start_filter`/`stop_filter` description adds many details (stack via `_filter_renderers`, `cbook._get_nonzero_slices()`, division by 255, dtype checking) that are not in the documentation. The documentation only says start_filter creates a new canvas (saving the old one) and stop_filter saves as image and applies post_processing.",
      "The documentation says `stop_filter`'s post_processing returns `new_image, offset_x, offset_y`, which is consistent with the answer's description.",
      "The answer says `draw_mathtext` 'passes slant=0 and extend=1 for all glyphs' \u2014 this is not verifiable from the doc which only says it draws mathtext using matplotlib.mathtext."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for the full AGG name and caching key details not visible in the provided source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_24",
    "repo": "matplotlib",
    "question": "How does `from_any` convert size inputs, and how does the AxesDivider use it when appending new axes?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `from_any` function in `lib/mpl_toolkits/axes_grid1/axes_size.py` works as follows:\n\n1. **Type dispatch on the `size` parameter**: It uses `isinstance(size, Real)` (where `Real` is imported from Python's `numbers` module) to check if the input is numeric. If so, it returns `Fixed(size)`. If the input is a string ending with `%`, it parses the percentage by stripping the `%` character, converting to float, dividing by 100, and returning `Fraction(float(size[:-1]) / 100, fraction_ref)`. For any other input type or a string not ending in `%`, it raises `ValueError(\"Unknown format\")`.\n\n2. **Fixed class validation**: When `Fixed(size)` is created, it calls `_api.check_isinstance(Real, fixed_size=fixed_size)` which validates the input is a `numbers.Real` instance. The `Fixed.get_size()` method always returns `(0., self.fixed_size)` \u2014 relative size 0 and absolute size equal to the fixed_size value.\n\n3. **Fraction class behavior**: When `Fraction(fraction, ref_size)` is created, it validates `fraction` via `_api.check_isinstance(Real, fraction=fraction)`. Its `get_size()` method has special handling: if `_fraction_ref` is `None`, it returns `(self._fraction, 0.)` (treating the fraction as a relative size); otherwise it calls `self._fraction_ref.get_size(renderer)` and multiplies both the relative and absolute parts by the fraction.\n\n4. **Integration with AxesDivider**: In `axes_divider.py`, `from_any` is called within `new_horizontal` and `new_vertical` methods only after checking `if not isinstance(size, Size._Base)` \u2014 so if a size is already a `_Base` instance, `from_any` is skipped. The `fraction_ref` passed is `self._xref` (an `AxesX` instance) for horizontal and `self._yref` (an `AxesY` instance) for vertical operations.\n\n5. **Default pad computation**: When `pad=None` in `new_horizontal`, the default is computed as `mpl.rcParams[\"figure.subplot.wspace\"] * self._xref`, which invokes `_Base.__mul__`, creating `Fraction(wspace_value, self._xref)` \u2014 already a `_Base` instance, so `from_any` is not called for it.\n\n6. **Divider consumption of sizes**: The `Divider._calc_k` static method computes a scaling factor `k` such that `sum(rel_size * k + abs_size) == total` for all sizes. The `_calc_offsets` method then computes cumulative positions using `sizes @ [k, 1]` (a dot product of each (rel, abs) pair with [k, 1]).",
    "rubric": [
      "Explains that from_any checks if size is a Real (numbers.Real) and returns Fixed(size) if so",
      "Explains that from_any handles strings ending with '%' by stripping the %, converting to float, dividing by 100, and returning a Fraction with the fraction_ref",
      "Explains that from_any raises ValueError('Unknown format') for unrecognized inputs",
      "Explains that Fixed validates its input with check_isinstance(Real) and get_size returns (0., fixed_size)",
      "Explains that Fraction's get_size returns (fraction, 0.) if _fraction_ref is None, or multiplies both relative and absolute parts of ref's size by the fraction otherwise",
      "Explains that in new_horizontal/new_vertical, from_any is only called if the size/pad is not already a _Base instance",
      "Explains that _xref (AxesX) is used as fraction_ref for horizontal and _yref (AxesY) for vertical",
      "Explains the default pad computation uses rcParams wspace/hspace multiplied by _xref/_yref (which uses _Base.__mul__ to create a Fraction)",
      "Explains how Divider._calc_k computes a scaling factor k from (total - abs_sum) / rel_sum and _calc_offsets uses cumulative sums of sizes @ [k, 1]"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/axes_size.py",
      "lib/mpl_toolkits/axes_grid1/axes_divider.py",
      "lib/mpl_toolkits/axes_grid1/inset_locator.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/axes_size.py] lib.mpl_toolkits.axes_grid1.axes_size.from_any\nCreate a Fixed unit when the first argument is a float, or a\n    Fraction unit if that is a string that ends with %. The second\n    argument is only meaningful when Fraction unit is created.\n\n    >>> from mpl_toolkits.axes_grid1.axes_size import from_any\n    >>> a = from_any(1.2) # => Fixed(1.2)\n    >>> from_any(\"50%\", a) # => Fraction(0.5, a)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Only the line number reference was not verifiable from stripped code; all other claims are directly visible in the source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_21",
    "repo": "matplotlib",
    "question": "What are the key implementation techniques and design choices in the SVG backend's `_XMLWriter` class and its associated helper functions?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `_XMLWriter` class in `lib/matplotlib/backends/backend_svg.py` is the core output mechanism for the SVG backend.\n\nKey implementation details:\n\n1. **Indentation mechanism**: The class stores a pre-computed string of 64 spaces (`self.__indentation = \" \" * 64`) and slices into it using the current nesting depth (`self.__tags` list length) to produce indentation. This limits practical indentation to 64 levels deep.\n\n2. **Self-closing tag optimization**: The `end()` method checks if `self.__open` is still set (meaning no data or child elements were added since the last `start()` call). If so, it writes `\"/>\n\"` instead of a separate closing tag, producing self-closing XML elements.\n\n3. **Attribute filtering**: In the `start()` method, attributes with falsy values (empty string, None, 0, etc.) are silently skipped due to the `if v:` check. This means attributes like empty strings are intentionally omitted from output.\n\n4. **Flush delegation**: The constructor conditionally assigns `file.flush` to `self.flush` only if the file object has a `flush` attribute. Otherwise, the `flush()` method remains as a no-op (`pass`).\n\n5. **Attribute quoting strategy**: The `_quote_escape_attrib` function uses a preference hierarchy: it prefers double quotes if the value contains no double quotes, then tries single quotes, and falls back to double quotes with full escaping (including `&quot;` replacement) as a last resort.\n\n6. **Comment escaping**: The `_escape_comment` function uses a regex `r'-(?=-)'` to replace consecutive dashes by inserting a space between them (`'- '`).\n\n7. **Integration with RendererSVG**: The `RendererSVG` creates an `_XMLWriter` instance (`self.writer = _XMLWriter(svgwriter)`). The `start()` call for the root `<svg>` element returns an ID stored as `self._start_id`, which is later passed to `self.writer.close(self._start_id)` in `finalize()` to close all remaining open elements.\n\n8. **Deprecated public API**: The public `XMLWriter` class (deprecated since 3.11 via `@mpl._api.deprecated(\"3.11\")`) simply subclasses `_XMLWriter` with no additional functionality.\n\n9. **Usage of `comment()`**: The `_XMLWriter` class provides a `comment()` method that writes properly escaped XML comments with appropriate indentation.",
    "rubric": [
      "Explains the indentation mechanism: pre-computed 64-space string sliced by current tag nesting depth (length of __tags list), limiting practical indentation to 64 levels",
      "Describes self-closing tag optimization: end() checks if __open is still set and writes '/>' instead of a separate closing tag when no data/children were added",
      "Notes attribute filtering: start() silently skips attributes with falsy values due to the 'if v:' check",
      "Explains flush delegation: constructor conditionally assigns file.flush to self.flush only if the file object has a flush attribute, otherwise flush() remains a no-op (pass)",
      "Describes _quote_escape_attrib's quoting strategy: prefers double quotes, falls back to single quotes if value contains double quotes, and uses full escaping with double quotes as last resort",
      "Explains comment escaping: _escape_comment uses regex r'-(?=-)' to insert a space between consecutive dashes",
      "Describes integration with RendererSVG: creates _XMLWriter instance, start() returns an ID stored as _start_id, later passed to close() in finalize() to close all remaining open elements",
      "Mentions the deprecated public XMLWriter class (deprecated since 3.11) that subclasses _XMLWriter with no additional functionality"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_svg.py",
      "lib/matplotlib/backends/backend_mixed.py",
      "lib/matplotlib/backend_bases.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_svg.py] lib.matplotlib.backends.backend_svg._XMLWriter\nlib.matplotlib.backends.backend_svg._XMLWriter:\n  Parameters\n    ----------\n    file : writable text file-like object\n\nlib.matplotlib.backends.backend_svg._XMLWriter.start:\n  Open a new element.  Attributes can be given as keyword\n        arguments, or as a string/string dictionary. The method returns\n        an opaque identifier that can be passed to the :meth:`close`\n        method, to close all open elements up to and including this one.\n\n        Parameters\n        ----------\n        tag\n            Element tag.\n        attrib\n            Attribute dictionary.  Alternatively, attributes can be given as\n            keyword arguments.\n\n        Returns\n        -------\n        An element identifier.\n\nlib.matplotlib.backends.backend_svg._XMLWriter.comment:\n  Add a comment to the output stream.\n\n        Parameters\n        ----------\n        comment : str\n            Comment text.\n\nlib.matplotlib.backends.backend_svg._XMLWriter.data:\n  Add character data to the output stream.\n\n        Parameters\n        ----------\n        text : str\n            Character data.\n\nlib.matplotlib.backends.backend_svg._XMLWriter.end:\n  Close the current element (opened by the most recent call to\n        :meth:`start`).\n\n        Parameters\n        ----------\n        tag\n            Element tag.  If given, the tag must match the start tag.  If\n            omitted, the current element is closed.\n        indent : bool, default: True\n\nlib.matplotlib.backends.backend_svg._XMLWriter.close:\n  Close open elements, up to (and including) the element identified\n        by the given identifier.\n\n        Parameters\n        ----------\n        id\n            Element identifier, as returned by the :meth:`start` method.\n\nlib.matplotlib.backends.backend_svg._XMLWriter.element:\n  Add an entire element.  This is the same as calling :meth:`start`,\n        :meth:`data`, and :meth:`end` in sequence. The *text* argument can be\n        omitted.\n\nlib.matplotlib.backends.backend_svg._XMLWriter.flush:\n  Flush the output stream.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer (points 1-9) go far beyond what is documented in the provided documentation chunk. The documentation only describes the public API (parameters, return values, and brief descriptions of methods), not internal implementation details like indentation mechanisms, self-closing tag optimization, attribute filtering logic, flush delegation, attribute quoting strategies, comment escaping, integration with RendererSVG, deprecated public API, or specific usage patterns.",
      "While none of these claims directly contradict the documentation, they cannot be verified from the provided documentation and may constitute hallucinations or assumptions about implementation details.",
      "The basic structural claims about what methods exist and their general purpose are consistent with the documentation (start, comment, data, end, close, element, flush).",
      "The claim that start() returns an identifier that can be passed to close() is directly supported by the documentation.",
      "The claim about the file parameter being a 'writable text file-like object' is supported by the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "The gold answer mostly describes code-visible behavior, with some documentation-only claims about historical origin and design rationale.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_22",
    "repo": "matplotlib",
    "question": "Why does FigureCanvasWebAggCore use diff-based image transmission and various flags like _png_is_old and _force_full instead of just sending full frames?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `FigureCanvasWebAggCore` class employs several performance optimizations for sending rendered images over WebSocket connections:\n\n**1. Diff Image Mode (Primary Performance Optimization):**\nThe most significant performance optimization is the diff image mode in `get_diff_image()`. Instead of sending a full image every frame, the system computes pixel-level differences between the current and previous frame. In `get_diff_image()`, the renderer's RGBA buffer is viewed as `np.uint32` so that entire 4-byte pixels can be compared in a single numpy operation rather than comparing each RGBA channel separately. When in 'diff' mode, only changed pixels are sent (`output = np.where(diff, buff, 0)`), with unchanged pixels set to zero (fully transparent black).\n\nThe diff mode has a correctness constraint: if any pixel in the current frame has transparency (alpha != 255), the system automatically falls back to 'full' mode (`(pixels[:, :, 3] != 255).any()`). In full mode, the JavaScript client calls `clearRect()` before `drawImage()` to clear the canvas before drawing the new image.\n\n**2. ACK-Based Bidirectional Traffic Flow:**\nThe `handle_ack()` method does nothing on the server side (contains only `pass`). The JavaScript client sends an 'ack' message back to the server after each binary image frame is received.\n\n**3. Draw Throttling via `waiting` Flag:**\nOn the JavaScript side, `send_draw_message()` uses a `waiting` boolean flag. Once a draw message is sent, `waiting` is set to `true` and no further draw messages are sent until a binary image frame is received back from the server, which resets `waiting` to `false`. This throttles draw requests to avoid sending multiple redundant requests.\n\n**4. PNG Staleness Caching (`_png_is_old`):**\nThe `_png_is_old` flag avoids redundant diff computations. It is set to `True` when the canvas is drawn or resized, and set to `False` after `get_diff_image()` computes and returns a new PNG. If `get_diff_image()` is called when `_png_is_old` is False, it returns `None`, and `refresh_all()` skips sending binary data to sockets (`if diff is not None`).\n\n**5. `_force_full` State Transitions:**\nThe `_force_full` flag is set to `True` in four situations: initialization, `handle_refresh`, `handle_resize`, and device pixel ratio changes. This ensures that after structural changes, the first frame is always a complete image, then subsequent frames can use efficient diff encoding.\n\n**6. `set_image_mode` and `set_capture_scroll` Only-On-Change Guards:**\nBoth `set_image_mode` and `set_capture_scroll` check whether the new value differs from the current value before sending events to clients, avoiding unnecessary WebSocket messages.",
    "rubric": [
      "Explains the diff image mode: only changed pixels are sent (unchanged pixels set to zero/transparent) by comparing uint32-viewed RGBA buffers, reducing data transmitted over WebSocket",
      "Explains the transparency fallback: if any pixel has alpha != 255, diff mode cannot work (can't overlay transparent pixels) so it falls back to full mode with clearRect on the JS side",
      "Explains _png_is_old flag: avoids redundant diff computations by returning None from get_diff_image when the PNG hasn't changed, and refresh_all skips sending in that case",
      "Explains _force_full flag: set to True on init, refresh, resize, and device pixel ratio changes to ensure a complete frame is sent after structural changes before switching to diff encoding",
      "Explains the uint32 comparison trick: RGBA buffer is viewed as np.uint32 so entire pixels are compared in one numpy operation instead of comparing each channel separately",
      "Explains client-side draw throttling: the waiting flag in JavaScript prevents sending multiple draw requests before the server responds with an image frame",
      "Mentions the ACK-based bidirectional traffic flow: the JS client sends ack messages after receiving image frames to improve network latency, even though the server's handle_ack is just pass"
    ],
    "key_files": [
      "lib/matplotlib/backends/backend_webagg_core.py",
      "lib/matplotlib/backends/web_backend/js/mpl.js",
      "lib/matplotlib/backends/backend_agg.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/backend_webagg_core.py] lib.matplotlib.backends.backend_webagg_core.FigureCanvasWebAggCore\nlib.matplotlib.backends.backend_webagg_core.FigureCanvasWebAggCore.set_image_mode:\n  Set the image mode for any subsequent images which will be sent\n        to the clients. The modes may currently be either 'full' or 'diff'.\n\n        Note: diff images may not contain transparency, therefore upon\n        draw this mode may be changed if the resulting image has any\n        transparent component.\n\nlib.matplotlib.backends.backend_webagg_core.FigureCanvasWebAggCore.set_capture_scroll:\n  Set whether the scroll events on the canvas will scroll the page.\n\n        Parameters\n        ----------\n        capture : bool\n\nlib.matplotlib.backends.backend_webagg_core.FigureCanvasWebAggCore.get_capture_scroll:\n  Get whether scroll events are currently captured by the canvas.\n\n        Returns\n        -------\n        bool",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides docstrings for `set_image_mode`, `set_capture_scroll`, and `get_capture_scroll`. The vast majority of the answer's claims (diff image mode details, ACK-based traffic flow, draw throttling, PNG staleness caching, _force_full state transitions, specific line numbers, JavaScript client behavior) go far beyond what is in the provided documentation and cannot be verified from it.",
      "The claim about `set_image_mode` modes being 'full' or 'diff' is supported by the documentation.",
      "The claim that diff images may not contain transparency and mode may be changed upon draw is supported by the documentation.",
      "The claim about `set_capture_scroll` taking a `capture: bool` parameter is supported by the documentation.",
      "The claim about `set_image_mode` checking whether the new value differs before sending events (point 6) is not mentioned in the documentation \u2014 it's an extrapolation from code not provided.",
      "Most specific line numbers and implementation details (np.uint32 comparison, _png_is_old flag, handle_ack, JavaScript mpl.js details) are plausible code-level details but entirely unverifiable from the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rationale explanations about PNG compression, TCP bidirectionality, and ghosting prevention come from comments/docs.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_23",
    "repo": "matplotlib",
    "question": "Where is `backend_registry` defined, and where is it consumed across the Matplotlib codebase?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `BackendRegistry` singleton (`backend_registry`) is defined at module level in `lib/matplotlib/backends/registry.py` and exported from `lib/matplotlib/backends/__init__.py`. It is consumed by several key parts of Matplotlib:\n\n**Where it is used (callers):**\n1. `lib/matplotlib/rcsetup.py`: `validate_backend()` calls `backend_registry.is_valid_backend(s)` to validate the `rcParams['backend']` value.\n2. `lib/matplotlib/pyplot.py`: `switch_backend()` calls `backend_registry.load_backend_module(newbackend)` to load the backend module.\n3. `lib/matplotlib/pyplot.py`: The auto-backend sentinel logic in `switch_backend()` calls `backend_registry.backend_for_gui_framework()` to find the backend matching the currently running framework.\n4. `lib/matplotlib/pyplot.py`: `install_repl_displayhook()` calls `backend_registry.resolve_backend(get_backend())` for IPython >= 8.24 integration.\n\n**Key implementation details:**\n- The `_name_to_module` dict is initialized with only one entry: `{\"notebook\": \"nbagg\"}`, mapping the 'notebook' alias to the nbagg module.\n- `_backend_module_name()` handles a double-indirection: it first looks up the backend in `_name_to_module`, and if the result starts with `\"module://\"`, it strips that prefix; otherwise it formats as `f\"matplotlib.backends.backend_{backend}\"`.\n- `is_valid_backend()` contains backward-compatibility mappings that convert `\"module://ipympl.backend_nbagg\"` to `\"widget\"` and `\"module://matplotlib_inline.backend_inline\"` to `\"inline\"`.\n- Entry points are read from the `\"matplotlib.backend\"` group in `_read_entry_points`.\n- For backward compatibility, if `matplotlib_inline` version < `(0, 1, 7)`, an \"inline\" entry point is synthesized targeting `\"matplotlib_inline.backend_inline\"`. If `ipympl` version < `(0, 9, 4)`, both \"ipympl\" and \"widget\" entry points are created targeting `\"ipympl.backend_nbagg\"`.\n- `_validate_and_store_entry_points` uses `set(entries)` to deduplicate identical (name, module) pairs before iterating, which allows multiple identical entry points.\n- `resolve_backend()` handles non-string backends (e.g., `_auto_backend_sentinel` or `None`) by calling `matplotlib.get_backend()` to get the current backend.\n- `_get_gui_framework_by_loading()` loads a backend module and reads `module.FigureCanvas.required_interactive_framework`, returning `\"headless\"` if it's None/falsy.\n- The `_GUI_FRAMEWORK_TO_BACKEND` mapping includes `\"qt6\": \"qtagg\"`, making \"qt6\" a valid framework that maps to the same backend as \"qt\".\n- The `_clear()` method simply calls `self.__init__()` to reset all state.\n- In `pyplot.switch_backend()`, the auto-backend candidate list is: `[\"macosx\", \"qtagg\", \"gtk4agg\", \"gtk3agg\", \"tkagg\", \"wxagg\"]`, with a final fallback to \"agg\".",
    "rubric": [
      "States that `backend_registry` is a singleton instance of `BackendRegistry` created at module level in the backends registry module and exported from the backends __init__",
      "Mentions `validate_backend()` in rcsetup calls `backend_registry.is_valid_backend()` to validate backend names",
      "Mentions `switch_backend()` in pyplot calls `backend_registry.load_backend_module()` to load the backend module",
      "Mentions `switch_backend()` in pyplot calls `backend_registry.backend_for_gui_framework()` for auto-backend selection when current framework is detected",
      "Mentions `install_repl_displayhook()` in pyplot calls `backend_registry.resolve_backend()` for IPython >= 8.24 integration",
      "Describes that `_name_to_module` is initialized with only one hardcoded entry mapping 'notebook' to 'nbagg'",
      "Explains the double-indirection in `_backend_module_name`: looks up in `_name_to_module`, then checks for 'module://' prefix or formats as 'matplotlib.backends.backend_{name}'",
      "Mentions backward-compatibility mappings in `is_valid_backend()` that convert 'module://ipympl.backend_nbagg' to 'widget' and 'module://matplotlib_inline.backend_inline' to 'inline'",
      "Describes entry point loading from the 'matplotlib.backend' group, including backward-compat synthesized entry points for old matplotlib_inline (< 0.1.7) and ipympl (< 0.9.4)",
      "Notes the auto-backend candidate list in switch_backend: macosx, qtagg, gtk4agg, gtk3agg, tkagg, wxagg, with final fallback to agg"
    ],
    "key_files": [
      "lib/matplotlib/backends/registry.py",
      "lib/matplotlib/pyplot.py",
      "lib/matplotlib/rcsetup.py",
      "lib/matplotlib/backends/__init__.py",
      "lib/matplotlib/backend_bases.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/backends/registry.py] lib.matplotlib.backends.registry.BackendRegistry\nlib.matplotlib.backends.registry.BackendRegistry:\n  Registry of backends available within Matplotlib.\n\n    This is the single source of truth for available backends.\n\n    All use of ``BackendRegistry`` should be via the singleton instance\n    ``backend_registry`` which can be imported from ``matplotlib.backends``.\n\n    Each backend has a name, a module name containing the backend code, and an\n    optional GUI framework that must be running if the backend is interactive.\n    There are three sources of backends: built-in (source code is within the\n    Matplotlib repository), explicit ``module://some.backend`` syntax (backend is\n    obtained by loading the module), or via an entry point (self-registering\n    backend in an external package).\n\n    .. versionadded:: 3.9\n\nlib.matplotlib.backends.registry.BackendRegistry.backend_for_gui_framework:\n  Return the name of the backend corresponding to the specified GUI framework.\n\n        Parameters\n        ----------\n        framework : str\n            GUI framework such as \"qt\".\n\n        Returns\n        -------\n        str or None\n            Backend name or None if GUI framework not recognised.\n\nlib.matplotlib.backends.registry.BackendRegistry.is_valid_backend:\n  Return True if the backend name is valid, False otherwise.\n\n        A backend name is valid if it is one of the built-in backends or has been\n        dynamically added via an entry point. Those beginning with ``module://`` are\n        always considered valid and are added to the current list of all backends\n        within this function.\n\n        Even if a name is valid, it may not be importable or usable. This can only be\n        determined by loading and using the backend module.\n\n        Parameters\n        ----------\n        backend : str\n            Name of backend.\n\n        Returns\n        -------\n        bool\n            True if backend is valid, False otherwise.\n\nlib.matplotlib.backends.registry.BackendRegistry.list_all:\n  Return list of all known backends.\n\n        These include built-in backends and those obtained at runtime either from entry\n        points or explicit ``module://some.backend`` syntax.\n\n        Entry points will be loaded if they haven't been already.\n\n        Returns\n        -------\n        list of str\n            Backend names.\n\nlib.matplotlib.backends.registry.BackendRegistry.list_builtin:\n  Return list of backends that are built into Matplotlib.\n\n        Parameters\n        ----------\n        filter_ : `~.BackendFilter`, optional\n            Filter to apply to returned backends. For example, to return only\n            non-interactive backends use `.BackendFilter.NON_INTERACTIVE`.\n\n        Returns\n        -------\n        list of str\n            Backend names.\n\nlib.matplotlib.backends.registry.BackendRegistry.list_gui_frameworks:\n  Return list of GUI frameworks used by Matplotlib backends.\n\n        Returns\n        -------\n        list of str\n            GUI framework names.\n\nlib.matplotlib.backends.registry.BackendRegistry.load_backend_module:\n  Load and return the module containing the specified backend.\n\n        Parameters\n        ----------\n        backend : str\n            Name of backend to load.\n\n        Returns\n        -------\n        Module\n            Module containing backend.\n\nlib.matplotlib.backends.registry.BackendRegistry.resolve_backend:\n  Return the backend and GUI framework for the specified backend name.\n\n        If the GUI framework is not yet known then it will be determined by loading the\n        backend module and checking the ``FigureCanvas.required_interactive_framework``\n        attribute.\n\n        This function only loads entry points if they have not already been loaded and\n        the backend is not built-in and not of ``module://some.backend`` format.\n\n        Parameters\n        ----------\n        backend : str or None\n            Name of backend, or None to use the default backend.\n\n        Returns\n        -------\n        backend : str\n            The backend name.\n        framework : str or None\n            The GUI framework, which will be None for a backend that is non-interactive.\n\nlib.matplotlib.backends.registry.BackendRegistry.resolve_gui_or_backend:\n  Return the backend and GUI framework for the specified string that may be\n        either a GUI framework or a backend name, tested in that order.\n\n        This is for use with the IPython %matplotlib magic command which may be a GUI\n        framework such as ``%matplotlib qt`` or a backend name such as\n        ``%matplotlib qtagg``.\n\n        This function only loads entry points if they have not already been loaded and\n        the backend is not built-in and not of ``module://some.backend`` format.\n\n        Parameters\n        ----------\n        gui_or_backend : str or None\n            Name of GUI framework or backend, or None to use the default backend.\n\n        Returns\n        -------\n        backend : str\n            The backend name.\n        framework : str or None\n            The GUI framework, which will be None for a backend that is non-interactive.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed unverifiable claims about truncated code sections and stripped line number references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_25",
    "repo": "matplotlib",
    "question": "What classes, inheritance chains, and key design decisions are involved in creating an inset axes via `inset_axes` in the axes_grid1 toolkit?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `inset_axes` function in `lib/mpl_toolkits/axes_grid1/inset_locator.py` creates an inset axes by combining several architectural components:\n\n**Class Hierarchy:**\n- `AnchoredSizeLocator` \u2192 `AnchoredLocatorBase` \u2192 `AnchoredOffsetbox` \u2192 `OffsetBox` \u2192 `Artist`\n- The default `axes_class` is `HostAxes`, which is dynamically generated via `cbook._make_class_factory(HostAxesBase, \"{}HostAxes\", \"_base_axes_class\")` applied to `mpl_axes.Axes` (which itself inherits from `matplotlib.axes.Axes`).\n\n**Key Implementation Details:**\n1. `_add_inset_axes` helper explicitly sets `\"navigate\": False` in the axes kwargs dict, meaning inset axes are non-navigable by default. However, user-provided `axes_kwargs` can override this since the dict unpacking is `{\"navigate\": False, **axes_kwargs, \"axes_locator\": axes_locator}` \u2014 wait, actually `axes_kwargs` comes after so it *can* override `navigate`, but `axes_locator` always wins.\n\n2. In `AnchoredLocatorBase.__init__`, `pad=0.` is explicitly passed to `AnchoredOffsetbox.__init__` while `borderpad` is the one that controls spacing. This means `AnchoredSizeLocator.get_bbox` applies `self.pad * fontsize` = 0 padding to the computed bbox (since `pad=0.`), while the `borderpad` parameter controls spacing via `get_offset` in the parent class.\n\n3. `Size.from_any()` is called without a `fraction_ref` parameter (line 47-48). When the size is a percentage string (e.g., '40%'), it creates `Fraction(0.4, None)`. The `Fraction.get_size()` method returns `(self._fraction, 0.)` when `_fraction_ref is None`, making it a purely relative size.\n\n4. In `AnchoredSizeLocator.get_bbox`, the width calculation is `bbox.width * r + a * dpi` where `dpi = renderer.points_to_pixels(72.)`. For percentage sizes, `r` is the fraction and `a=0`. For inch sizes (Fixed), `r=0` and `a=fixed_size`.\n\n5. The `AnchoredLocatorBase.__call__` method returns a `TransformedBbox(bbox_canvas, fig.transSubfigure.inverted())` for positioning.\n\n6. The `_get_anchored_bbox` function (in offsetbox.py) maps location codes to anchor points: `[None, \"NE\", \"NW\", \"SW\", \"SE\", \"E\", \"W\", \"E\", \"S\", \"N\", \"C\"]`. Note location code 5 (\"right\") and 7 (\"center right\") both map to \"E\".\n\n7. The `AnchoredOffsetbox.get_offset` method (decorated with `_compat_get_offset`) handles tuple borderpad: it tries `borderpad_x, borderpad_y = self.borderpad` and falls back to using the same value for both on TypeError.\n\n8. `HostAxes` (the default axes_class) has a `draw` method in `HostAxesBase` that calls `locator(self, renderer)` to get position, then calls `self.set_position(pos, which=\"active\")` followed by `self.apply_aspect(pos)` \u2014 this is how the inset axes gets positioned at render time.",
    "rubric": [
      "Identifies the class hierarchy: AnchoredSizeLocator \u2192 AnchoredLocatorBase \u2192 AnchoredOffsetbox (\u2192 OffsetBox \u2192 Artist)",
      "Identifies that the default axes_class is HostAxes, generated via cbook._make_class_factory(HostAxesBase, ...) applied to mpl_axes.Axes",
      "Explains that _add_inset_axes sets 'navigate': False by default but axes_kwargs can override it due to dict unpacking order, while axes_locator always wins",
      "Explains that AnchoredLocatorBase.__init__ passes pad=0. to AnchoredOffsetbox, so the pad-based padding in get_bbox is always zero, while borderpad controls actual spacing",
      "Explains that Size.from_any() is called without fraction_ref, so percentage strings create Fraction(value, None), making get_size return (fraction, 0.) \u2014 a purely relative size",
      "Describes the width/height calculation in AnchoredSizeLocator.get_bbox: bbox.width * r + a * dpi, where r and a come from get_size()",
      "Explains that AnchoredLocatorBase.__call__ returns a TransformedBbox(bbox_canvas, fig.transSubfigure.inverted())",
      "Mentions _get_anchored_bbox maps location codes to anchor strings like 'NE', 'NW', 'SW', 'SE', etc., noting that codes 5 and 7 both map to 'E'",
      "Explains that AnchoredOffsetbox.get_offset handles tuple borderpad by trying to unpack into borderpad_x, borderpad_y with a TypeError fallback",
      "Describes that HostAxesBase.draw calls locator(self, renderer) to get position, then set_position and apply_aspect to position the inset at render time"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/inset_locator.py",
      "lib/matplotlib/offsetbox.py",
      "lib/mpl_toolkits/axes_grid1/axes_size.py",
      "lib/mpl_toolkits/axes_grid1/parasite_axes.py",
      "lib/mpl_toolkits/axes_grid1/mpl_axes.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/inset_locator.py] lib.mpl_toolkits.axes_grid1.inset_locator.inset_axes\nCreate an inset axes with a given width and height.\n\n    Both sizes used can be specified either in inches or percentage.\n    For example,::\n\n        inset_axes(parent_axes, width='40%%', height='30%%', loc='lower left')\n\n    creates in inset axes in the lower left corner of *parent_axes* which spans\n    over 30%% in height and 40%% in width of the *parent_axes*. Since the usage\n    of `.inset_axes` may become slightly tricky when exceeding such standard\n    cases, it is recommended to read :doc:`the examples\n    </gallery/axes_grid1/inset_locator_demo>`.\n\n    Notes\n    -----\n    The meaning of *bbox_to_anchor* and *bbox_to_transform* is interpreted\n    differently from that of legend. The value of bbox_to_anchor\n    (or the return value of its get_points method; the default is\n    *parent_axes.bbox*) is transformed by the bbox_transform (the default\n    is Identity transform) and then interpreted as points in the pixel\n    coordinate (which is dpi dependent).\n\n    Thus, following three calls are identical and creates an inset axes\n    with respect to the *parent_axes*::\n\n       axins = inset_axes(parent_axes, \"30%%\", \"40%%\")\n       axins = inset_axes(parent_axes, \"30%%\", \"40%%\",\n                          bbox_to_anchor=parent_axes.bbox)\n       axins = inset_axes(parent_axes, \"30%%\", \"40%%\",\n                          bbox_to_anchor=(0, 0, 1, 1),\n                          bbox_transform=parent_axes.transAxes)\n\n    Parameters\n    ----------\n    parent_axes : `matplotlib.axes.Axes`\n        Axes to place the inset axes.\n\n    width, height : float or str\n        Size of the inset axes to create. If a float is provided, it is\n        the size in inches, e.g. *width=1.3*. If a string is provided, it is\n        the size in relative units, e.g. *width='40%%'*. By default, i.e. if\n        neither *bbox_to_anchor* nor *bbox_transform* are specified, those\n        are relative to the parent_axes. Otherwise, they are to be understood\n        relative to the bounding box provided via *bbox_to_anchor*.\n\n    loc : str, default: 'upper right'\n        Location to place the inset axes.  Valid locations are\n        'upper left', 'upper center', 'upper right',\n        'center left', 'center', 'center right',\n        'lower left', 'lower center', 'lower right'.\n        For backward compatibility, numeric values are accepted as well.\n        See the parameter *loc* of `.Legend` for details.\n\n    bbox_to_anchor : tuple or `~matplotlib.transforms.BboxBase`, optional\n        Bbox that the inset axes will be anchored to. If None,\n        a tuple of (0, 0, 1, 1) is used if *bbox_transform* is set\n        to *parent_axes.transAxes* or *parent_axes.figure.transFigure*.\n        Otherwise, *parent_axes.bbox* is used. If a tuple, can be either\n        [left, bottom, width, height], or [left, bottom].\n        If the kwargs *width* and/or *height* are specified in relative units,\n        the 2-tuple [left, bottom] cannot be used. Note that,\n        unless *bbox_transform* is set, the units of the bounding box\n        are interpreted in the pixel coordinate. When using *bbox_to_anchor*\n        with tuple, it almost always makes sense to also specify\n        a *bbox_transform*. This might often be the axes transform\n        *parent_axes.transAxes*.\n\n    bbox_transform : `~matplotlib.transforms.Transform`, optional\n        Transformation for the bbox that contains the inset axes.\n        If None, a `.transforms.IdentityTransform` is used. The value\n        of *bbox_to_anchor* (or the return value of its get_points method)\n        is transformed by the *bbox_transform* and then interpreted\n        as points in the pixel coordinate (which is dpi dependent).\n        You may provide *bbox_to_anchor* in some normalized coordinate,\n        and give an appropriate transform (e.g., *parent_axes.transAxes*).\n\n    axes_class : `~matplotlib.axes.Axes` type, default: `.HostAxes`\n        The type of the newly created inset axes.\n\n    axes_kwargs : dict, optional\n        Keyword arguments to pass to the constructor of the inset axes.\n        Valid arguments include:\n\n        %(Axes:kwdoc)s\n\n    borderpad : float or (float, float), default: 0.5\n        Padding between inset axes and the bbox_to_anchor.\n        If a float, the same padding is used for both x and y.\n        If a tuple of two floats, it specifies the (x, y) padding.\n        The units are axes font size, i.e. for a default font size of 10 points\n        *borderpad = 0.5* is equivalent to a padding of 5 points.\n\n        .. versionadded:: 3.11\n           The *borderpad* parameter now accepts a tuple of (x, y) paddings.\n\n    Returns\n    -------\n    inset_axes : *axes_class*\n        Inset axes object created.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims are about implementation details not covered in the documentation, making them impossible to verify from the doc alone, but they don't contradict what the documentation states.",
      "The claim about default axes_class being HostAxes is consistent with the documentation which states 'axes_class : ~matplotlib.axes.Axes type, default: .HostAxes'.",
      "The claim about borderpad accepting tuple since 3.11 is consistent with the documentation's '.. versionadded:: 3.11' note.",
      "The claim about default loc is not explicitly stated in the answer but the documentation says 'loc : str, default: upper right'.",
      "The claim about borderpad default 0.5 is consistent with documentation stating 'borderpad : float or (float, float), default: 0.5'."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for interpretation language and version references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_27",
    "repo": "matplotlib",
    "question": "Where does the data flow when CenteredNorm is called to normalize values, and how does it maintain symmetry around vcenter?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "When `CenteredNorm.__call__` is invoked, the data flows through the following path:\n\n1. **`CenteredNorm` does NOT override `__call__`** \u2014 it inherits the `Normalize.__call__` method (line 2526 of `lib/matplotlib/colors.py`). This is a key architectural difference from `TwoSlopeNorm`, which DOES override `__call__` with its own `np.interp`-based piecewise linear mapping.\n\n2. **`Normalize.__call__` flow**: The inherited `__call__` first calls `self.process_value(value)` to convert input to a masked array. If `vmin` or `vmax` is `None`, it calls `self.autoscale_None(result)` \u2014 which IS overridden by `CenteredNorm`.\n\n3. **`CenteredNorm.autoscale_None`** (line 2773): Checks if `self.halfrange is None` AND `A.size > 0`, then delegates to `self.autoscale(A)`.\n\n4. **`CenteredNorm.autoscale`** (line 2765): Computes `halfrange = max(vcenter - A.min(), A.max() - vcenter)` and sets it via the `halfrange` setter.\n\n5. **`halfrange` setter** (line 2821): When a non-None value is set, it calls `self.vmin = self.vcenter - abs(halfrange)` and `self.vmax = self.vcenter + abs(halfrange)`, using `abs()` to ensure non-negative halfrange.\n\n6. **`CenteredNorm.vmin` setter** (line 2783): Overrides `Normalize.vmin` setter to enforce symmetry \u2014 when `vmin` is set, it also sets `self._vmax = 2*self.vcenter - value`, maintaining the center. Similarly, the `vmax` setter sets `self._vmin = 2*self.vcenter - value`.\n\n7. **Callback propagation**: Both the `vmin` and `vmax` setters call `self._changed()` (inherited from `Norm` ABC at line 2376), which calls `self.callbacks.process('changed')`. The `Colorizer` class (in `lib/matplotlib/colorizer.py`, line 103) connects to this signal via `self.norm.callbacks.connect('changed', self.changed)`, so any change to `CenteredNorm`'s limits triggers the colorizer to update.\n\n8. **`vcenter` setter** (line 2807): When vcenter is changed, it re-triggers the halfrange setter with the current halfrange value (`self.halfrange = self.halfrange`), which recalculates vmin/vmax around the new center, then calls `self._changed()`.\n\n9. **The normalization formula**: Since `CenteredNorm` inherits `Normalize.__call__`, the actual normalization is `(value - vmin) / (vmax - vmin)` (lines 2548-2550), operating in-place on the masked array's data. Because `vmin = vcenter - halfrange` and `vmax = vcenter + halfrange`, this simplifies to `(value - vcenter + halfrange) / (2 * halfrange)`, producing 0.5 at vcenter.\n\n10. **Class hierarchy**: `CenteredNorm` \u2192 `Normalize` \u2192 `Norm(ABC)`. The `Norm` ABC (line 2304) uses `CallbackRegistry` with signal name `'changed'` and declares abstract properties `vmin`, `vmax`, `clip`, `n_components` and abstract methods `__call__`, `autoscale`, `autoscale_None`, `scaled`. `Normalize` sets `n_components` to 1 (line 2614).\n\n11. **`_sanitize_extrema`** (line 211): Called in every `vmin`/`vmax` setter. It converts numpy scalar types to Python native types via `.item()`, or falls back to `float(ex)` for non-numpy types. Returns `None` unchanged.\n\n12. **`Normalize.__init__`** is called by `CenteredNorm.__init__` with `vmin=None, vmax=None` always (line 2760), ignoring any vcenter/halfrange initially. The `Normalize.__init__` sets `self._scale = None`.",
    "rubric": [
      "CenteredNorm does not override __call__ \u2014 it inherits Normalize.__call__",
      "Normalize.__call__ calls self.autoscale_None which IS overridden by CenteredNorm",
      "CenteredNorm.autoscale_None delegates to self.autoscale which computes halfrange as max(vcenter - min, max - vcenter)",
      "The halfrange setter sets vmin = vcenter - abs(halfrange) and vmax = vcenter + abs(halfrange)",
      "CenteredNorm's vmin setter enforces symmetry by also setting vmax = 2*vcenter - vmin (and vice versa for vmax setter)",
      "The actual normalization formula is (value - vmin) / (vmax - vmin), which simplifies to (value - vcenter + halfrange) / (2*halfrange), producing 0.5 at vcenter",
      "The vcenter setter recalculates vmin/vmax by re-triggering the halfrange setter with the current halfrange",
      "Changes to vmin/vmax/vcenter trigger self._changed() which fires callbacks via CallbackRegistry with signal 'changed'"
    ],
    "key_files": [
      "lib/matplotlib/colors.py",
      "lib/matplotlib/colorizer.py",
      "lib/matplotlib/tests/test_colors.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/colors.py] lib.matplotlib.colors.CenteredNorm\nlib.matplotlib.colors.CenteredNorm.__init__:\n  Normalize symmetrical data around a center (0 by default).\n\n        Unlike `TwoSlopeNorm`, `CenteredNorm` applies an equal rate of change\n        around the center.\n\n        Useful when mapping symmetrical data around a conceptual center\n        e.g., data that range from -2 to 4, with 0 as the midpoint, and\n        with equal rates of change around that midpoint.\n\n        Parameters\n        ----------\n        vcenter : float, default: 0\n            The data value that defines ``0.5`` in the normalization.\n        halfrange : float, optional\n            The range of data values that defines a range of ``0.5`` in the\n            normalization, so that *vcenter* - *halfrange* is ``0.0`` and\n            *vcenter* + *halfrange* is ``1.0`` in the normalization.\n            Defaults to the largest absolute difference to *vcenter* for\n            the values in the dataset.\n        clip : bool, default: False\n            Determines the behavior for mapping values outside the range\n            ``[vmin, vmax]``.\n\n            If clipping is off, values outside the range ``[vmin, vmax]`` are\n            also transformed, resulting in values outside ``[0, 1]``.  This\n            behavior is usually desirable, as colormaps can mark these *under*\n            and *over* values with specific colors.\n\n            If clipping is on, values below *vmin* are mapped to 0 and values\n            above *vmax* are mapped to 1. Such values become indistinguishable\n            from regular boundary values, which may cause misinterpretation of\n            the data.\n\n        Examples\n        --------\n        This maps data values -2 to 0.25, 0 to 0.5, and 4 to 1.0\n        (assuming equal rates of change above and below 0.0):\n\n            >>> import matplotlib.colors as mcolors\n            >>> norm = mcolors.CenteredNorm(halfrange=4.0)\n            >>> data = [-2., 0., 4.]\n            >>> norm(data)\n            array([0.25, 0.5 , 1.  ])\n\nlib.matplotlib.colors.CenteredNorm.autoscale:\n  Set *halfrange* to ``max(abs(A-vcenter))``, then set *vmin* and *vmax*.\n\nlib.matplotlib.colors.CenteredNorm.autoscale_None:\n  Set *vmin* and *vmax*.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims involve internal implementation details (line numbers, specific setter logic, callback mechanisms, class hierarchy details) that go far beyond the provided documentation and cannot be verified from it alone.",
      "The core behavior described (CenteredNorm normalizes symmetrically around vcenter, halfrange defaults to max absolute difference from vcenter, autoscale sets halfrange to max(abs(A-vcenter))) is consistent with the documentation.",
      "The normalization formula derivation (producing 0.5 at vcenter) is consistent with the documented example where norm(0) = 0.5 with halfrange=4.0.",
      "The example in the documentation confirms: with halfrange=4.0 and vcenter=0, data [-2, 0, 4] maps to [0.25, 0.5, 1.0], which matches the formula (value - vcenter + halfrange) / (2 * halfrange).",
      "The claim about autoscale setting halfrange to max(vcenter - A.min(), A.max() - vcenter) is consistent with the doc statement 'Set halfrange to max(abs(A-vcenter))' \u2014 these are equivalent.",
      "The description of clip behavior (clipping off allows values outside [0,1], clipping on maps to 0/1) is consistent with documentation."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code \u2014 no documentation-only claims detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_26",
    "repo": "matplotlib",
    "question": "Why does TwoSlopeNorm ignore the clip parameter, omit it from its constructor, and duplicate monotonicity validation in both __init__ and __call__?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `TwoSlopeNorm` class makes several deliberate design decisions that reveal its rationale:\n\n**1. Why `clip` is ignored:** `TwoSlopeNorm.__call__` accepts a `clip` parameter but never uses it. Instead, it uses `left=-np.inf, right=np.inf` in `np.interp()`. This means values below `vmin` map to `-inf` and values above `vmax` map to `+inf`, rather than being clipped to [0, 1]. The parent `Normalize.__call__` properly handles clip by applying `np.clip` when clip is True, but `TwoSlopeNorm` completely overrides this behavior.\n\n**2. Why `TwoSlopeNorm.__init__` doesn't expose `clip`:** The constructor signature is `__init__(self, vcenter, vmin=None, vmax=None)` \u2014 notably missing `clip`. When it calls `super().__init__(vmin=vmin, vmax=vmax)`, the parent `Normalize.__init__` defaults `clip=False`. This means clip is always False on the instance, but it doesn't matter because `__call__` ignores clip anyway.\n\n**3. Why `autoscale_None` mirrors vcenter into the range:** The `autoscale_None` method uses `>=` and `<=` comparisons (not just `>` and `<`) to defensively expand the range. If `vmin >= vcenter`, it mirrors `vmax` around `vcenter` to set a new `vmin`. If `vmax <= vcenter`, it mirrors `vmin` around `vcenter` to set a new `vmax`. This ensures that `vcenter` always strictly lies within `[vmin, vmax]`, preventing a degenerate piecewise-linear interpolation. It first calls `super().autoscale_None(A)` which sets vmin/vmax from the data's min/max if they're None.\n\n**4. Why validation is duplicated:** The monotonicity check `vmin < vcenter < vmax` is enforced in two separate places: `__init__` checks using `>=`/`<=` with a message about 'ascending order', while `__call__` checks with `not self.vmin <= self.vcenter <= self.vmax` with a message about 'monotonically'. The `__init__` check validates construction parameters, while the `__call__` check validates at call time, which matters because vmin/vmax can be changed after construction via property setters.\n\n**5. Why `inverse` also uses `-np.inf`/`np.inf` extrapolation:** The `inverse` method mirrors the same extrapolation pattern as `__call__`, using `np.interp(value, [0, 0.5, 1], [vmin, vcenter, vmax], left=-np.inf, right=np.inf)`. This ensures the inverse is consistent \u2014 values outside [0, 1] that resulted from the forward extrapolation can be properly handled.\n\n**6. The class hierarchy:** `TwoSlopeNorm` inherits from `Normalize`, which inherits from `Norm(ABC)`. The `Norm` ABC defines abstract properties for `vmin`, `vmax`, `clip`, abstract methods `__call__`, `autoscale`, `autoscale_None`, `scaled`, and provides a `_changed()` method that processes a 'changed' signal via a `CallbackRegistry`. The `vcenter` property setter also calls `self._changed()` when the value changes, ensuring observers are notified \u2014 this is the same pattern used by the `vmin` and `vmax` setters in `Normalize`.\n\n**7. Why `process_value` is used:** Both `__call__` and `inverse` use `self.process_value()` (a static method inherited from Normalize) to convert input to masked arrays with appropriate float dtypes. The method preserves float32 when possible and converts small integer types to float32 (larger integers to float64).",
    "rubric": [
      "Explains that __call__ accepts clip but never uses it, instead using left=-np.inf, right=np.inf in np.interp to extrapolate rather than clip values to [0,1]",
      "Notes that the constructor omits clip and the parent Normalize.__init__ defaults clip to False, so clip is always False on the instance",
      "Explains that __init__ validates monotonicity at construction time using >= and <= checks, while __call__ validates again because vmin/vmax can be changed after construction via property setters",
      "Describes autoscale_None's defensive mirroring logic: if vmin >= vcenter, it mirrors vmax around vcenter (and vice versa) to ensure vcenter lies strictly within [vmin, vmax]",
      "Mentions the inverse method also uses -np.inf/np.inf extrapolation to stay consistent with __call__",
      "Notes the class hierarchy: TwoSlopeNorm inherits from Normalize, which inherits from Norm(ABC), and that vcenter's property setter calls _changed() to notify observers like vmin/vmax setters do",
      "Mentions process_value is used to convert input to masked arrays with appropriate float dtypes (preserving float32 for small types)"
    ],
    "key_files": [
      "lib/matplotlib/colors.py",
      "lib/matplotlib/cbook.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/colors.py] lib.matplotlib.colors.TwoSlopeNorm\nlib.matplotlib.colors.TwoSlopeNorm.__init__:\n  Normalize data with a set center.\n\n        Useful when mapping data with an unequal rates of change around a\n        conceptual center, e.g., data that range from -2 to 4, with 0 as\n        the midpoint.\n\n        Parameters\n        ----------\n        vcenter : float\n            The data value that defines ``0.5`` in the normalization.\n        vmin : float, optional\n            The data value that defines ``0.0`` in the normalization.\n            Defaults to the min value of the dataset.\n        vmax : float, optional\n            The data value that defines ``1.0`` in the normalization.\n            Defaults to the max value of the dataset.\n\n        Examples\n        --------\n        This maps data value -4000 to 0., 0 to 0.5, and +10000 to 1.0; data\n        between is linearly interpolated::\n\n            >>> import matplotlib.colors as mcolors\n            >>> offset = mcolors.TwoSlopeNorm(vmin=-4000.,\n            ...                               vcenter=0., vmax=10000)\n            >>> data = [-4000., -2000., 0., 2500., 5000., 7500., 10000.]\n            >>> offset(data)\n            array([0., 0.25, 0.5, 0.625, 0.75, 0.875, 1.0])\n\nlib.matplotlib.colors.TwoSlopeNorm.autoscale_None:\n  Get vmin and vmax.\n\n        If vcenter isn't in the range [vmin, vmax], either vmin or vmax\n        is expanded so that vcenter lies in the middle of the modified range\n        [vmin, vmax].\n\nlib.matplotlib.colors.TwoSlopeNorm.__call__:\n  Map value to the interval [0, 1]. The *clip* argument is unused.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, exact code patterns, internal methods) that are not present in the provided documentation. These appear to be extrapolations from actual source code knowledge rather than from the given doc chunk.",
      "The documentation states that __call__ maps value to [0, 1] with clip argument unused - the answer's claim about extrapolation to -inf/+inf is not directly supported by the doc but is consistent with the statement that clip is unused.",
      "The documentation for autoscale_None says 'vcenter lies in the middle of the modified range' - the answer's description of mirroring is a plausible implementation but not explicitly stated in the doc.",
      "Claims about specific line numbers (2693, 2543-2546, 2679-2682, 2653-2658, 2691-2692, 2708-2709, 2304, 2664-2668) cannot be verified from the documentation provided.",
      "The class hierarchy claim (TwoSlopeNorm -> Normalize -> Norm(ABC)) includes details about Norm being an ABC with CallbackRegistry that are not in the provided documentation.",
      "The description of process_value's behavior with float32 and integer types is not supported by the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed comment-based rationale about tick locators and docstring references for process_value.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_28",
    "repo": "matplotlib",
    "question": "How does ConciseDateFormatter's format_ticks method determine the appropriate date granularity level and handle zero-value ticks?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `ConciseDateFormatter` implements a multi-level date formatting algorithm that determines the most appropriate level of date granularity. Here's how it works in detail:\n\n**Class hierarchy and overrides:**\n`ConciseDateFormatter` inherits from `ticker.Formatter` (which inherits from `TickHelper`). It overrides three key methods: `__call__`, `format_ticks`, and `format_data_short`. Crucially, it completely overrides `format_ticks` rather than relying on the base class implementation. The base `Formatter.format_ticks` calls `set_locs(values)` and then delegates to `__call__` for each value individually. `ConciseDateFormatter.format_ticks` skips `set_locs` entirely and processes all tick values collectively.\n\n**The `__call__` fallback:** When called individually (not through `format_ticks`), `__call__` creates a fresh `DateFormatter` with `self.defaultfmt` (hardcoded to `'%Y'`) and delegates to it. This means individual tick formatting always falls back to year-only display.\n\n**Level detection algorithm in `format_ticks`:**\n1. All tick values are converted from matplotlib float dates to datetimes via `num2date`, then their timetuple's first 6 components (year, month, day, hour, minute, second) are extracted into a numpy array.\n2. The algorithm iterates from level 5 (seconds) down to level 0 (years), checking `np.unique(tickdate[:, level])`. The first level with more than one unique value becomes the display level.\n3. Special case: if level < 2 (years or months) and the value 1 is among the unique values, `show_offset` is set to False.\n4. If all 6 columns are identical (edge case), level is set to 5 (seconds).\n\n**Zero-value detection:**\nA `zerovals` array `[0, 1, 1, 0, 0, 0, 0]` defines what counts as a \"zero\" at each level. For months and days, 1 is the zero value; for hours, minutes, seconds, 0 is the zero. When a tick's value at the detected level matches its zero value, the `zero_formats` string is used instead of the regular format.\n\n**Special seconds/microseconds handling (level >= 5):**\nFor seconds-level formatting, the zero detection uses `tickdatetime[nn].second == tickdatetime[nn].microsecond == 0` rather than the zerovals array. After formatting, trailing zeros in decimal portions are stripped uniformly \u2014 the minimum number of trailing zeros across all labels containing a decimal point is computed, then that many characters are removed from all decimal labels, with any remaining trailing dot also stripped. This ensures consistent decimal precision across labels.\n\n**Offset string logic:**\nThe offset string is formatted from the *last* tick datetime by default. However, if the locator's axis is an actual 'xaxis' or 'yaxis' (checked via `self._locator.axis.__name__`) AND the axis is inverted (`get_inverted()`), the *first* tick datetime is used instead.\n\n**Zero-formats fallback logic in `__init__`:**\nWhen custom `formats` are provided but `zero_formats` are not, `zero_formats` is set to `[''] + self.formats[:-1]` (shifting formats one level up). When neither is provided, the same shifting happens but with an additional override: `zero_formats[3]` is set to `'%b-%d'` instead of the shifted `'%d'`.\n\n**TeX wrapping:**\nThe `_wrap_in_tex` function wraps text for TeX rendering by: (1) replacing alphabetic sequences with `}$text$\\mathdefault{`, (2) wrapping `-` in braces as `{-}` and `:` as `{:}`, (3) replacing spaces with `\\;`, and (4) wrapping the whole string in `$\\mathdefault{...}$`.\n\n**Integration with `ConciseDateConverter`:**\nThe formatter can be auto-applied via `_SwitchableDateConverter`, which checks `rcParams['date.converter']` \u2014 when set to `'concise'`, a `ConciseDateConverter` is used, which creates both `AutoDateLocator` and `ConciseDateFormatter` together in its `axisinfo` method. This converter is registered globally for `np.datetime64`, `datetime.date`, and `datetime.datetime` types.",
    "rubric": [
      "Explains that tick values are converted to datetimes and their timetuple's first 6 components are extracted into a numpy array",
      "Describes the level detection: iterating from level 5 (seconds) down to level 0 (years), selecting the first level with more than one unique value",
      "Mentions the edge case where all columns are identical results in level being set to 5",
      "Explains the zerovals array [0, 1, 1, 0, 0, 0, 0] where months and days use 1 as zero, and hours/minutes/seconds use 0",
      "Describes that when a tick's value matches its zero value, the zero_formats string is used instead of the regular format",
      "Explains the special seconds-level handling where both second and microsecond must equal 0 to trigger zero format",
      "Mentions trailing zero stripping for level >= 5: computes minimum trailing zeros across decimal labels and strips uniformly",
      "Describes the show_offset special case: if level < 2 and 1 is among unique values, show_offset is set to False"
    ],
    "key_files": [
      "lib/matplotlib/dates.py",
      "lib/matplotlib/ticker.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/dates.py] lib.matplotlib.dates.ConciseDateFormatter\nlib.matplotlib.dates.ConciseDateFormatter:\n  A `.Formatter` which attempts to figure out the best format to use for the\n    date, and to make it as compact as possible, but still be complete. This is\n    most useful when used with the `AutoDateLocator`::\n\n    >>> locator = AutoDateLocator()\n    >>> formatter = ConciseDateFormatter(locator)\n\n    Parameters\n    ----------\n    locator : `.ticker.Locator`\n        Locator that this axis is using.\n\n    tz : str or `~datetime.tzinfo`, default: :rc:`timezone`\n        Ticks timezone, passed to `.dates.num2date`.\n\n    formats : list of 6 strings, optional\n        Format strings for 6 levels of tick labelling: mostly years,\n        months, days, hours, minutes, and seconds.  Strings use\n        the same format codes as `~datetime.datetime.strftime`.  Default is\n        ``['%Y', '%b', '%d', '%H:%M', '%H:%M', '%S.%f']``\n\n    zero_formats : list of 6 strings, optional\n        Format strings for tick labels that are \"zeros\" for a given tick\n        level.  For instance, if most ticks are months, ticks around 1 Jan 2005\n        will be labeled \"Dec\", \"2005\", \"Feb\".  The default is\n        ``['', '%Y', '%b', '%b-%d', '%H:%M', '%H:%M']``\n\n    offset_formats : list of 6 strings, optional\n        Format strings for the 6 levels that is applied to the \"offset\"\n        string found on the right side of an x-axis, or top of a y-axis.\n        Combined with the tick labels this should completely specify the\n        date.  The default is::\n\n            ['', '%Y', '%Y-%b', '%Y-%b-%d', '%Y-%b-%d', '%Y-%b-%d %H:%M']\n\n    show_offset : bool, default: True\n        Whether to show the offset or not.\n\n    usetex : bool, default: :rc:`text.usetex`\n        To enable/disable the use of TeX's math mode for rendering the results\n        of the formatter.\n\n    Examples\n    --------\n    See :doc:`/gallery/ticks/date_concise_formatter`\n\n    .. plot::\n\n        import datetime\n        import matplotlib.dates as mdates\n\n        base = datetime.datetime(2005, 2, 1)\n        dates = np.array([base + datetime.timedelta(hours=(2 * i))\n                          for i in range(732)])\n        N = len(dates)\n        np.random.seed(19680801)\n        y = np.cumsum(np.random.randn(N))\n\n        fig, ax = plt.subplots(constrained_layout=True)\n        locator = mdates.AutoDateLocator()\n        formatter = mdates.ConciseDateFormatter(locator)\n        ax.xaxis.set_major_locator(locator)\n        ax.xaxis.set_major_formatter(formatter)\n\n        ax.plot(dates, y)\n        ax.set_title('Concise Date Formatter')\n\nlib.matplotlib.dates.ConciseDateFormatter.__init__:\n  Autoformat the date labels.  The default format is used to form an\n        initial string, and then redundant elements are removed.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (e.g., zerovals array values, _wrap_in_tex internals, _SwitchableDateConverter, axis inversion logic, trailing zeros stripping for seconds) that go far beyond what the documentation provides. These are plausible code-level extrapolations but cannot be verified from the given documentation alone.",
      "The claim that `__call__` uses a hardcoded `'%Y'` defaultfmt is not mentioned in the documentation and cannot be verified from it.",
      "The detailed level detection algorithm (iterating from level 5 down to level 0, checking np.unique) is not described in the documentation \u2014 it's a plausible implementation detail but unverifiable from the doc.",
      "The zero_formats fallback logic description ('zero_formats[3] is set to %b-%d') is consistent with the documented default of '%b-%d' at index 3 of zero_formats, so this is supported.",
      "The default formats, zero_formats, and offset_formats listed in the answer's descriptions are consistent with the documentation's stated defaults.",
      "The claim about ConciseDateConverter and _SwitchableDateConverter is entirely outside the scope of the provided documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale phrases were removed or trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_29",
    "repo": "matplotlib",
    "question": "What is the architecture and lazy-parsing strategy of `PsfontsMap`, and how does it handle duplicate keys, line validation, and effects parsing?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `PsfontsMap` class in `lib/matplotlib/dviread.py` is a lazy-parsing font map that maps TeX font names to PostScript font information. Here is a detailed technical description of its architecture and behavior:\n\n**Data Structure**: `PsfontsMap` uses `__slots__ = ('_filename', '_unparsed', '_parsed')` to restrict instance attributes. It employs a two-tier lazy parsing approach:\n1. `_unparsed`: A dict mapping byte-string tfmnames to lists of raw (unparsed) lines from the map file.\n2. `_parsed`: A dict mapping byte-string tfmnames to `PsFont` namedtuple instances.\n\n**Singleton Caching**: The `__new__` method is decorated with `@lru_cache` (from `functools`), making instances cached by filename. Calling `PsfontsMap(filename)` with the same filename returns the exact same object.\n\n**Lazy On-Demand Parsing**: During construction (`__new__`), the file is opened in binary mode and lines are stored in `_unparsed` keyed by the first whitespace-delimited word (the tfmname), using `setdefault` to allow multiple lines per key. Actual parsing only happens in `__getitem__` when a font is looked up.\n\n**Duplicate Key Handling**: When `__getitem__` is called, all lines for that tfmname are popped from `_unparsed` and iterated. The iteration **breaks at the first line that parses successfully** (i.e., where `_parse_and_cache_line` returns `True`). This means only the first valid entry for a duplicate key is used.\n\n**PsFont namedtuple**: The parsed result is stored as a `PsFont = namedtuple('PsFont', 'texname psname effects encoding filename')`.\n\n**Line Validation and Rejection**: `_parse_and_cache_line` skips lines starting with space, `%`, `*`, `;`, or `#`. It uses the regex `br'\"([^\"]*)(?:\"|$)|(\\S+)'` to tokenize lines into quoted and unquoted segments. Lines are rejected (return `None`) if:\n- A TrueType font (`.ttf`/`.ttc`) is subsetted (`<` prefix) but has no encoding file\n- Slant/Extend effects are specified on non-Type-1 fonts (TrueType or OTF)\n- Absolute slant value > 1\n- Absolute extend value > 2\n\n**Effects Parsing**: The special (double-quoted) field is parsed by calling `reversed(special.split())` and using `next(words)` to read the numeric value preceding `SlantFont` or `ExtendFont` keywords. The use of `reversed` means that in a string like `\"-0.1 SlantFont 1.2 ExtendFont\"`, each keyword is encountered before its associated numeric value.\n\n**File Resolution**: Both `encodingfile` and `fontfile` are resolved via `find_tex_file()`, which first tries a long-lived `luatex` subprocess (via `_LuatexKpsewhich` singleton that runs `kpsewhich.lua`), falling back to directly invoking `kpsewhich`. The `kpsewhich.lua` script calls `kpse.set_program_name(\"latex\")` and `kpse.init_prog(\"\", 600, \"ljfour\")` then loops reading filenames and printing lookup results.\n\n**Usage Context**: `PsfontsMap` is used by `DviFont.resolve_path()` and `DviFont._index_dvi_to_freetype()` to find font paths and encoding information, and by `backend_pdf.py`'s `_embedTeXFont` to embed Type-1 fonts in PDF output with proper encoding and effects applied.",
    "rubric": [
      "Mentions the three-slot data structure: `_filename`, `_unparsed`, and `_parsed`, with `__slots__` restricting attributes",
      "Explains that `__new__` is decorated with `@lru_cache` to cache instances by filename (singleton per filename)",
      "Describes the two-tier lazy parsing: lines stored in `_unparsed` keyed by tfmname during construction, parsed on demand in `__getitem__`",
      "Explains duplicate key handling: all lines for a tfmname are popped from `_unparsed` and iterated, breaking at the first successfully parsed line",
      "Identifies the `PsFont` namedtuple with fields: texname, psname, effects, encoding, filename",
      "Lists line rejection conditions: lines starting with space/`%`/`*`/`;`/`#`; subsetted TrueType without encoding; effects on non-Type-1 fonts; slant > 1; extend > 2",
      "Describes effects parsing: `reversed(special.split())` is iterated and `next(words)` reads the numeric value that precedes `SlantFont`/`ExtendFont` keywords",
      "Explains file resolution via `find_tex_file()`, which uses `_LuatexKpsewhich` (a long-lived luatex subprocess) with fallback to directly calling kpsewhich"
    ],
    "key_files": [
      "lib/matplotlib/dviread.py",
      "lib/matplotlib/backends/backend_pdf.py",
      "lib/matplotlib/mpl-data/kpsewhich.lua",
      "lib/matplotlib/tests/test_dviread.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/dviread.py] lib.matplotlib.dviread.PsfontsMap\nlib.matplotlib.dviread.PsfontsMap:\n  A psfonts.map formatted file, mapping TeX fonts to PS fonts.\n\n    Parameters\n    ----------\n    filename : str or path-like\n\n    Notes\n    -----\n    For historical reasons, TeX knows many Type-1 fonts by different\n    names than the outside world. (For one thing, the names have to\n    fit in eight characters.) Also, TeX's native fonts are not Type-1\n    but Metafont, which is nontrivial to convert to PostScript except\n    as a bitmap. While high-quality conversions to Type-1 format exist\n    and are shipped with modern TeX distributions, we need to know\n    which Type-1 fonts are the counterparts of which native fonts. For\n    these reasons a mapping is needed from internal font names to font\n    file names.\n\n    A texmf tree typically includes mapping files called e.g.\n    :file:`psfonts.map`, :file:`pdftex.map`, or :file:`dvipdfm.map`.\n    The file :file:`psfonts.map` is used by :program:`dvips`,\n    :file:`pdftex.map` by :program:`pdfTeX`, and :file:`dvipdfm.map`\n    by :program:`dvipdfm`. :file:`psfonts.map` might avoid embedding\n    the 35 PostScript fonts (i.e., have no filename for them, as in\n    the Times-Bold example above), while the pdf-related files perhaps\n    only avoid the \"Base 14\" pdf fonts. But the user may have\n    configured these files differently.\n\n    Examples\n    --------\n    >>> map = PsfontsMap(find_tex_file('pdftex.map'))\n    >>> entry = map[b'ptmbo8r']\n    >>> entry.texname\n    b'ptmbo8r'\n    >>> entry.psname\n    b'Times-Bold'\n    >>> entry.encoding\n    '/usr/local/texlive/2008/texmf-dist/fonts/enc/dvips/base/8r.enc'\n    >>> entry.effects\n    {'slant': 0.16700000000000001}\n    >>> entry.filename\n\nlib.matplotlib.dviread.PsfontsMap._parse_and_cache_line:\n  Parse a line in the font mapping file.\n\n        The format is (partially) documented at\n        http://mirrors.ctan.org/systems/doc/pdftex/manual/pdftex-a.pdf\n        https://tug.org/texinfohtml/dvips.html#psfonts_002emap\n        Each line can have the following fields:\n\n        - tfmname (first, only required field),\n        - psname (defaults to tfmname, must come immediately after tfmname if\n          present),\n        - fontflags (integer, must come immediately after psname if present,\n          ignored by us),\n        - special (SlantFont and ExtendFont, only field that is double-quoted),\n        - fontfile, encodingfile (optional, prefixed by <, <<, or <[; << always\n          precedes a font, <[ always precedes an encoding, < can precede either\n          but then an encoding file must have extension .enc; < and << also\n          request different font subsetting behaviors but we ignore that; < can\n          be separated from the filename by whitespace).\n\n        special, fontfile, and encodingfile can appear in any order.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not mention __slots__, _unparsed, _parsed attributes, or the two-tier lazy parsing approach - these are plausible implementation details but not verifiable from the provided documentation.",
      "The documentation does not mention @lru_cache singleton caching on __new__ - this is a plausible implementation detail but not verifiable.",
      "The documentation does not mention the specific regex br'\"([^\"]*)(?:\"|$)|(\\S+)' - this is a plausible implementation detail but not verifiable.",
      "The documentation does not mention TrueType font validation rules, absolute slant > 1 or extend > 2 rejection criteria - these are plausible but not verifiable.",
      "The documentation does not mention the reversed(special.split()) parsing approach - this is a plausible implementation detail but not verifiable.",
      "The documentation does not mention find_tex_file(), _LuatexKpsewhich, kpsewhich.lua, or the fallback mechanism - these are plausible but not verifiable.",
      "The documentation does not mention DviFont.resolve_path(), DviFont._index_dvi_to_freetype(), or backend_pdf.py usage context - these are plausible but not verifiable.",
      "The claim about PsFont namedtuple 'defined at line 1048' is very specific and unverifiable from the documentation.",
      "The effects example in the documentation shows {'slant': 0.16700000000000001} which is consistent with the answer's description of SlantFont/ExtendFont parsing, though the specific reversed() mechanism is not documented.",
      "The _parse_and_cache_line documentation says lines can have fields including 'special (SlantFont and ExtendFont, only field that is double-quoted)' which is consistent with the answer's description of effects parsing.",
      "The documentation mentions '<, <<, or <[' prefixes for fontfile and encodingfile, which is consistent with the answer's mention of '< prefix' for subsetting."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rationale comments ('hundreds of milliseconds') and a line number reference needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_31",
    "repo": "matplotlib",
    "question": "Where does the `pyplot.xticks()` call chain lead, and how are the underlying axis methods wrapped on both regular and secondary axes?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `pyplot.xticks()` function is defined in `lib/matplotlib/pyplot.py`. The key mechanisms in the call chain are:\n\n1. **Getting ticks (no ticks arg)**: Uses `ax.get_xticks(minor=minor)`, which is an `_axis_method_wrapper` wrapping the corresponding `xaxis` method.\n\n2. **The `_axis_method_wrapper` class** in `_base.py` uses the `__set_name__` descriptor protocol. It creates a wrapper function using `operator.attrgetter` and copies the signature from `maxis.Axis.<method_name>` using `inspect.signature`. The wrapper explicitly sets `__module__`, `__name__`, `__qualname__`, `__doc__`, and `__signature__` on the generated function. It also validates that expected doc_sub keys are present in the docstring, raising `ValueError` if not.\n\n3. **SecondaryAxis.set_xticks** in `_secondary_axes.py` additionally sets `self._ticks_set = True` and raises `TypeError` if called on a y-oriented secondary axis (`if self._orientation == \"y\": raise TypeError(\"Cannot set xticks on a secondary y-axis\")`). Similarly, `set_yticks` raises `TypeError` for x-oriented secondary axes.",
    "rubric": [
      "Identifies that pyplot.xticks() is defined in pyplot.py and delegates to ax.get_xticks/ax.set_xticks on the current axes",
      "Explains that get_xticks (and similar methods) are created via the _axis_method_wrapper descriptor class in _base.py",
      "Describes that _axis_method_wrapper uses __set_name__ to generate a wrapper function using operator.attrgetter that delegates to the corresponding xaxis/yaxis method",
      "Notes that _axis_method_wrapper copies __module__, __name__, __qualname__, __doc__, and __signature__ from the Axis method onto the generated wrapper",
      "Mentions that _axis_method_wrapper validates doc_sub keys are present in the docstring and raises ValueError if not",
      "Explains that SecondaryAxis.set_xticks sets self._ticks_set = True and raises TypeError if called on a y-oriented secondary axis (and vice versa for set_yticks)"
    ],
    "key_files": [
      "lib/matplotlib/pyplot.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/axis.py",
      "lib/matplotlib/ticker.py",
      "lib/matplotlib/axes/_secondary_axes.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/pyplot.py] lib.matplotlib.pyplot.xticks\nGet or set the current tick locations and labels of the x-axis.\n\n    Pass no arguments to return the current values without modifying them.\n\n    Parameters\n    ----------\n    ticks : array-like, optional\n        The list of xtick locations.  Passing an empty list removes all xticks.\n    labels : array-like, optional\n        The labels to place at the given *ticks* locations.  This argument can\n        only be passed if *ticks* is passed as well.\n    minor : bool, default: False\n        If ``False``, get/set the major ticks/labels; if ``True``, the minor\n        ticks/labels.\n    **kwargs\n        `.Text` properties can be used to control the appearance of the labels.\n\n        .. warning::\n\n            This only sets the properties of the current ticks, which is\n            only sufficient if you either pass *ticks*, resulting in a\n            fixed list of ticks, or if the plot is static.\n\n            Ticks are not guaranteed to be persistent. Various operations\n            can create, delete and modify the Tick instances. There is an\n            imminent risk that these settings can get lost if you work on\n            the figure further (including also panning/zooming on a\n            displayed figure).\n\n            Use `~.pyplot.tick_params` instead if possible.\n\n\n    Returns\n    -------\n    locs\n        The list of xtick locations.\n    labels\n        The list of xlabel `.Text` objects.\n\n    Notes\n    -----\n    Calling this function with no arguments (e.g. ``xticks()``) is the pyplot\n    equivalent of calling `~.Axes.get_xticks` and `~.Axes.get_xticklabels` on\n    the current Axes.\n    Calling this function with arguments is the pyplot equivalent of calling\n    `~.Axes.set_xticks` and `~.Axes.set_xticklabels` on the current Axes.\n\n    Examples\n    --------\n    >>> locs, labels = xticks()  # Get the current locations and labels.\n    >>> xticks(np.arange(0, 1, step=0.2))  # Set label locations.\n    >>> xticks(np.arange(3), ['Tom', 'Dick', 'Sue'])  # Set text labels.\n    >>> xticks([0, 1, 2], ['January', 'February', 'March'],\n    ...        rotation=20)  # Set text labels and properties.\n    >>> xticks([])  # Disable xticks.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.65,
    "strip_verify_summary": "Most claims reference function bodies and line numbers from truncated/unavailable code sections; only the _axis_method_wrapper descriptor pattern and SecondaryAxis override behavior are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_30",
    "repo": "matplotlib",
    "question": "Why does PolarAxes use in-place matrix manipulation and invalidation for theta direction/offset instead of creating new transform objects?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `PolarAxes` class in `lib/matplotlib/projections/polar.py` uses a specific design pattern where theta direction and offset are implemented as in-place modifications to existing `Affine2D` transform matrices rather than creating new transforms.\n\n**Key design decisions:**\n\n1. **Direct matrix manipulation for theta direction/offset**: `set_theta_direction()` and `set_theta_offset()` directly modify entries in pre-existing `Affine2D` matrices (`self._direction` and `self._theta_offset`) rather than creating new transform objects. For example, `set_theta_direction` sets `mtx[0, 0]` to either -1 or 1 (the x-scale factor), and `set_theta_offset` sets `mtx[0, 2]` (the x-translation). They then call `.invalidate()` to propagate the change up the transform tree via weak references to parent nodes. The `transData` pipeline is a 6-step composite: transScale + transShift + transProjection + transProjectionAffine + transWedge + transAxes.\n\n2. **`_originViewLim` as a LockableBbox**: The `set_rorigin()` method uses `self._originViewLim.locked_y0 = rorigin` which sets a locked value in a `LockableBbox`. This `LockableBbox` wraps `self.viewLim` and overrides certain bounds (y0 in this case) with fixed values while letting other bounds pass through from the child bbox. When `rorigin` is `None`, the mask allows the original viewLim value to show through.\n\n3. **`can_zoom()` returns False while `can_pan()` returns True**: The pan button is repurposed: button 1 allows dragging r-labels (only when clicking near the label angle, within \u00b1\u03c0/45 radians), while button 3 performs radial zooming by scaling rmax based on the ratio of current to starting radial positions.\n\n4. **The `draw()` method switches the y-axis text transform** between `_yaxis_transform` (for partial circles, placing labels on the start edge) and `_r_label_position + transData` (for full circles, placing labels at the configurable rlabel_position angle). This switch triggers `yaxis.reset_ticks()` and `yaxis.set_clip_path()` to update the tick display.\n\n5. **`ThetaAxis._set_scale` raises NotImplementedError for non-linear scales**: After setting linear scale, it customizes the AutoLocator with `steps=[1, 1.5, 3, 4.5, 9, 10]`.\n\n6. **`RadialAxis` wraps any locator in `RadialLocator`**: The `set_major_locator` override ensures all locators are wrapped in `RadialLocator`, which filters out ticks at or below the radial origin for full-circle non-annular views (where `rmin <= rorigin`).\n\n7. **`get_rsign()` enables inverted radial axes**: It returns `np.sign(self._originViewLim.y1 - self._originViewLim.y0)`, which is -1 when y1 < y0 (inverted r-axis). This sign is multiplied into the polar transform's radius calculation.\n\n8. **`set_thetalim` delegates to `set_xlim` then validates**: It converts degree kwargs to radians, calls `set_xlim`, and then checks if `abs(new_max - new_min) > 2*pi`. If invalid, it un-does the change by restoring orig_lim before raising ValueError.",
    "rubric": [
      "Explains that set_theta_direction and set_theta_offset directly modify entries in pre-existing Affine2D matrices (self._direction and self._theta_offset) rather than replacing them with new transform objects",
      "Mentions that set_theta_direction sets mtx[0,0] to -1 or 1 and set_theta_offset sets mtx[0,2] (x-translation)",
      "Explains that invalidate() is called to propagate changes through the transform tree (via weak references to parent/dependent nodes)",
      "Describes the transData pipeline as a multi-step composite (transScale + transShift + transProjection + transProjectionAffine + transWedge + transAxes) that would break if objects were replaced rather than mutated",
      "Notes that _originViewLim is a LockableBbox wrapping viewLim, used by set_rorigin to lock y0 while letting other bounds pass through",
      "Mentions that the draw() method switches yaxis text transform between _yaxis_transform and _r_label_position + transData depending on whether the plot is a full circle",
      "Describes that RadialAxis.set_major_locator wraps any locator in RadialLocator which filters ticks at or below radial origin for full-circle non-annular views",
      "Notes that can_zoom() returns False while can_pan() returns True, with button 1 dragging r-labels and button 3 performing radial zoom"
    ],
    "key_files": [
      "lib/matplotlib/projections/polar.py",
      "lib/matplotlib/transforms.py",
      "lib/matplotlib/projections/__init__.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/projections/polar.py] lib.matplotlib.projections.polar.PolarAxes\nlib.matplotlib.projections.polar.PolarAxes:\n  A polar graph projection, where the input dimensions are *theta*, *r*.\n\n    Theta starts pointing east and goes anti-clockwise.\n\nlib.matplotlib.projections.polar.PolarAxes.set_thetamax:\n  Set the maximum theta limit in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.get_thetamax:\n  Return the maximum theta limit in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.set_thetamin:\n  Set the minimum theta limit in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.get_thetamin:\n  Get the minimum theta limit in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.set_thetalim:\n  Set the minimum and maximum theta values.\n\n        Can take the following signatures:\n\n        - ``set_thetalim(minval, maxval)``: Set the limits in radians.\n        - ``set_thetalim(thetamin=minval, thetamax=maxval)``: Set the limits\n          in degrees.\n\n        where minval and maxval are the minimum and maximum limits. Values are\n        wrapped in to the range :math:`[0, 2\\pi]` (in radians), so for example\n        it is possible to do ``set_thetalim(-np.pi / 2, np.pi / 2)`` to have\n        an axis symmetric around 0. A ValueError is raised if the absolute\n        angle difference is larger than a full circle.\n\nlib.matplotlib.projections.polar.PolarAxes.set_theta_offset:\n  Set the offset for the location of 0 in radians.\n\nlib.matplotlib.projections.polar.PolarAxes.get_theta_offset:\n  Get the offset for the location of 0 in radians.\n\nlib.matplotlib.projections.polar.PolarAxes.set_theta_zero_location:\n  Set the location of theta's zero.\n\n        This simply calls `set_theta_offset` with the correct value in radians.\n\n        Parameters\n        ----------\n        loc : str\n            May be one of \"N\", \"NW\", \"W\", \"SW\", \"S\", \"SE\", \"E\", or \"NE\".\n        offset : float, default: 0\n            An offset in degrees to apply from the specified *loc*. **Note:**\n            this offset is *always* applied counter-clockwise regardless of\n            the direction setting.\n\nlib.matplotlib.projections.polar.PolarAxes.set_theta_direction:\n  Set the direction in which theta increases.\n\n        clockwise, -1:\n           Theta increases in the clockwise direction\n\n        counterclockwise, anticlockwise, 1:\n           Theta increases in the counterclockwise direction\n\nlib.matplotlib.projections.polar.PolarAxes.get_theta_direction:\n  Get the direction in which theta increases.\n\n        -1:\n           Theta increases in the clockwise direction\n\n        1:\n           Theta increases in the counterclockwise direction\n\nlib.matplotlib.projections.polar.PolarAxes.set_rmax:\n  Set the outer radial limit.\n\n        Parameters\n        ----------\n        rmax : float\n\nlib.matplotlib.projections.polar.PolarAxes.get_rmax:\n  Returns\n        -------\n        float\n            Outer radial limit.\n\nlib.matplotlib.projections.polar.PolarAxes.set_rmin:\n  Set the inner radial limit.\n\n        Parameters\n        ----------\n        rmin : float\n\nlib.matplotlib.projections.polar.PolarAxes.get_rmin:\n  Returns\n        -------\n        float\n            The inner radial limit.\n\nlib.matplotlib.projections.polar.PolarAxes.set_rorigin:\n  Update the radial origin.\n\n        Parameters\n        ----------\n        rorigin : float\n\nlib.matplotlib.projections.polar.PolarAxes.get_rorigin:\n  Returns\n        -------\n        float\n\nlib.matplotlib.projections.polar.PolarAxes.set_rlim:\n  Set the radial axis view limits.\n\n        This function behaves like `.Axes.set_ylim`, but additionally supports\n        *rmin* and *rmax* as aliases for *bottom* and *top*.\n\n        See Also\n        --------\n        .Axes.set_ylim\n\nlib.matplotlib.projections.polar.PolarAxes.get_rlabel_position:\n  Returns\n        -------\n        float\n            The theta position of the radius labels in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.set_rlabel_position:\n  Update the theta position of the radius labels.\n\n        Parameters\n        ----------\n        value : number\n            The angular position of the radius labels in degrees.\n\nlib.matplotlib.projections.polar.PolarAxes.set_thetagrids:\n  Set the theta gridlines in a polar plot.\n\n        Parameters\n        ----------\n        angles : tuple with floats, degrees\n            The angles of the theta gridlines.\n\n        labels : tuple with strings or None\n            The labels to use at each theta gridline. The\n            `.projections.polar.ThetaFormatter` will be used if None.\n\n        fmt : str or None\n            Format string used in `matplotlib.ticker.FormatStrFormatter`.\n            For example '%f'. Note that the angle that is used is in\n            radians.\n\n        Returns\n        -------\n        lines : list of `.lines.Line2D`\n            The theta gridlines.\n\n        labels : list of `.text.Text`\n            The tick labels.\n\n        Other Parameters\n        ----------------\n        **kwargs\n            *kwargs* are optional `.Text` properties for the labels.\n\n            .. warning::\n\n                This only sets the properties of the current ticks.\n                Ticks are not guaranteed to be persistent. Various operations\n                can create, delete and modify the Tick instances. There is an\n                imminent risk that these settings can get lost if you work on\n                the figure further (including also panning/zooming on a\n                displayed figure).\n\n                Use `.set_tick_params` instead if possible.\n\n        See Also\n        --------\n        .PolarAxes.set_rgrids\n        .Axis.get_gridlines\n        .Axis.get_ticklabels\n\nlib.matplotlib.projections.polar.PolarAxes.set_rgrids:\n  Set the radial gridlines on a polar plot.\n\n        Parameters\n        ----------\n        radii : tuple with floats\n            The radii for the radial gridlines\n\n        labels : tuple with strings or None\n            The labels to use at each radial gridline. The\n            `matplotlib.ticker.ScalarFormatter` will be used if None.\n\n        angle : float\n            The angular position of the radius labels in degrees.\n\n        fmt : str or None\n            Format string used in `matplotlib.ticker.FormatStrFormatter`.\n            For example '%f'.\n\n        Returns\n        -------\n        lines : list of `.lines.Line2D`\n            The radial gridlines.\n\n        labels : list of `.text.Text`\n            The tick labels.\n\n        Other Parameters\n        ----------------\n        **kwargs\n            *kwargs* are optional `.Text` properties for the labels.\n\n            .. warning::\n\n                This only sets the properties of the current ticks.\n                Ticks are not guaranteed to be persistent. Various operations\n                can create, delete and modify the Tick instances. There is an\n                imminent risk that these settings can get lost if you work on\n                the figure further (including also panning/zooming on a\n                displayed figure).\n\n                Use `.set_tick_params` instead if possible.\n\n        See Also\n        --------\n        .PolarAxes.set_thetagrids\n        .Axis.get_gridlines\n        .Axis.get_ticklabels\n\nlib.matplotlib.projections.polar.PolarAxes.get_data_ratio:\n  Return the aspect ratio of the data itself.  For a polar plot,\n        this should always be 1.0\n\nlib.matplotlib.projections.polar.PolarAxes.can_zoom:\n  Return whether this Axes supports the zoom box button functionality.\n\n        A polar Axes does not support zoom boxes.\n\nlib.matplotlib.projections.polar.PolarAxes.can_pan:\n  Return whether this Axes supports the pan/zoom button functionality.\n\n        For a polar Axes, this is slightly misleading. Both panning and\n        zooming are performed by the same button. Panning is performed\n        in azimuth while zooming is done along the radial.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (matrix manipulation, LockableBbox, RadialLocator, get_rsign, etc.) that are not present in the documentation and cannot be verified from it. These appear to be plausible code-level details but are not supported by the provided documentation.",
      "Point 8 claims set_thetalim 'delegates to set_xlim then validates' - the documentation says values are 'wrapped in to the range [0, 2\u03c0]' and 'A ValueError is raised if the absolute angle difference is larger than a full circle', which is consistent with the claim about validation, but the delegation to set_xlim is not documented.",
      "Point 3 claims can_zoom() returns False - the documentation says 'A polar Axes does not support zoom boxes' which is consistent. The claim that can_pan() returns True is consistent with the documentation saying it supports pan/zoom button functionality.",
      "The documentation states set_thetalim positional arguments are in radians while keyword arguments (thetamin/thetamax) are in degrees. The answer's point 8 says it 'converts degree kwargs to radians' which is consistent with this.",
      "Most of the answer's detailed claims about internal implementation (transform chains, matrix entries, weak references, specific method behaviors) go far beyond what the documentation covers and appear to be based on source code reading rather than the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale statements rely on documentation/comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_32",
    "repo": "matplotlib",
    "question": "How does `pyplot.yticks()` handle the get/set branching for tick locations and labels, including how tick locations and formatters are set internally?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `pyplot.yticks()` function (lines 2398-2480 in `lib/matplotlib/pyplot.py`) implements a dual get/set pattern that branches based on whether `ticks` and `labels` arguments are provided:\n\n**Algorithm/Implementation Steps:**\n\n1. **Get current axes**: Calls `gca()` which delegates to `gcf().gca()` to obtain the current Axes object.\n\n2. **Ticks handling (two branches)**:\n   - If `ticks is None` (getter mode): calls `ax.get_yticks(minor=minor)`, which is an `_axis_method_wrapper` around `yaxis.get_ticklocs()`. The `get_ticklocs()` method simply returns `self.get_minorticklocs()` if minor else `self.get_majorticklocs()`. `get_majorticklocs()` directly calls `self.major.locator()`. If `labels` is also provided without `ticks`, raises a `TypeError` with message \"yticks(): Parameter 'labels' can't be set without setting 'ticks'\".\n   - If `ticks` is provided (setter mode): calls `ax.set_yticks(ticks, minor=minor)`, which wraps `yaxis.set_ticks()`. Internally, `set_ticks` calls `_set_tick_locations()` which: (a) converts units via `self.convert_units(ticks)`, (b) creates a `FixedLocator` from the ticks, (c) if ticks are non-empty, expands the view interval on all shared axes to include `min(ticks)` to `max(ticks)` and fires a `ylim_changed` callback if limits changed, (d) sets the locator as either minor or major, and (e) returns the Tick objects (not tick locations) via `get_major_ticks(len(ticks))` or `get_minor_ticks(len(ticks))`.\n\n3. **Labels handling (two branches)**:\n   - If `labels is None`: calls `ax.get_yticklabels(minor=minor)`, which wraps `yaxis.get_ticklabels()`. Then iterates over each label Text object and calls `l._internal_update(kwargs)` to apply any `**kwargs` (Text properties like `rotation`, `fontsize`, etc.) directly. `_internal_update` is defined in `Artist` and calls `_update_props` without prenormalizing kwargs.\n   - If `labels` is provided: calls `ax.set_yticklabels(labels, minor=minor, **kwargs)`, which wraps `yaxis.set_ticklabels()`. Inside `set_ticklabels`: it first extracts text from any Text objects in labels. Then it chooses a formatter: `NullFormatter` for empty labels, `FuncFormatter` with a dict-based lookup (`_format_with_dict`) if the locator is already a `FixedLocator` (with validation that label count matches locator location count), or `FixedFormatter` (with a warning) otherwise. It then iterates through ticks setting `tick.label1` and `tick.label2` text and applying kwargs via `_internal_update`. Only visible labels are returned.\n\n4. **Return value**: Always returns a tuple `(locs, labels_out)` \u2014 locations and a list of Text objects.\n\n**Key implementation details:**\n- When ticks are set, `_set_tick_locations` uses `FixedLocator` and expands view limits across shared axes.\n- The `_axis_method_wrapper` class uses `__set_name__` descriptor protocol to replace itself with a proper wrapper function at class definition time, copying the signature from `Axis`'s method.\n- When labels are provided without ticks, a `TypeError` is raised (not silently ignored).\n- `set_ticklabels` uses `_format_with_dict` (a static method that returns `tickd.get(x, \"\")`) to create a `FuncFormatter` when a `FixedLocator` is in use, rather than always using `FixedFormatter`.\n- `get_minorticklocs()` removes minor ticks that duplicate major ticks by comparing transformed locations within a tolerance of `(hi - lo) * 1e-5`.",
    "rubric": [
      "Explains that yticks() branches based on whether ticks is None (getter) or provided (setter), calling get_yticks or set_yticks accordingly",
      "Mentions that when ticks is provided without labels, _set_tick_locations converts units, creates a FixedLocator, and expands the view interval on shared axes",
      "Notes that providing labels without ticks raises a TypeError",
      "Explains the labels branch: if labels is None, gets existing labels and applies kwargs via _internal_update; if provided, calls set_yticklabels which delegates to set_ticklabels",
      "Describes that set_ticklabels chooses a formatter: NullFormatter for empty labels, FuncFormatter (using _format_with_dict) when a FixedLocator is already set, or FixedFormatter otherwise",
      "Mentions the _axis_method_wrapper descriptor mechanism that uses __set_name__ to replace itself with a wrapper function delegating to the corresponding Axis method",
      "States that the function always returns a tuple of (locations, label Text objects)"
    ],
    "key_files": [
      "lib/matplotlib/pyplot.py",
      "lib/matplotlib/axis.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/artist.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/pyplot.py] lib.matplotlib.pyplot.yticks\nGet or set the current tick locations and labels of the y-axis.\n\n    Pass no arguments to return the current values without modifying them.\n\n    Parameters\n    ----------\n    ticks : array-like, optional\n        The list of ytick locations.  Passing an empty list removes all yticks.\n    labels : array-like, optional\n        The labels to place at the given *ticks* locations.  This argument can\n        only be passed if *ticks* is passed as well.\n    minor : bool, default: False\n        If ``False``, get/set the major ticks/labels; if ``True``, the minor\n        ticks/labels.\n    **kwargs\n        `.Text` properties can be used to control the appearance of the labels.\n\n        .. warning::\n\n            This only sets the properties of the current ticks, which is\n            only sufficient if you either pass *ticks*, resulting in a\n            fixed list of ticks, or if the plot is static.\n\n            Ticks are not guaranteed to be persistent. Various operations\n            can create, delete and modify the Tick instances. There is an\n            imminent risk that these settings can get lost if you work on\n            the figure further (including also panning/zooming on a\n            displayed figure).\n\n            Use `~.pyplot.tick_params` instead if possible.\n\n    Returns\n    -------\n    locs\n        The list of ytick locations.\n    labels\n        The list of ylabel `.Text` objects.\n\n    Notes\n    -----\n    Calling this function with no arguments (e.g. ``yticks()``) is the pyplot\n    equivalent of calling `~.Axes.get_yticks` and `~.Axes.get_yticklabels` on\n    the current Axes.\n    Calling this function with arguments is the pyplot equivalent of calling\n    `~.Axes.set_yticks` and `~.Axes.set_yticklabels` on the current Axes.\n\n    Examples\n    --------\n    >>> locs, labels = yticks()  # Get the current locations and labels.\n    >>> yticks(np.arange(0, 1, step=0.2))  # Set label locations.\n    >>> yticks(np.arange(3), ['Tom', 'Dick', 'Sue'])  # Set text labels.\n    >>> yticks([0, 1, 2], ['January', 'February', 'March'],\n    ...        rotation=45)  # Set text labels and properties.\n    >>> yticks([])  # Disable yticks.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code logic; only one small design rationale phrase came from a comment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_33",
    "repo": "matplotlib",
    "question": "What does matplotlib's validating `cycler()` function do, and how does it relate to the external cycler package?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `cycler()` function in `lib/matplotlib/rcsetup.py` is matplotlib's validating wrapper around the `cycler.cycler` function from the external `cycler` package. Here are the key implementation details:\n\n**Import aliasing**: The original `cycler` package's `cycler` function is imported as `ccycler`, its `Cycler` class as `Cycler`, and its `concat` function as `cconcat` (line 39: `from cycler import Cycler, concat as cconcat, cycler as ccycler`). This aliasing avoids a name collision with matplotlib's own validating `cycler` function defined in the same module.\n\n**Property validation**: The function uses a `_prop_validators` dictionary (line 700) that maps recognized property names to their specific validators: `color`, `linewidth`, `linestyle`, `facecolor`, `edgecolor`, `joinstyle`, `capstyle`, `fillstyle`, `markerfacecolor`, `markersize`, `markeredgewidth`, `markeredgecolor`, `markevery`, `alpha`, `marker`, `hatch`, and `dashes`.\n\n**Alias normalization**: Property aliases are defined in `_prop_aliases` (line 720) which maps 8 short aliases to their canonical names: `c`\u2192`color`, `lw`\u2192`linewidth`, `ls`\u2192`linestyle`, `fc`\u2192`facecolor`, `ec`\u2192`edgecolor`, `mfc`\u2192`markerfacecolor`, `mec`\u2192`markeredgecolor`, `mew`\u2192`markeredgewidth`, `ms`\u2192`markersize`. The function normalizes aliases to canonical names before building the cycler.\n\n**Combining multi-property cyclers**: When multiple keyword arguments are given, the function creates individual `ccycler(k, v)` objects for each validated property and combines them using `functools.reduce` with `operator.add`.\n\n**Form 1 (copy constructor)**: Delegates to `validate_cycler()` which performs additional checks: it verifies that no unknown artist properties are used, prevents both a canonical name and its alias from appearing in the same cycler, detects alias collisions, uses `cycler_inst.change_key()` to normalize alias keys, and validates all values through the appropriate `_prop_validators`.\n\n**Color-specific validation**: The `color` property validator uses `_listify_validator(validate_color_for_prop_cycle, allow_stringlist=True)`. The `validate_color_for_prop_cycle` function explicitly rejects cycle reference strings matching `^C[0-9]$` (like 'C0', 'C1') to prevent self-referential cycles. The `allow_stringlist=True` flag enables the special `_single_string_color_list` function which can interpret a string as either a named color sequence (via `mpl.color_sequences`) or as a sequence of single-letter color names.\n\n**String parsing**: `validate_cycler` accepts string representations which are parsed safely via `_parse_cycler_string` using `ast.parse` (not `eval`). The AST-based parser (`_eval_cycler_expr`) supports `+` and `*` binary operators, only allows calls to `cycler()` and `concat()`, and supports slicing syntax on cycler objects.\n\n**Default prop_cycle**: The default value of `axes.prop_cycle` is `cycler('color', [...])` with 10 RGB tuple colors (the default matplotlib color cycle), defined at line 2149.\n\n**Re-export chain**: The validating `cycler` is re-exported from `matplotlib.__init__` (line 164: `from matplotlib.rcsetup import cycler`), while `matplotlib.pyplot` imports the *original* `cycler` from the `cycler` package directly (`from cycler import cycler`), not the validating one.\n\n**Downstream usage**: The `_PropCycle` class in `lib/matplotlib/axes/_base.py` consumes the validated cycler. It expands the Cycler iterator into an explicit indexed list (`self._cycler_items = [*cycler]`), and defaults to returning `'k'` (black) when no color key is present in a cycle entry.",
    "rubric": [
      "Explains that the external cycler package's `cycler` function is imported as `ccycler`, `Cycler` class as `Cycler`, and `concat` as `cconcat` to avoid name collision",
      "Describes the `_prop_validators` dictionary mapping property names (color, linewidth, linestyle, etc.) to their validators",
      "Describes the `_prop_aliases` dictionary mapping short aliases (c, lw, ls, fc, ec, mfc, mec, mew, ms) to canonical property names",
      "Explains that multiple kwargs are combined using functools.reduce with operator.add over individual ccycler(k, v) calls",
      "Explains that when a Cycler instance is passed (Form 1/copy constructor), it delegates to validate_cycler which normalizes aliases via change_key() and validates values",
      "Mentions that validate_color_for_prop_cycle rejects CN color cycle references (matching ^C[0-9]$) to prevent self-referential cycles",
      "Explains that string representations are parsed safely via AST (ast.parse) rather than eval, supporting +, * operators and cycler()/concat() calls",
      "Notes that matplotlib re-exports the validating cycler from matplotlib.__init__, while matplotlib.pyplot imports the original cycler from the cycler package directly",
      "Mentions the _PropCycle class in axes/_base.py that consumes the validated cycler, expanding it into an indexed list and defaulting to 'k' when no color key is present"
    ],
    "key_files": [
      "lib/matplotlib/rcsetup.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/__init__.py",
      "lib/matplotlib/_api/__init__.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/rcsetup.py] lib.matplotlib.rcsetup.cycler\nCreate a `~cycler.Cycler` object much like :func:`cycler.cycler`,\n    but includes input validation.\n\n    Call signatures::\n\n      cycler(cycler)\n      cycler(label=values, label2=values2, ...)\n      cycler(label, values)\n\n    Form 1 copies a given `~cycler.Cycler` object.\n\n    Form 2 creates a `~cycler.Cycler` which cycles over one or more\n    properties simultaneously. If multiple properties are given, their\n    value lists must have the same length.\n\n    Form 3 creates a `~cycler.Cycler` for a single property. This form\n    exists for compatibility with the original cycler. Its use is\n    discouraged in favor of the kwarg form, i.e. ``cycler(label=values)``.\n\n    Parameters\n    ----------\n    cycler : Cycler\n        Copy constructor for Cycler.\n\n    label : str\n        The property key. Must be a valid `.Artist` property.\n        For example, 'color' or 'linestyle'. Aliases are allowed,\n        such as 'c' for 'color' and 'lw' for 'linewidth'.\n\n    values : iterable\n        Finite-length iterable of the property values. These values\n        are validated and will raise a ValueError if invalid.\n\n    Returns\n    -------\n    Cycler\n        A new :class:`~cycler.Cycler` for the given properties.\n\n    Examples\n    --------\n    Creating a cycler for a single property:\n\n    >>> c = cycler(color=['red', 'green', 'blue'])\n\n    Creating a cycler for simultaneously cycling over multiple properties\n    (e.g. red circle, green plus, blue cross):\n\n    >>> c = cycler(color=['red', 'green', 'blue'],\n    ...            marker=['o', '+', 'x'])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation lists aliases as 'c' for 'color' and 'lw' for 'linewidth' as examples, and the answer expands this to 8 specific aliases with specific mappings - these are plausible but not verified by the provided documentation alone.",
      "The answer claims 14 recognized property names in _prop_validators but also lists 17 properties (color, linewidth, linestyle, facecolor, edgecolor, joinstyle, capstyle, fillstyle, markerfacecolor, markersize, markeredgewidth, markeredgecolor, markevery, alpha, marker, hatch, dashes) - this is inconsistent (17 vs 14).",
      "Many implementation details (line numbers, specific internal functions, AST parsing, re-export chain, _PropCycle class) go well beyond the documentation and cannot be verified from it - they are plausible code-level details but could contain errors.",
      "The claim about pyplot importing the original cycler from the cycler package directly rather than the validating one is not supported by the documentation and could be incorrect.",
      "The claim that the default color cycle contains '10 RGB tuple colors' is not verifiable from the documentation provided."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale phrases ('reduce alias handling code elsewhere', 'to support conditional advancing'); core claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_34",
    "repo": "matplotlib",
    "question": "Why does `scale_factory` use a `_scale_has_axis_parameter` dictionary instead of checking the scale class's signature each time?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `_scale_has_axis_parameter` dictionary in scale.py is used to avoid calling `inspect.signature()` on every invocation of `scale_factory()`. The signature check is done only once: either hardcoded for built-in scales or computed at registration time via `register_scale()` (line 945: `has_axis_parameter = \"axis\" in inspect.signature(scale_class).parameters`).\n\nThe `_make_axis_parameter_optional` decorator wrapper uses an `isinstance(args[0], mpl.axis.Axis)` runtime check to determine whether the axis was passed. The wrapper injects `None` as the first positional argument when no axis is provided (line 178: `return init_func(self, axis, *args, **kwargs)` where axis=None from `kwargs.pop('axis', None)`).\n\nFor third-party scales that don't have an axis parameter at all, `scale_factory` bypasses even the isinstance check entirely by calling `scale_cls(**kwargs)` without passing axis (line 924-925), avoiding wrapper overhead altogether.",
    "rubric": [
      "Explains that the dictionary caches signature information to avoid calling inspect.signature() on every scale_factory invocation (performance optimization)",
      "Mentions that built-in scales have their values hardcoded in the dictionary",
      "Mentions that third-party scales get their value computed once at registration time via register_scale()",
      "Explains how the _make_axis_parameter_optional wrapper uses isinstance(args[0], Axis) to detect whether axis was passed",
      "Notes that for scales without an axis parameter, scale_factory bypasses the axis handling entirely by calling scale_cls(**kwargs) without passing axis"
    ],
    "key_files": [
      "lib/matplotlib/scale.py",
      "lib/matplotlib/artist.py",
      "lib/matplotlib/axis.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/scale.py] lib.matplotlib.scale._make_axis_parameter_optional\nDecorator to allow leaving out the *axis* parameter in scale constructors.\n\n    This decorator ensures backward compatibility for scale classes that\n    previously required an *axis* parameter. It allows constructors to be\n    callerd with or without the *axis* parameter.\n\n    For simplicity, this does not handle the case when *axis*\n    is passed as a keyword. However,\n    scanning GitHub, there's no evidence that that is used anywhere.\n\n    Parameters\n    ----------\n    init_func : callable\n        The original __init__ method of a scale class.\n\n    Returns\n    -------\n    callable\n        A wrapped version of *init_func* that handles the optional *axis*.\n\n    Notes\n    -----\n    If the wrapped constructor defines *axis* as its first argument, the\n    parameter is preserved when present. Otherwise, the value `None` is injected\n    as the first argument.\n\n    Examples\n    --------\n    >>> from matplotlib.scale import ScaleBase\n    >>> class CustomScale(ScaleBase):\n    ...     @_make_axis_parameter_optional\n    ...     def __init__(self, axis, custom_param=1):\n    ...         self.custom_param = custom_param",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (891-903, 945, 178, 924-925, 1580) and code structures (_scale_has_axis_parameter dictionary, register_scale(), scale_factory(), artist.py) that are not present in the provided documentation chunk. These could be accurate details from the actual source code, but they cannot be verified from the given documentation alone.",
      "The answer discusses caching mechanisms, performance optimizations, and a _scale_has_axis_parameter dictionary that are entirely outside the scope of the provided documentation, which only describes the _make_axis_parameter_optional decorator.",
      "The claim about the wrapper injecting None as the first positional argument is supported by the documentation's Notes section: 'the value `None` is injected as the first argument'.",
      "The answer's description of isinstance(args[0], mpl.axis.Axis) runtime check is plausible but not documented in the provided chunk.",
      "The answer appears to be responding to a different question (about performance optimization and caching) than what the documentation describes (the decorator's purpose and behavior)."
    ],
    "strip_verify_leakage": 0.33,
    "strip_verify_summary": "Several claims about performance rationale and comments are doc-derived; structural/code-flow claims are verifiable from stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_35",
    "repo": "matplotlib",
    "question": "Where is the `PlotDirective` class defined, and how does its execution flow connect to figure rendering, code execution, and Sphinx registration?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `PlotDirective` class and its associated `run()` function are located in `lib/matplotlib/sphinxext/plot_directive.py`. Here is a detailed breakdown of where key components are defined and how they connect:\n\n**Primary class location:** `PlotDirective` inherits from `docutils.parsers.rst.directives.Directive`. It has `has_content = True`, `required_arguments = 0`, `optional_arguments = 2`, and `final_argument_whitespace = False`. Its `run()` method delegates to a module-level `run()` function that does the actual work.\n\n**Registration:** The `setup()` function registers the directive with Sphinx via `app.add_directive('plot', PlotDirective)`. It also connects the `mark_plot_labels` function to the `'doctree-read'` event, `_copy_css_file` to `'build-finished'`, and `init_filename_registry` to `'builder-inited'`. It registers a `_FilenameCollector` (an `EnvironmentCollector` subclass from Sphinx) via `app.add_env_collector`. The `setup()` function returns metadata with `parallel_read_safe: True` and `parallel_write_safe: True`, and version set to `matplotlib.__version__`.\n\n**Module-level state:** The module sets `matplotlib.use('agg')` at import time. The module-level `__version__` is set to integer `2`. A module-level `plot_context = dict()` is defined which is used in conjunction with the `:context:` option.\n\n**Code execution:** `_run_code()` executes plot code. It patches `FigureManagerBase.show` to a no-op lambda (`lambda self: None`) using `cbook._setattr_cm` context manager, and redirects stdout to `StringIO()`. When `plot_pre_code` is `None`, it defaults to executing `'import numpy as np\\nfrom matplotlib import pyplot as plt\\n'`. If `\"__main__\"` appears in the code, it sets `ns['__name__'] = '__main__'`.\n\n**Figure rendering:** `render_figures()` is defined to handle figure generation after code execution. `Gcf` is a class in `lib/matplotlib/_pylab_helpers.py` that maintains an `OrderedDict` (`figs = OrderedDict()`) mapping figure numbers to managers, using classmethods for all operations.\n\n**DPI defaults:** `get_plot_formats()` uses hardcoded default DPIs: `{'png': 80, 'hires.png': 200, 'pdf': 200}`. For unknown format strings, the fallback DPI is 80.\n\n**CSS file:** `_copy_css_file()` copies `plot_directive.css` from matplotlib's data path (`cbook._get_data_path('plot_directive/plot_directive.css')`) to the output `_static` directory on build completion.\n\n**Related module:** The `figure-mpl` directive used by `TEMPLATE_SRCSET` is implemented in `lib/matplotlib/sphinxext/figmpl_directive.py` by the `FigureMpl` class, which inherits from `docutils.parsers.rst.directives.images.Figure`. It creates `figmplnode` nodes (a custom node inheriting from `nodes.General` and `nodes.Element`). Its `setup()` function registers the `figure-mpl` directive and adds the node with separate HTML/LaTeX visitor functions (`visit_figmpl_html`, `visit_figmpl_latex`).\n\n**Duplicate filename detection:** `check_output_base_name()` validates that `output_base` contains no dots, slashes, or backslashes, and checks against `env.mpl_plot_image_basenames` (a `defaultdict(set)`) to ensure uniqueness across all documents.",
    "rubric": [
      "PlotDirective is located in lib/matplotlib/sphinxext/plot_directive.py and inherits from docutils.parsers.rst.directives.Directive",
      "PlotDirective's run() method delegates to a module-level run() function",
      "setup() registers the directive via app.add_directive('plot', PlotDirective) and connects event handlers (mark_plot_labels to doctree-read, _copy_css_file to build-finished, init_filename_registry to builder-inited)",
      "The module sets matplotlib.use('agg') at import time and defines a module-level plot_context dict for the :context: option",
      "_run_code() patches FigureManagerBase.show to a no-op lambda using cbook._setattr_cm and redirects stdout to StringIO",
      "When plot_pre_code is None, _run_code() defaults to executing 'import numpy as np\\nfrom matplotlib import pyplot as plt\\n'",
      "get_plot_formats() uses hardcoded default DPIs: png=80, hires.png=200, pdf=200, with fallback DPI of 80 for unknown formats",
      "_copy_css_file() copies plot_directive.css from matplotlib's data path to the output _static directory",
      "The figure-mpl directive used in TEMPLATE_SRCSET is implemented in figmpl_directive.py by FigureMpl class inheriting from docutils's Figure directive",
      "setup() returns metadata with parallel_read_safe and parallel_write_safe both True, and version set to matplotlib.__version__"
    ],
    "key_files": [
      "lib/matplotlib/sphinxext/plot_directive.py",
      "lib/matplotlib/sphinxext/figmpl_directive.py",
      "lib/matplotlib/_pylab_helpers.py",
      "lib/matplotlib/cbook.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/sphinxext/plot_directive.py] lib.matplotlib.sphinxext.plot_directive\nA directive for including a Matplotlib plot in a Sphinx document\n================================================================\n\nThis is a Sphinx extension providing a reStructuredText directive\n``.. plot::`` for including a plot in a Sphinx document.\n\nIn HTML output, ``.. plot::`` will include a .png file with a link\nto a high-res .png and .pdf.  In LaTeX output, it will include a .pdf.\n\nThe plot content may be defined in one of three ways:\n\n1. **A path to a source file** as the argument to the directive::\n\n     .. plot:: path/to/plot.py\n\n   When a path to a source file is given, the content of the\n   directive may optionally contain a caption for the plot::\n\n     .. plot:: path/to/plot.py\n\n        The plot caption.\n\n   Additionally, one may specify the name of a function to call (with\n   no arguments) immediately after importing the module::\n\n     .. plot:: path/to/plot.py plot_function1\n\n2. Included as **inline content** to the directive::\n\n     .. plot::\n\n        import matplotlib.pyplot as plt\n        plt.plot([1, 2, 3], [4, 5, 6])\n        plt.title(\"A plotting example\")\n\n3. Using **doctest** syntax::\n\n     .. plot::\n\n        A plotting example:\n        >>> import matplotlib.pyplot as plt\n        >>> plt.plot([1, 2, 3], [4, 5, 6])\n\nOptions\n-------\n\nThe ``.. plot::`` directive supports the following options:\n\n``:filename-prefix:`` : str\n    The base name (without the extension) of the outputted image and script\n    files. The default is to use the same name as the input script, or the\n    name of the RST document if no script is provided. The filename-prefix for\n    each plot directive must be unique.\n\n``:format:`` : {'python', 'doctest'}\n    The format of the input.  If unset, the format is auto-detected.\n\n``:include-source:`` : bool\n    Whether to display the source code. The default can be changed using\n    the ``plot_include_source`` variable in :file:`conf.py` (which itself\n    defaults to False).\n\n``:show-source-link:`` : bool\n    Whether to show a link to the source in HTML. The default can be\n    changed using the ``plot_html_show_source_link`` variable in\n    :file:`conf.py` (which itself defaults to True).\n\n``:context:`` : bool or str\n    If provided, the code will be run in the context of all previous plot\n    directives for which the ``:context:`` option was specified.  This only\n    applies to inline code plot directives, not those run from files. If\n    the ``:context: reset`` option is specified, the context is reset\n    for this and future plots, and previous figures are closed prior to\n    running the code. ``:context: close-figs`` keeps the context but closes\n    previous figures before running the code.\n\n``:nofigs:`` : bool\n    If specified, the code block will be run, but no figures will be\n    inserted.  This is usually useful with the ``:context:`` option.\n\n``:caption:`` : str\n    If specified, the option's argument will be used as a caption for the\n    figure. This overwrites the caption given in the content, when the plot\n    is generated from a file.\n\n``:code-caption:`` : str\n    If specified, the option's argument will be used as a caption for the\n    code block (when ``:include-source:`` is used). This is added as the\n    ``:caption:`` option to the ``.. code-block::`` directive.\n\nAdditionally, this directive supports all the options of the `image directive\n<https://docutils.sourceforge.io/docs/ref/rst/directives.html#image>`_,\nexcept for ``:target:`` (since plot will add its own target).  These include\n``:alt:``, ``:height:``, ``:width:``, ``:scale:``, ``:align:`` and ``:class:``.\n\nConfiguration options\n---------------------\n\nThe plot directive has the following configuration options:\n\nplot_include_source\n    Default value for the include-source option (default: False).\n\nplot_html_show_source_link\n    Whether to show a link to the source in HTML (default: True).\n\nplot_pre_code\n    Code that should be executed before each plot. If None (the default),\n    it will default to a string containing::\n\n        import numpy as np\n        from matplotlib import pyplot as plt\n\nplot_basedir\n    Base directory, to which ``plot::`` file names are relative to.\n    If None or empty (the default), file names are relative to the\n    directory where the file containing the directive is.\n\nplot_formats\n    File formats to generate (default: ['png', 'hires.png', 'pdf']).\n    List of tuples or strings::\n\n        [(suffix, dpi), suffix, ...]\n\n    that determine the file format and the DPI. For entries whose\n    DPI was omitted, sensible defaults are chosen. When passing from\n    the command line through sphinx_build the list should be passed as\n    suffix:dpi,suffix:dpi, ...\n\nplot_html_show_formats\n    Whether to show links to the files in HTML (default: True).\n\nplot_rcparams\n    A dictionary containing any non-standard rcParams that should\n    be applied before each plot (default: {}).\n\nplot_apply_rcparams\n    By default, rcParams are applied when ``:context:`` option is not used\n    in a plot directive.  If set, this configuration option overrides this\n    behavior and applies rcParams before each plot.\n\nplot_working_directory\n    By default, the working directory will be changed to the directory of\n    the example, so the code can get at its data files, if any.  Also its\n    path will be added to `sys.path` so it can import any helper modules\n    sitting beside it.  This configuration option can be used to specify\n    a central directory (also added to `sys.path`) where data files and\n    helper modules for all code are located.\n\nplot_template\n    Provide a customized template for preparing restructured text.\n\nplot_srcset\n    Allow the srcset image option for responsive image resolutions. List of\n    strings with the multiplicative factors followed by an \"x\".\n    e.g. [\"2.0x\", \"1.5x\"].  \"2.0x\" will create a png with the default \"png\"\n    resolution from plot_formats, multiplied by 2. If plot_srcset is\n    specified, the plot directive uses the\n    :doc:`/api/sphinxext_figmpl_directive_api` (instead of the usual figure\n    directive) in the intermediary rst file that is generated.\n    The plot_srcset option is incompatible with *singlehtml* builds, and an\n    error will be raised.\n\nNotes on how it works\n---------------------\n\nThe plot directive runs the code it is given, either in the source file or the\ncode under the directive. The figure created (if any) is saved in the sphinx\nbuild directory under a subdirectory named ``plot_directive``.  It then creates\nan intermediate rst file that calls a ``.. figure:`` directive (or\n``.. figmpl::`` directive if ``plot_srcset`` is being used) and has links to\nthe ``*.png`` files in the ``plot_directive`` directory.  These translations can\nbe customized by changing the *plot_template*.  See the source of\n:doc:`/api/sphinxext_plot_directive_api` for the templates defined in *TEMPLATE*\nand *TEMPLATE_SRCSET*.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific code-level details (line numbers, class names, function signatures, internal implementation details) that cannot be verified from the provided documentation alone. These are plausible extrapolations from source code but are not supported by the given doc chunk.",
      "The claim about default DPIs {'png': 80, 'hires.png': 200, 'pdf': 200} cannot be verified from the documentation, which only mentions 'sensible defaults are chosen' for omitted DPI values.",
      "The claim about plot_pre_code defaulting to 'import numpy as np\\nfrom matplotlib import pyplot as plt\\n' is consistent with the documentation which states the same.",
      "The claim about plot_formats default ['png', 'hires.png', 'pdf'] is consistent with the documentation.",
      "The claim about plot_include_source defaulting to False is consistent with the documentation.",
      "The claim about plot_html_show_source_link defaulting to True is consistent with the documentation.",
      "The claim about TEMPLATE_SRCSET and figmpl directive usage is consistent with the documentation's description of plot_srcset behavior.",
      "The claim that dots in output_base are replaced with hyphens to prevent LaTeX issues is a plausible implementation detail but not verifiable from the doc.",
      "The claim about parallel_read_safe and parallel_write_safe being True cannot be verified from the doc.",
      "The claim about __version__ = 2 at module level cannot be verified from the doc."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few claims about truncated code sections or documentation-based rationale needed minor rewrites.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_36",
    "repo": "matplotlib",
    "question": "How does Matplotlib's Table class implement cell layout, font sizing, edge rendering, and positioning within axes?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Table` class in `lib/matplotlib/table.py` implements table rendering in Matplotlib through the following key mechanisms:\n\n**Class Hierarchy:** `Table` extends `Artist` (from `matplotlib.artist`), while `Cell` extends `Rectangle` (from `matplotlib.patches`), which itself extends `Patch` \u2192 `Artist`. The module-level alias `CustomCell = Cell` exists.\n\n**Cell Positioning Algorithm (`_do_cell_alignment`):** Rows are sorted in *reverse* order (highest row index first) to compute bottom positions, which means row 0 is at the top. Columns are sorted in ascending order for left positions. This is how the grid layout places (0,0) at the top-left. The `ypos` starts at 0 and accumulates upward as rows are iterated in reverse.\n\n**Drawing Order:** In `Table.draw()`, cells are drawn in sorted key order (`for key in sorted(self._cells)`), meaning they are drawn row-by-row, column-by-column based on their (row, col) tuple keys.\n\n**Auto Font Size Algorithm (`Cell.auto_set_font_size`):** The font size starts at the cell's current fontsize and decrements by 1 point at a time in a while loop until either the required width fits within the cell width or the fontsize reaches 1. In `Table._auto_set_font_size`, cells in auto-width columns (`_autoColumns`) are skipped, and the minimum fontsize across all remaining cells is applied to ALL cells in the table.\n\n**Visible Edges Path Generation (`Cell.get_path`):** Overrides `Rectangle.get_path()`. It constructs a `Path` with vertices `[[0,0], [1,0], [1,1], [0,1], [0,0]]` using `Path.LINETO` for visible edges and `Path.MOVETO` for hidden edges. The edge order in `_edges` is 'BRTL' (corresponding to Bottom, Right, Top, Left based on the vertex ordering). If all edges are visible (no MOVETO in codes[1:]), the last code becomes `Path.CLOSEPOLY`.\n\n**Location Positioning (`_update_positions`):** The `'best'` location (code 0) is treated identically to 'upper right' \u2014 it positions the table at both the right edge and upper edge with `AXESPAD` (0.02) padding. When a `bbox` is provided, the table is scaled to fit that bbox by calling `self.scale(rw/w, rh/h)` followed by a second `_do_cell_alignment()` call.\n\n**Approximate Text Height:** Calculated as `(FONTSIZE / 72.0 * figure.dpi / axes.bbox.height * 1.2)` where FONTSIZE is the class constant 10.\n\n**Pandas DataFrame Support:** The `table()` factory function checks for DataFrames using `_is_pandas_dataframe()` from `cbook`. If cellText is a DataFrame, it extracts `index` for rowLabels, `columns` for colLabels, and `.values` for cellText content. It raises ValueError if rowLabels or colLabels are explicitly provided alongside a DataFrame.\n\n**Integration with Axes:** The `table()` factory function is imported as `mtable` in `_base.py`. Row label cells are added at column index -1.",
    "rubric": [
      "Table extends Artist and Cell extends Rectangle (with CustomCell = Cell alias)",
      "_do_cell_alignment sorts rows in reverse order (highest index first) so row 0 is at the top, and columns in ascending order",
      "Cells are drawn in sorted key order (row, col) tuples in Table.draw()",
      "Cell.auto_set_font_size decrements fontsize by 1 in a while loop until text fits or fontsize reaches 1",
      "Table._auto_set_font_size skips cells in _autoColumns and applies the minimum fontsize found to ALL cells",
      "Cell.get_path constructs a Path with vertices [[0,0],[1,0],[1,1],[0,1],[0,0]] using LINETO for visible edges and MOVETO for hidden edges, with edge order 'BRTL'",
      "The 'best' location (code 0) is treated identically to 'upper right' with AXESPAD (0.02) padding",
      "When a bbox is provided, the table is scaled via self.scale(rw/w, rh/h) followed by a second _do_cell_alignment() call",
      "The table() factory function detects pandas DataFrames via _is_pandas_dataframe() and extracts index, columns, and values",
      "Row label cells are added at column index -1"
    ],
    "key_files": [
      "lib/matplotlib/table.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/patches.py",
      "lib/matplotlib/cbook.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/table.py] lib.matplotlib.table.Table\nlib.matplotlib.table.Table:\n  A table of cells.\n\n    .. note::\n\n        ``table()`` has some fundamental design limitations and will not be\n        developed further. If you need more functionality, consider\n        `blume <https://github.com/swfiua/blume>`__.\n\n\n    The table consists of a grid of cells, which are indexed by (row, column).\n\n    For a simple table, you'll have a full grid of cells with indices from\n    (0, 0) to (num_rows-1, num_cols-1), in which the cell (0, 0) is positioned\n    at the top left. However, you can also add cells with negative indices.\n    You don't have to add a cell to every grid position, so you can create\n    tables that have holes.\n\n    *Note*: You'll usually not create an empty table from scratch. Instead use\n    `~matplotlib.table.table` to create a table from data.\n\nlib.matplotlib.table.Table.__init__:\n  Parameters\n        ----------\n        ax : `~matplotlib.axes.Axes`\n            The `~.axes.Axes` to plot the table into.\n        loc : str, optional\n            The position of the cell with respect to *ax*. This must be one of\n            the `~.Table.codes`.\n        bbox : `.Bbox` or [xmin, ymin, width, height], optional\n            A bounding box to draw the table into. If this is not *None*, this\n            overrides *loc*.\n\n        Other Parameters\n        ----------------\n        **kwargs\n            `.Artist` properties.\n\nlib.matplotlib.table.Table.add_cell:\n  Create a cell and add it to the table.\n\n        Parameters\n        ----------\n        row : int\n            Row index.\n        col : int\n            Column index.\n        *args, **kwargs\n            All other parameters are passed on to `Cell`.\n\n        Returns\n        -------\n        `.Cell`\n            The created cell.\n\nlib.matplotlib.table.Table.__setitem__:\n  Set a custom cell in a given position.\n\nlib.matplotlib.table.Table.__getitem__:\n  Retrieve a custom cell from a given position.\n\nlib.matplotlib.table.Table.edges:\n  The default value of `~.Cell.visible_edges` for newly added\n        cells using `.add_cell`.\n\n        Notes\n        -----\n        This setting does currently only affect newly created cells using\n        `.add_cell`.\n\n        To change existing cells, you have to set their edges explicitly::\n\n            for c in tab.get_celld().values():\n                c.visible_edges = 'horizontal'\n\nlib.matplotlib.table.Table._get_grid_bbox:\n  Get a bbox, in axes coordinates for the cells.\n\n        Only include those in the range (0, 0) to (maxRow, maxCol).\n\nlib.matplotlib.table.Table.get_children:\n  Return the Artists contained by the table.\n\nlib.matplotlib.table.Table._do_cell_alignment:\n  Calculate row heights and column widths; position cells accordingly.\n\nlib.matplotlib.table.Table.auto_set_column_width:\n  Automatically set the widths of given columns to optimal sizes.\n\n        Parameters\n        ----------\n        col : int or sequence of ints\n            The indices of the columns to auto-scale.\n\nlib.matplotlib.table.Table._auto_set_column_width:\n  Automatically set width for column.\n\nlib.matplotlib.table.Table.auto_set_font_size:\n  Automatically set font size.\n\nlib.matplotlib.table.Table.scale:\n  Scale column widths by *xscale* and row heights by *yscale*.\n\nlib.matplotlib.table.Table.set_fontsize:\n  Set the font size, in points, of the cell text.\n\n        Parameters\n        ----------\n        size : float\n\n        Notes\n        -----\n        As long as auto font size has not been disabled, the value will be\n        clipped such that the text fits horizontally into the cell.\n\n        You can disable this behavior using `.auto_set_font_size`.\n\n        >>> the_table.auto_set_font_size(False)\n        >>> the_table.set_fontsize(20)\n\n        However, there is no automatic scaling of the row height so that the\n        text may exceed the cell boundary.\n\nlib.matplotlib.table.Table._offset:\n  Move all the artists by ox, oy (axes coords).\n\nlib.matplotlib.table.Table.get_celld:\n  Return a dict of cells in the table mapping *(row, column)* to\n        `.Cell`\\s.\n\n        Notes\n        -----\n        You can also directly index into the Table object to access individual\n        cells::\n\n            cell = table[row, col]",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not describe the class hierarchy details (Cell extends Rectangle, CustomCell = Cell alias), though these are plausible code-level details.",
      "The documentation does not describe the _do_cell_alignment algorithm internals (reverse row sorting, ypos accumulation), though the docstring mentions 'Calculate row heights and column widths; position cells accordingly' \u2014 the specific details are extrapolated from source code.",
      "The documentation does not describe the drawing order details (sorted key order), auto font size algorithm (decrement by 1 point), visible edges path generation details, or the 'best' location behavior \u2014 these are all code-level implementation details beyond the provided documentation.",
      "The documentation does not describe pandas DataFrame support, the table() factory function's DataFrame handling, or the integration mechanism with Axes (add_table, _children, etc.).",
      "The documentation does not describe the approximate text height formula or the row labels column index (-1) or the 1e-15 width detail.",
      "The claim about 'best' being treated identically to 'upper right' is not in the documentation and could be incorrect \u2014 this is an unsupported implementation claim.",
      "The FONTSIZE constant of 10, AXESPAD of 0.02, and _edges = 'BRTL' are code-level details not present in the documentation.",
      "The claim that the edges property 'does currently only affect newly created cells using .add_cell' is supported by the documentation.",
      "The claim that cell (0,0) is positioned at the top left is directly supported by the documentation.",
      "The claim about Table consisting of a grid of cells indexed by (row, column) is supported.",
      "The set_fontsize behavior description about auto font size clipping is supported by the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from the stripped code; a few claims about truncated code sections and one about internal implementation of _is_pandas_dataframe needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_40",
    "repo": "matplotlib",
    "question": "How does `LinearTriInterpolator` compute interpolated values and gradients from initialization through to the final result?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `LinearTriInterpolator` performs interpolation over a triangular grid using plane coefficients computed for each triangle. Here's how it works in detail:\n\n**Initialization (`__init__`):**\n1. It calls the parent `TriInterpolator.__init__`, which validates that `triangulation` is a `Triangulation` instance, `z` has the same shape as `triangulation.x`, and `trifinder` is either a `TriFinder` or `None`. If no trifinder is provided, it defaults to `self._triangulation.get_trifinder()`, which lazily creates a `TrapezoidMapTriFinder` (implemented via a C++ trapezoid map algorithm).\n2. The parent class sets `_unit_x = 1.0`, `_unit_y = 1.0` (no scaling) and `_tri_renum = None` (no triangle renumbering). LinearTriInterpolator does not change these, unlike CubicTriInterpolator which does.\n3. It precomputes plane coefficients by calling `self._triangulation.calculate_plane_coefficients(self._z)`, which delegates to the underlying C++ triangulation object (`_tri.Triangulation.calculate_plane_coefficients`). The result is stored as `self._plane_coefficients` with shape `(ntri, 3)`, where `z = coeffs[tri, 0]*x + coeffs[tri, 1]*y + coeffs[tri, 2]`.\n\n**Interpolation (`__call__`):**\n1. `__call__(x, y)` delegates to `_interpolate_multikeys(x, y, tri_index=None, return_keys=('z',))[0]`.\n2. `_interpolate_multikeys` (defined in the parent `TriInterpolator` class) handles: flattening multi-dimensional input arrays, scaling by `_unit_x`/`_unit_y`, finding containing triangles via `self._trifinder(x, y)` (which returns -1 for points outside the triangulation), filtering valid triangle indices using `mask_in = (tri_index != -1)`, calling `_interpolate_single_key` only for valid points, filling invalid locations with `np.nan`, and wrapping results in `np.ma.masked_invalid`.\n3. `_interpolate_single_key` for `return_key='z'` computes: `coeffs[tri, 0]*x + coeffs[tri, 1]*y + coeffs[tri, 2]` \u2014 a simple dot product with the precomputed plane coefficients.\n\n**Gradient (`gradient`):**\n1. Calls `_interpolate_multikeys` with `return_keys=('dzdx', 'dzdy')`, returning a list of 2 masked arrays.\n2. For `'dzdx'`, returns `self._plane_coefficients[tri_index, 0]` (the x-coefficient of the plane).\n3. For `'dzdy'`, returns `self._plane_coefficients[tri_index, 1]` (the y-coefficient of the plane).\n4. Since the returned values are constant per triangle (not dependent on x, y position within the triangle), derivatives are constant within each triangle.\n\n**Triangle finding:**\nThe default `TrapezoidMapTriFinder` uses a C++ implementation (`_tri.TrapezoidMapTriFinder`) initialized with the C++ triangulation. Points outside the triangulation return index -1.\n\n**Key design difference from CubicTriInterpolator:**\nLinearTriInterpolator does NOT use triangle renumbering (`_tri_renum` stays `None`) or scaling (`_unit_x`, `_unit_y` stay `1.0`). CubicTriInterpolator uses `TriAnalyzer._get_compressed_triangulation()` to renumber triangles and compute scaling factors based on `np.ptp` of coordinates.\n\n**Validation:**\nThe `_interpolate_single_key` method uses `_api.check_in_list(['z', 'dzdx', 'dzdy'], return_key=return_key)` to validate the requested key. The parent class maps return keys to indices via `{'z': 0, 'dzdx': 1, 'dzdy': 2}` and applies scaling factors `[1., 1./unit_x, 1./unit_y]` \u2014 but since `unit_x` and `unit_y` are both 1.0 for LinearTriInterpolator, no actual scaling occurs.",
    "rubric": [
      "Explains that __init__ calls the parent TriInterpolator.__init__ which validates the triangulation, z array shape, and trifinder, defaulting to the triangulation's get_trifinder()",
      "Mentions that the parent sets _unit_x=1.0, _unit_y=1.0 (no scaling) and _tri_renum=None (no renumbering)",
      "Explains that plane coefficients are precomputed via calculate_plane_coefficients(z) with shape (ntri, 3) where z = coeffs[tri,0]*x + coeffs[tri,1]*y + coeffs[tri,2]",
      "Describes that __call__ delegates to _interpolate_multikeys which flattens input, finds containing triangles via self._trifinder, filters out invalid points (tri_index == -1), and fills those with NaN/masked values",
      "Explains that _interpolate_single_key for 'z' computes coeffs[tri,0]*x + coeffs[tri,1]*y + coeffs[tri,2]",
      "Explains that gradient returns dzdx and dzdy which are simply the per-triangle plane coefficients (constant within each triangle): coeffs[tri_index, 0] and coeffs[tri_index, 1]"
    ],
    "key_files": [
      "lib/matplotlib/tri/_triinterpolate.py",
      "lib/matplotlib/tri/_triangulation.py",
      "lib/matplotlib/tri/_trifinder.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_triinterpolate.py] lib.matplotlib.tri._triinterpolate.LinearTriInterpolator\nlib.matplotlib.tri._triinterpolate.LinearTriInterpolator:\n  Linear interpolator on a triangular grid.\n\n    Each triangle is represented by a plane so that an interpolated value at\n    point (x, y) lies on the plane of the triangle containing (x, y).\n    Interpolated values are therefore continuous across the triangulation, but\n    their first derivatives are discontinuous at edges between triangles.\n\n    Parameters\n    ----------\n    triangulation : `~matplotlib.tri.Triangulation`\n        The triangulation to interpolate over.\n    z : (npoints,) array-like\n        Array of values, defined at grid points, to interpolate between.\n    trifinder : `~matplotlib.tri.TriFinder`, optional\n        If this is not specified, the Triangulation's default TriFinder will\n        be used by calling `.Triangulation.get_trifinder`.\n\n    Methods\n    -------\n    `__call__` (x, y) : Returns interpolated values at (x, y) points.\n    `gradient` (x, y) : Returns interpolated derivatives at (x, y) points.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: reference to de Berg et al. textbook and design rationale for cubic scaling are not in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_37",
    "repo": "matplotlib",
    "question": "What is the full class hierarchy and internal architecture of EngFormatter, including how it overrides or reuses behavior from its parent classes?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `EngFormatter` class inherits from `ScalarFormatter` (which inherits from `Formatter`, which inherits from `TickHelper`), forming the hierarchy `TickHelper \u2192 Formatter \u2192 ScalarFormatter \u2192 EngFormatter`. \n\nKey architectural details not in the documentation:\n\n1. **Class-level `ENG_PREFIXES` dictionary**: Defined as a class attribute mapping integer powers of 10 (multiples of 3) to SI prefix strings, ranging from -30 (\"q\" for quecto) to +30 (\"Q\" for quetta). The prefix for -6 uses the Unicode MICRO SIGN character (`\\N{MICRO SIGN}`), and the prefix for 0 is an empty string.\n\n2. **Inheritance from ScalarFormatter**: The `__init__` method calls `super().__init__(useOffset=useOffset, useMathText=useMathText, useLocale=False, usetex=usetex)` \u2014 notably always passing `useLocale=False` to the parent class, meaning locale-based number formatting is never used in EngFormatter.\n\n3. **Overridden `set_locs` method**: Unlike `ScalarFormatter.set_locs` which calls `_compute_offset()` then `_set_order_of_magnitude()` then `_set_format()`, `EngFormatter.set_locs` calls `_compute_offset()` (inherited), then replaces the offset with `round((vmin + vmax)/2, 3)` if an offset was justified, then computes `orderOfMagnitude` using `math.floor(math.log(vmax - vmin, 1000))*3` (log base 1000 instead of log base 10), and finally calls `_set_format()` (inherited from ScalarFormatter). It never calls `_set_order_of_magnitude()` from `ScalarFormatter`.\n\n4. **`__call__` dual behavior**: When `self.locs` is empty or `self.offset == 0`, it returns `self.fix_minus(self.format_data(x))` which uses engineering notation. When offset is non-zero, it falls back to the parent's numeric formatting via `self._format_maybe_minus_and_locale(self.format, xp)` where `xp = (x - self.offset) / (10. ** self.orderOfMagnitude)`.\n\n5. **`format_data` algorithm**: Determines the power-of-10 by dividing `math.log10(value)` by 3 and flooring, then clips to the range [-30, 30] using `np.clip`. It has a special rounding correction: if formatting the mantissa produces a value >= 1000 (e.g., 999.9 rounding up), it divides the mantissa by 1000 and increments pow10 by 3, but only if pow10 < max(ENG_PREFIXES) (30). When value is zero, it forces `value = 0.0` to avoid negative zero formatting issues.\n\n6. **Suffix logic**: A suffix (separator + prefix + unit) is only appended if either `self.unit` is non-empty or `unit_prefix` is non-empty. This means for values in [1, 1000) with no unit, no suffix or separator is added.\n\n7. **`get_offset` method**: Simplified compared to `ScalarFormatter.get_offset` \u2014 it only returns a non-empty string if `self.offset` is truthy (non-zero). It formats both the `sciNotStr` (power of magnitude) and `offsetStr` (the offset value) using `self.format_data()` which produces engineering notation. For positive offsets, a '+' is prepended.\n\n8. **`format_eng` is an alias**: The method `format_eng(self, num)` simply calls and returns `self.format_data(num)`.\n\n9. **`format_data_short` is inherited from `ScalarFormatter`** (not overridden by EngFormatter), which uses pixel-resolution-based formatting with `cbook._g_sig_digits`.\n\n10. **`fix_minus` is a static method on `Formatter`**: It replaces '-' with Unicode MINUS SIGN (U+2212) when `rcParams['axes.unicode_minus']` is True.",
    "rubric": [
      "Identifies the inheritance chain: TickHelper \u2192 Formatter \u2192 ScalarFormatter \u2192 EngFormatter",
      "Describes the ENG_PREFIXES class-level dictionary mapping powers of 10 (multiples of 3, from -30 to +30) to SI prefix strings, noting the Unicode MICRO SIGN for -6",
      "Notes that __init__ always passes useLocale=False to ScalarFormatter's __init__",
      "Explains the overridden set_locs method: calls _compute_offset(), replaces offset with round((vmin+vmax)/2, 3) if offset was justified, computes orderOfMagnitude using log base 1000 instead of log base 10, and calls _set_format() but never _set_order_of_magnitude()",
      "Explains the dual behavior of __call__: uses format_data with fix_minus when no offset, falls back to parent's _format_maybe_minus_and_locale when offset is non-zero",
      "Describes the format_data algorithm: determines pow10 via floor(log10/3)*3, clips to [-30,30], handles rounding correction when mantissa >= 1000, and handles negative zero",
      "Explains suffix logic: suffix (separator + prefix + unit) is only appended when unit or unit_prefix is non-empty",
      "Notes that get_offset is simplified compared to ScalarFormatter and returns non-empty string only when offset is truthy, formatting via format_data in engineering notation",
      "Mentions that format_eng is simply an alias for format_data",
      "Notes that format_data_short is inherited from ScalarFormatter (not overridden)"
    ],
    "key_files": [
      "lib/matplotlib/ticker.py",
      "lib/matplotlib/tests/test_ticker.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/ticker.py] lib.matplotlib.ticker.EngFormatter\nlib.matplotlib.ticker.EngFormatter:\n  Format axis values using engineering prefixes to represent powers\n    of 1000, plus a specified unit, e.g., 10 MHz instead of 1e7.\n\nlib.matplotlib.ticker.EngFormatter.__init__:\n  Parameters\n        ----------\n        unit : str, default: \"\"\n            Unit symbol to use, suitable for use with single-letter\n            representations of powers of 1000. For example, 'Hz' or 'm'.\n\n        places : int, default: None\n            Precision with which to display the number, specified in\n            digits after the decimal point (there will be between one\n            and three digits before the decimal point). If it is None,\n            the formatting falls back to the floating point format '%g',\n            which displays up to 6 *significant* digits, i.e. the equivalent\n            value for *places* varies between 0 and 5 (inclusive).\n\n        sep : str, default: \" \"\n            Separator used between the value and the prefix/unit. For\n            example, one get '3.14 mV' if ``sep`` is \" \" (default) and\n            '3.14mV' if ``sep`` is \"\". Besides the default behavior, some\n            other useful options may be:\n\n            * ``sep=\"\"`` to append directly the prefix/unit to the value;\n            * ``sep=\"\\N{THIN SPACE}\"`` (``U+2009``);\n            * ``sep=\"\\N{NARROW NO-BREAK SPACE}\"`` (``U+202F``);\n            * ``sep=\"\\N{NO-BREAK SPACE}\"`` (``U+00A0``).\n\n        usetex : bool, default: :rc:`text.usetex`\n            To enable/disable the use of TeX's math mode for rendering the\n            numbers in the formatter.\n\n        useMathText : bool, default: :rc:`axes.formatter.use_mathtext`\n            To enable/disable the use mathtext for rendering the numbers in\n            the formatter.\n        useOffset : bool or float, default: False\n            Whether to use offset notation with :math:`10^{3*N}` based prefixes.\n            This features allows showing an offset with standard SI order of\n            magnitude prefix near the axis. Offset is computed similarly to\n            how `ScalarFormatter` computes it internally, but here you are\n            guaranteed to get an offset which will make the tick labels exceed\n            3 digits. See also `.set_useOffset`.\n\n            .. versionadded:: 3.10\n\nlib.matplotlib.ticker.EngFormatter.__call__:\n  Return the format for tick value *x* at position *pos*.\n\n        If there is no currently offset in the data, it returns the best\n        engineering formatting that fits the given argument, independently.\n\nlib.matplotlib.ticker.EngFormatter.format_eng:\n  Alias to EngFormatter.format_data\n\nlib.matplotlib.ticker.EngFormatter.format_data:\n  Format a number in engineering notation, appending a letter\n        representing the power of 1000 of the original number.\n        Some examples:\n\n        >>> format_data(0)        # for self.places = 0\n        '0'\n\n        >>> format_data(1000000)  # for self.places = 1\n        '1.0 M'\n\n        >>> format_data(-1e-6)  # for self.places = 2\n        '-1.00 \u00b5'",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that EngFormatter inherits from ScalarFormatter is not stated in the documentation. While plausible given the references to ScalarFormatter behavior and the useOffset parameter description mentioning 'Offset is computed similarly to how ScalarFormatter computes it internally', this inheritance claim is not explicitly confirmed by the provided documentation.",
      "The detailed algorithm descriptions (set_locs, format_data, __call__ dual behavior, get_offset) are not in the documentation and represent claims about implementation details that cannot be verified from the given doc chunk alone. These could be accurate code-level details or hallucinations.",
      "The claim about ENG_PREFIXES ranging from -30 to +30 with specific prefix letters (q for quecto, Q for quetta) is not in the documentation but is plausible based on modern SI prefix standards.",
      "The claim that useLocale is always False is not mentioned in the documentation but is a plausible implementation detail.",
      "The claim about format_data_short being inherited from ScalarFormatter is not verifiable from the documentation.",
      "The documentation says format_eng is 'Alias to EngFormatter.format_data', and the answer says it 'simply calls and returns self.format_data(num)' - this is consistent.",
      "The __call__ documentation says 'If there is no currently offset in the data, it returns the best engineering formatting that fits the given argument, independently' which is consistent with the answer's description of dual behavior."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from stripped source code alone \u2014 no documentation-only claims detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_39",
    "repo": "matplotlib",
    "question": "Where does data flow through `_interpolate_multikeys` in the triangulation interpolation system, and what calls it?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `_interpolate_multikeys` method in `TriInterpolator` is the central data flow hub for interpolation in matplotlib's triangulation system. Here's how data flows through it:\n\n**Call chain (who calls it):**\n1. `LinearTriInterpolator.__call__` and `.gradient` call `_interpolate_multikeys` with `return_keys=('z',)` or `return_keys=('dzdx', 'dzdy')` respectively (lines 265-271).\n2. `CubicTriInterpolator.__call__` and `.gradient` call it similarly (lines 421-427).\n3. `UniformTriRefiner.refine_field` in `_trirefine.py` (line 167) calls `interp._interpolate_multikeys(refi_tri.x, refi_tri.y, tri_index=found_index)[0]`, passing pre-computed `tri_index` from `refine_triangulation`.\n\n**Data flow within `_interpolate_multikeys`:**\n1. Input x, y are cast to `np.float64`, shape is stored, then arrays are flattened via `np.ravel`.\n2. Scaled coordinates are computed: `x_scaled = x/self._unit_x`, `y_scaled = y/self._unit_y`.\n3. If `tri_index` is None, the `self._trifinder(x, y)` is called \u2014 this dispatches to `TrapezoidMapTriFinder.__call__`, which calls the C++ `_tri.TrapezoidMapTriFinder.find_many` method.\n4. A boolean mask `mask_in = (tri_index != -1)` separates valid from invalid points.\n5. If `self._tri_renum` is not None (used by `CubicTriInterpolator`), the triangle indices are remapped: `valid_tri_index = self._tri_renum[tri_index[mask_in]]`.\n6. For each `return_key`, a scale factor is applied: `[1., 1./self._unit_x, 1./self._unit_y]` indexed by `{'z': 0, 'dzdx': 1, 'dzdy': 2}`.\n7. Results from `_interpolate_single_key` are multiplied by scale and placed into `ret_loc[mask_in]`; invalid points get `np.nan`.\n8. The result is wrapped with `np.ma.masked_invalid(ret_loc.reshape(sh_ret), copy=False)` to produce masked arrays.\n\n**Downstream callees (`_interpolate_single_key` implementations):**\n- `LinearTriInterpolator._interpolate_single_key` uses precomputed `self._plane_coefficients` (computed via `Triangulation.calculate_plane_coefficients` which delegates to C++ `_tri.Triangulation.calculate_plane_coefficients`). For 'z', it returns `coeff[tri,0]*x + coeff[tri,1]*y + coeff[tri,2]`.\n- `CubicTriInterpolator._interpolate_single_key` computes barycentric coordinates via `_get_alpha_vec`, extracts eccentricities and DOFs per triangle, then calls `_ReducedHCT_Element.get_function_values` or `get_function_derivatives`.\n\n**Triangle renumbering (`_tri_renum`):**\n- `LinearTriInterpolator` sets `_tri_renum = None` (no renumbering).\n- `CubicTriInterpolator` gets `tri_renum` from `TriAnalyzer._get_compressed_triangulation()`, which maps original triangle indices to compressed (unmasked-only) triangle indices, with -1 for masked triangles.\n\n**Scaling:**\n- `LinearTriInterpolator` uses default `_unit_x = _unit_y = 1.0` (no scaling).\n- `CubicTriInterpolator` sets `_unit_x = np.ptp(compressed_x)` and `_unit_y = np.ptp(compressed_y)` \u2014 the peak-to-peak range of the compressed coordinate arrays.",
    "rubric": [
      "Identifies that LinearTriInterpolator.__call__ and .gradient call _interpolate_multikeys with return_keys ('z',) or ('dzdx', 'dzdy')",
      "Identifies that CubicTriInterpolator.__call__ and .gradient call _interpolate_multikeys similarly",
      "Identifies that UniformTriRefiner.refine_field calls _interpolate_multikeys with a pre-computed tri_index",
      "Explains that x/y inputs are cast to float64, shape is saved, then flattened via ravel",
      "Explains that scaled coordinates are computed by dividing by _unit_x and _unit_y",
      "Explains that if tri_index is None, self._trifinder(x, y) is called to find containing triangles",
      "Explains the mask_in = (tri_index != -1) logic that separates valid from invalid points",
      "Explains that _tri_renum (if not None) remaps triangle indices for compressed triangulations",
      "Explains the scale factor logic: 1.0 for z, 1/_unit_x for dzdx, 1/_unit_y for dzdy",
      "Explains that results from _interpolate_single_key are placed into valid positions, NaN for invalid, and returned as masked arrays"
    ],
    "key_files": [
      "lib/matplotlib/tri/_triinterpolate.py",
      "lib/matplotlib/tri/_trifinder.py",
      "lib/matplotlib/tri/_trirefine.py",
      "lib/matplotlib/tri/_tritools.py",
      "lib/matplotlib/tri/_triangulation.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_triinterpolate.py] lib.matplotlib.tri._triinterpolate.TriInterpolator\nlib.matplotlib.tri._triinterpolate.TriInterpolator:\n  Abstract base class for classes used to interpolate on a triangular grid.\n\n    Derived classes implement the following methods:\n\n    - ``__call__(x, y)``,\n      where x, y are array-like point coordinates of the same shape, and\n      that returns a masked array of the same shape containing the\n      interpolated z-values.\n\n    - ``gradient(x, y)``,\n      where x, y are array-like point coordinates of the same\n      shape, and that returns a list of 2 masked arrays of the same shape\n      containing the 2 derivatives of the interpolator (derivatives of\n      interpolated z values with respect to x and y).\n\nlib.matplotlib.tri._triinterpolate.TriInterpolator._interpolate_multikeys:\n  Versatile (private) method defined for all TriInterpolators.\n\n        :meth:`_interpolate_multikeys` is a wrapper around method\n        :meth:`_interpolate_single_key` (to be defined in the child\n        subclasses).\n        :meth:`_interpolate_single_key actually performs the interpolation,\n        but only for 1-dimensional inputs and at valid locations (inside\n        unmasked triangles of the triangulation).\n\n        The purpose of :meth:`_interpolate_multikeys` is to implement the\n        following common tasks needed in all subclasses implementations:\n\n        - calculation of containing triangles\n        - dealing with more than one interpolation request at the same\n          location (e.g., if the 2 derivatives are requested, it is\n          unnecessary to compute the containing triangles twice)\n        - scaling according to self._unit_x, self._unit_y\n        - dealing with points outside of the grid (with fill value np.nan)\n        - dealing with multi-dimensional *x*, *y* arrays: flattening for\n          :meth:`_interpolate_params` call and final reshaping.\n\n        (Note that np.vectorize could do most of those things very well for\n        you, but it does it by function evaluations over successive tuples of\n        the input arrays. Therefore, this tends to be more time-consuming than\n        using optimized numpy functions - e.g., np.dot - which can be used\n        easily on the flattened inputs, in the child-subclass methods\n        :meth:`_interpolate_single_key`.)\n\n        It is guaranteed that the calls to :meth:`_interpolate_single_key`\n        will be done with flattened (1-d) array-like input parameters *x*, *y*\n        and with flattened, valid `tri_index` arrays (no -1 index allowed).\n\n        Parameters\n        ----------\n        x, y : array-like\n            x and y coordinates where interpolated values are requested.\n        tri_index : array-like of int, optional\n            Array of the containing triangle indices, same shape as\n            *x* and *y*. Defaults to None. If None, these indices\n            will be computed by a TriFinder instance.\n            (Note: For point outside the grid, tri_index[ipt] shall be -1).\n        return_keys : tuple of keys from {'z', 'dzdx', 'dzdy'}\n            Defines the interpolation arrays to return, and in which order.\n\n        Returns\n        -------\n        list of arrays\n            Each array-like contains the expected interpolated values in the\n            order defined by *return_keys* parameter.\n\nlib.matplotlib.tri._triinterpolate.TriInterpolator._interpolate_single_key:\n  Interpolate at points belonging to the triangulation\n        (inside an unmasked triangles).\n\n        Parameters\n        ----------\n        return_key : {'z', 'dzdx', 'dzdy'}\n            The requested values (z or its derivatives).\n        tri_index : 1D int array\n            Valid triangle index (cannot be -1).\n        x, y : 1D arrays, same shape as `tri_index`\n            Valid locations where interpolation is requested.\n\n        Returns\n        -------\n        1-d array\n            Returned array of the same size as *tri_index*",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many specific implementation details (line numbers, exact variable names, C++ method names, specific coefficient formulas, etc.) go far beyond what the documentation provides and cannot be verified from it alone.",
      "The documentation mentions '_interpolate_params' in the note about reshaping, but the answer doesn't reference this method name, instead using '_interpolate_single_key' consistently which is documented.",
      "Claims about LinearTriInterpolator, CubicTriInterpolator, UniformTriRefiner, TrapezoidMapTriFinder, TriAnalyzer, and their specific behaviors are not in the provided documentation - these are plausible extrapolations from code knowledge but unverifiable from the doc.",
      "The core documented behavior (flattening, triangle finding, scaling, handling invalid points, calling _interpolate_single_key with valid 1D arrays) is accurately represented in the answer.",
      "The return_keys documented as {'z', 'dzdx', 'dzdy'} matches what the answer describes.",
      "The documentation says _interpolate_multikeys deals with 'scaling according to self._unit_x, self._unit_y' which aligns with the answer's description of scaling."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only minor interpretive phrases about design rationale were trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_38",
    "repo": "matplotlib",
    "question": "Why does the `Locator` base class keep `__call__` abstract instead of delegating to `tick_values`, and why are `nonsingular` and `view_limits` separate methods?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `Locator` base class in `lib/matplotlib/ticker.py` has several key design decisions:\n\n1. **Why `__call__` is abstract and cannot simply delegate to `tick_values`**: `IndexLocator.__call__` uses `self.axis.get_data_interval()`, while `MaxNLocator.__call__` and `LogLocator.__call__` use `self.axis.get_view_interval()`. This asymmetry prevents a unified implementation in the base class.\n\n2. **`raise_if_exceeds` behavior**: The current implementation uses `_log.warning(...)` and still returns `locs` regardless of the count. The threshold is `MAXTICKS = 1000` (class variable), and the comparison uses `>=` (not `>`), so exactly 1000 ticks triggers the warning.\n\n3. **Different expander values in `nonsingular` vs `view_limits`**: In `nonsingular`, the expander is 0.05 (5%). In `view_limits`, the default expander is 0.001 (0.1%). The underlying `mtransforms._nonsingular` function handles the expansion: if the interval is smaller than `maxabsvalue * tiny`, it expands by `expander * abs(value)` on each side, or returns `(-expander, expander)` if both values are zero or non-finite.\n\n4. **Why a `_DummyAxis` exists**: The `TickHelper` base class has `create_dummy_axis()` which instantiates `_DummyAxis` if no real axis is set. `_DummyAxis` provides default data/view intervals of (0, 1) and a tick space of 9. This allows locators to function without a real Axis attached.\n\n5. **Why `nonsingular` and `view_limits` are separate methods**: In the autoscaling flow (`axes/_base.py`), `nonsingular` is called first to handle degenerate data ranges (inf, singular points), then margins are computed, and only when not in tight mode is `view_limits` called to potentially round the limits to nice values (e.g., `MaxNLocator.view_limits` rounds to tick boundaries when `axes.autolimit_mode == 'round_numbers'`).",
    "rubric": [
      "Explains that `IndexLocator.__call__` uses `get_data_interval()` while `MaxNLocator.__call__` (and others) use `get_view_interval()`, preventing a unified base implementation",
      "Describes `raise_if_exceeds`: uses `>=` comparison against MAXTICKS=1000, only logs a warning (does not actually raise), and still returns locs",
      "Notes different expander values: `nonsingular` uses 0.05 while `view_limits` defaults to 0.001 (via `mtransforms._nonsingular`'s default)",
      "Explains `_DummyAxis` exists so locators can work without a real Axis, providing default intervals (0,1) and tick_space of 9",
      "Explains the autoscaling flow: `nonsingular` is called first to handle degenerate/infinite ranges, then margins are applied, then `view_limits` is called (only when not tight) to potentially round limits to nice values"
    ],
    "key_files": [
      "lib/matplotlib/ticker.py",
      "lib/matplotlib/transforms.py",
      "lib/matplotlib/axes/_base.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/ticker.py] lib.matplotlib.ticker.Locator\nlib.matplotlib.ticker.Locator:\n  Determine tick locations.\n\n    Note that the same locator should not be used across multiple\n    `~matplotlib.axis.Axis` because the locator stores references to the Axis\n    data and view limits.\n\nlib.matplotlib.ticker.Locator.tick_values:\n  Return the values of the located ticks given **vmin** and **vmax**.\n\n        .. note::\n            To get tick locations with the vmin and vmax values defined\n            automatically for the associated ``axis`` simply call\n            the Locator instance::\n\n                >>> print(type(loc))\n                <type 'Locator'>\n                >>> print(loc())\n                [1, 2, 3, 4]\n\nlib.matplotlib.ticker.Locator.set_params:\n  Do nothing, and raise a warning. Any locator class not supporting the\n        set_params() function will call this.\n\nlib.matplotlib.ticker.Locator.__call__:\n  Return the locations of the ticks.\n\nlib.matplotlib.ticker.Locator.raise_if_exceeds:\n  Log at WARNING level if *locs* is longer than `Locator.MAXTICKS`.\n\n        This is intended to be called immediately before returning *locs* from\n        ``__call__`` to inform users in case their Locator returns a huge\n        number of ticks, causing Matplotlib to run out of memory.\n\n        The \"strange\" name of this method dates back to when it would raise an\n        exception instead of emitting a log.\n\nlib.matplotlib.ticker.Locator.nonsingular:\n  Adjust a range as needed to avoid singularities.\n\n        This method gets called during autoscaling, with ``(v0, v1)`` set to\n        the data limits on the Axes if the Axes contains any data, or\n        ``(-inf, +inf)`` if not.\n\n        - If ``v0 == v1`` (possibly up to some floating point slop), this\n          method returns an expanded interval around this value.\n        - If ``(v0, v1) == (-inf, +inf)``, this method returns appropriate\n          default view limits.\n        - Otherwise, ``(v0, v1)`` is returned without modification.\n\nlib.matplotlib.ticker.Locator.view_limits:\n  Select a scale for the range from vmin to vmax.\n\n        Subclasses should override this method to change locator behaviour.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., line 1755-1756, line 1824, line 2293, etc.) and code details that cannot be verified from the provided documentation alone \u2014 these are plausible code-level details but are not directly supported by the doc chunk.",
      "The claim about `__call__` being abstract and the reason it cannot delegate to `tick_values` is not explicitly stated in the documentation provided, though it is a plausible inference from the separation of `__call__` and `tick_values` methods.",
      "The claim about `raise_if_exceeds` using `MAXTICKS = 1000` and `>=` comparison is not verifiable from the documentation, which only says it logs at WARNING level if locs is longer than MAXTICKS.",
      "The documentation says 'longer than' MAXTICKS, which would suggest `>` not `>=`. The answer claims `>=` (exactly 1000 triggers warning), which could contradict 'longer than'.",
      "The claims about `expander=0.05` vs `expander=0.001` in `nonsingular` and `view_limits` are not mentioned in the documentation at all \u2014 these are code-level implementation details that cannot be verified.",
      "The claim about `_DummyAxis` and `TickHelper` base class is not mentioned anywhere in the provided documentation \u2014 this is plausible but unverifiable.",
      "The claim about the autoscaling flow in `axes/_base.py` is not covered by the documentation provided.",
      "The historical explanation for `raise_if_exceeds` name is correctly supported by the documentation: 'The strange name of this method dates back to when it would raise an exception instead of emitting a log.'"
    ],
    "strip_verify_leakage": 0.3,
    "strip_verify_summary": "Several claims reference comments/docstrings that would be stripped (historical rationale for method names, design purpose comments), while most algorithmic and structural claims are verifiable from code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_42",
    "repo": "matplotlib",
    "question": "Why does `triplot` draw lines and markers as two separate Line2D objects with NaN-separated edge arrays instead of plotting each triangle edge individually?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `triplot` function uses several deliberate design decisions to achieve correct, efficient rendering of triangular grids:\n\n**1. NaN insertion for performance optimization:**\nInstead of plotting each edge as a separate call to `ax.plot()` (which would be slow with many triangles), the code inserts NaN values between edge coordinate pairs using `np.insert(x[edges], 2, np.nan, axis=1)`. This creates a single flattened array where NaN values act as line-break separators, allowing the entire set of edges to be drawn as a single Line2D object.\n\n**2. Separate drawing of lines and markers:**\nMarkers are drawn separately from lines. Markers are plotted using just the raw `x, y` arrays (all points once), while lines use the edges array with NaN separators. The code creates `kw_lines` with `'marker': 'None'` and `kw_markers` with `'linestyle': 'None'` to enforce this separation.\n\n**3. zorder set to 1 instead of Line2D's default of 2:**\nThe line drawing sets `'zorder': kw.get('zorder', 1)`. This uses a default of 1 rather than `Line2D.zorder = 2` (defined in lines.py).\n\n**4. Label removed from markers to prevent duplicate legend entries:**\n`kw_markers.pop('label', None)` ensures that only the lines Line2D gets the label, not the markers.\n\n**5. Format string parsing uses _process_plot_format which has special defaulting behavior:**\nWhen an empty format string is passed (`fmt = args[0] if args else \"\"`), `_process_plot_format` (in axes/_base.py) defaults to `rcParams['lines.linestyle']` for linestyle and 'None' for marker when neither is explicitly specified. This means triplot with no format string draws lines but no markers by default.",
    "rubric": [
      "Explains that NaN insertion (np.insert with np.nan at position 2) creates line-break separators so all edges can be drawn in a single plot call for performance",
      "Explains that markers are drawn separately to avoid duplicate markers at shared vertices/edges (markers plotted using raw x, y arrays once each)",
      "Mentions that kw_lines forces marker='None' and kw_markers forces linestyle='None' to enforce the separation",
      "Notes that the label is popped from kw_markers to prevent duplicate legend entries",
      "Notes that zorder defaults to 1 instead of Line2D's default of 2"
    ],
    "key_files": [
      "lib/matplotlib/tri/_triplot.py",
      "lib/matplotlib/tri/_triangulation.py",
      "lib/matplotlib/axes/_base.py",
      "lib/matplotlib/lines.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_triplot.py] lib.matplotlib.tri._triplot.triplot\nDraw an unstructured triangular grid as lines and/or markers.\n\n    Call signatures::\n\n      triplot(triangulation, ...)\n      triplot(x, y, [triangles], *, [mask=mask], ...)\n\n    The triangular grid can be specified either by passing a `.Triangulation`\n    object as the first parameter, or by passing the points *x*, *y* and\n    optionally the *triangles* and a *mask*. If neither of *triangulation* or\n    *triangles* are given, the triangulation is calculated on the fly.\n\n    Parameters\n    ----------\n    triangulation : `.Triangulation`\n        An already created triangular grid.\n    x, y, triangles, mask\n        Parameters defining the triangular grid. See `.Triangulation`.\n        This is mutually exclusive with specifying *triangulation*.\n    other_parameters\n        All other args and kwargs are forwarded to `~.Axes.plot`.\n\n    Returns\n    -------\n    lines : `~matplotlib.lines.Line2D`\n        The drawn triangles edges.\n    markers : `~matplotlib.lines.Line2D`\n        The drawn marker nodes.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about code comments, design rationale from comments, and test file content were removed or rewritten as they rely on stripped documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_43",
    "repo": "matplotlib",
    "question": "Where is the triangle mesh refinement logic implemented, and how does it handle edge deduplication and child triangle construction?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `UniformTriRefiner` class and its associated refinement logic are located in `lib/matplotlib/tri/_trirefine.py`. Here are the key implementation details discovered from reading the source code:\n\n**Class hierarchy**: `UniformTriRefiner` inherits from `TriRefiner`, which is a base class defined in the same file. `TriRefiner.__init__` validates that the input is a `Triangulation` instance using `_api.check_isinstance` and stores it as `self._triangulation`.\n\n**`_refine_triangulation_once` is a `@staticmethod`**: The core refinement method is a static method, not an instance method. It takes `triangulation` and an optional `ancestors` parameter. It computes the number of new midside points using the formula `added_pts = (3*ntri + borders) // 2`, where `borders = np.sum(neighbors == -1)` counts border edges (edges with no neighboring triangle). The total refined point count is `npts + added_pts`.\n\n**Master/slave edge scheme**: In `_refine_triangulation_once`, edges shared between two triangles are deduplicated using a \"master/slave\" convention: the \"master\" triangle is the one with the **bigger** index (`mask_masters = (edge_elems > edge_neighbors)`). Border edges (neighbor == -1) are also treated as masters since all triangle indices are >= 0 and thus > -1. Midside node coordinates are computed as simple averages of the two endpoint coordinates: `x_add = (x[...] + x[...]) * 0.5`.\n\n**Child triangle construction order**: Each parent triangle produces 4 children stored at indices `[0::4]`, `[1::4]`, `[2::4]`, `[3::4]`. The first 3 children connect a corner to two adjacent midside nodes; the 4th child (index `3::4`) connects the three midside nodes together (the central triangle).\n\n**Mask propagation**: If the input triangulation has a mask, the child triangulation's mask is set by `np.repeat(triangulation.mask, 4)` \u2014 each masked parent produces 4 masked children.\n\n**Ancestor tracking**: When `ancestors` is provided, the returned `child_ancestors` is computed as `np.repeat(ancestors, 4)`, so all 4 children share their parent's ancestor index.\n\n**`refine_triangulation` found_index handling with masks**: When `return_tri_index=True` and the triangulation has a mask, the code applies masked ancestor indices first, then overwrites with unmasked ancestor indices.\n\n**`refine_field` default interpolator**: When no `triinterpolator` is passed to `refine_field`, it creates a `matplotlib.tri.CubicTriInterpolator` (not `LinearTriInterpolator`). The interpolation is done via `interp._interpolate_multikeys(refi_tri.x, refi_tri.y, tri_index=found_index)[0]`, where `[0]` extracts only the 'z' values (default `return_keys=('z',)`).\n\n**`_interpolate_multikeys` tri_index bypass**: The `found_index` from `refine_triangulation` is passed as `tri_index` to `_interpolate_multikeys`, which bypasses the TriFinder lookup that would otherwise be called. Points outside the grid have `found_index == -1` and are masked as `np.nan` in the output.\n\n**`found_index` initialization**: The `found_index` array is initialized with `-1` (not 0) via `np.full(refi_npts, -1, dtype=np.int32)`.",
    "rubric": [
      "Located in lib/matplotlib/tri/_trirefine.py with UniformTriRefiner inheriting from TriRefiner base class",
      "_refine_triangulation_once is a @staticmethod that takes triangulation and optional ancestors parameter",
      "Edge deduplication uses a master/slave scheme where the master triangle has the bigger index (edge_elems > edge_neighbors), and border edges (neighbor == -1) are automatically masters",
      "New midside point count formula: added_pts = (3*ntri + borders) // 2 where borders counts edges with no neighbor",
      "Each parent triangle produces 4 children at indices [0::4], [1::4], [2::4], [3::4]; first 3 connect a corner to two midside nodes, the 4th connects the three midside nodes (central triangle)",
      "Mask propagation uses np.repeat(triangulation.mask, 4) so each masked parent produces 4 masked children",
      "Ancestor tracking uses np.repeat(ancestors, 4) so all 4 children share their parent's ancestor index",
      "Midside node coordinates are simple averages: (x[...] + x[...]) * 0.5"
    ],
    "key_files": [
      "lib/matplotlib/tri/_trirefine.py",
      "lib/matplotlib/tri/_triinterpolate.py",
      "lib/matplotlib/tri/_triangulation.py",
      "lib/matplotlib/tri/_tritools.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_trirefine.py] lib.matplotlib.tri._trirefine.UniformTriRefiner\nlib.matplotlib.tri._trirefine.UniformTriRefiner:\n  Uniform mesh refinement by recursive subdivisions.\n\n    Parameters\n    ----------\n    triangulation : `~matplotlib.tri.Triangulation`\n        The encapsulated triangulation (to be refined)\n\nlib.matplotlib.tri._trirefine.UniformTriRefiner.refine_triangulation:\n  Compute a uniformly refined triangulation *refi_triangulation* of\n        the encapsulated :attr:`!triangulation`.\n\n        This function refines the encapsulated triangulation by splitting each\n        father triangle into 4 child sub-triangles built on the edges midside\n        nodes, recursing *subdiv* times.  In the end, each triangle is hence\n        divided into ``4**subdiv`` child triangles.\n\n        Parameters\n        ----------\n        return_tri_index : bool, default: False\n            Whether an index table indicating the father triangle index of each\n            point is returned.\n        subdiv : int, default: 3\n            Recursion level for the subdivision.\n            Each triangle is divided into ``4**subdiv`` child triangles;\n            hence, the default results in 64 refined subtriangles for each\n            triangle of the initial triangulation.\n\n        Returns\n        -------\n        refi_triangulation : `~matplotlib.tri.Triangulation`\n            The refined triangulation.\n        found_index : int array\n            Index of the initial triangulation containing triangle, for each\n            point of *refi_triangulation*.\n            Returned only if *return_tri_index* is set to True.\n\nlib.matplotlib.tri._trirefine.UniformTriRefiner.refine_field:\n  Refine a field defined on the encapsulated triangulation.\n\n        Parameters\n        ----------\n        z : (npoints,) array-like\n            Values of the field to refine, defined at the nodes of the\n            encapsulated triangulation. (``n_points`` is the number of points\n            in the initial triangulation)\n        triinterpolator : `~matplotlib.tri.TriInterpolator`, optional\n            Interpolator used for field interpolation. If not specified,\n            a `~matplotlib.tri.CubicTriInterpolator` will be used.\n        subdiv : int, default: 3\n            Recursion level for the subdivision.\n            Each triangle is divided into ``4**subdiv`` child triangles.\n\n        Returns\n        -------\n        refi_tri : `~matplotlib.tri.Triangulation`\n             The returned refined triangulation.\n        refi_z : 1D array of length: *refi_tri* node count.\n             The returned interpolated field (at *refi_tri* nodes).\n\nlib.matplotlib.tri._trirefine.UniformTriRefiner._refine_triangulation_once:\n  Refine a `.Triangulation` by splitting each triangle into 4\n        child-masked_triangles built on the edges midside nodes.\n\n        Masked triangles, if present, are also split, but their children\n        returned masked.\n\n        If *ancestors* is not provided, returns only a new triangulation:\n        child_triangulation.\n\n        If the array-like key table *ancestor* is given, it shall be of shape\n        (ntri,) where ntri is the number of *triangulation* masked_triangles.\n        In this case, the function returns\n        (child_triangulation, child_ancestors)\n        child_ancestors is defined so that the 4 child masked_triangles share\n        the same index as their father: child_ancestors.shape = (4 * ntri,).",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: design rationale for mask ordering and found_index initialization, plus .pyi stub claim not verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_41",
    "repo": "matplotlib",
    "question": "What is the full architecture of the `tripcolor` function \u2014 how is it attached to Axes, how does it resolve colors, and what collection types does it produce?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `tripcolor` function in `lib/matplotlib/tri/_tripcolor.py` is a standalone function (not a method) that takes `ax` as its first parameter and is attached to the `Axes` class via `_make_axes_method(mtri.tripcolor)` in `lib/matplotlib/axes/_axes.py`. The `_make_axes_method` helper patches the function's `__qualname__` to `\"Axes.tripcolor\"`.\n\nKey implementation details:\n\n1. **Color determination logic**: When `c` is provided positionally and its length equals `len(tri.x)` (number of points), it's treated as point colors \u2014 this takes **precedence** over face colors even if the number of points equals the number of triangles (the `if len(c) == len(tri.x)` check comes before the `elif len(c) == len(tri.triangles)` check). For flat shading with point colors, the color for each triangle is computed as the mean of the 3 vertex colors: `colors = point_colors[maskedTris].mean(axis=1)`.\n\n2. **Two collection types returned**: For `shading='flat'`, vertices are assembled using `tri.get_masked_triangles()` and `np.stack((tri.x[maskedTris], tri.y[maskedTris]), axis=-1)`, creating a `PolyCollection`. For `shading='gouraud'`, a `TriMesh` is created directly from the triangulation.\n\n3. **TriMesh class** (in `lib/matplotlib/collections.py`) is imported alongside `PolyCollection` in `_tripcolor.py`.\n\n4. **Collection** inherits from `mcolorizer.ColorizingArtist`, as visible in the class definition.\n\n5. **Default styling**: The function sets default `linewidths=(0.25,)`, `edgecolors='none'`, and when edgecolors is 'none', sets `antialiaseds=False`.\n\n6. **Triangulation parsing**: `Triangulation.get_from_args_and_kwargs` checks if the first positional arg is already a `Triangulation` instance; if not, it calls `_extract_triangulation_params` which attempts to convert the third positional argument to a (N,3) int32 array \u2014 if this fails (non-integer or wrong shape), it's left as a remaining arg (which `tripcolor` then interprets as the color value `c`).\n\n7. **After creating the collection**, the function calls `collection._scale_norm(norm, vmin, vmax)`, then `ax.grid(False)`, updates data limits using the triangulation's x/y min/max bounds, calls `ax.autoscale_view()`, and finally `ax.add_collection(collection, autolim=False)`.",
    "rubric": [
      "Explains that tripcolor is a standalone function taking `ax` as its first parameter, attached to Axes via `_make_axes_method` which patches __qualname__ to 'Axes.tripcolor'",
      "Describes the color precedence logic: when len(c) == len(tri.x), colors are treated as point colors, taking precedence over face colors even if the counts are equal",
      "Explains that for flat shading with point colors, the color per triangle is computed as the mean of the 3 vertex colors: point_colors[maskedTris].mean(axis=1)",
      "Identifies that shading='flat' returns a PolyCollection (with vertices assembled via get_masked_triangles and np.stack) while shading='gouraud' returns a TriMesh",
      "Mentions TriMesh inherits from Collection, which inherits from mcolorizer.ColorizingArtist",
      "Notes default styling: linewidths=(0.25,), edgecolors='none', and antialiaseds=False when edgecolors is 'none'",
      "Describes the triangulation parsing: get_from_args_and_kwargs checks if first arg is a Triangulation, otherwise calls _extract_triangulation_params which tries to convert the third arg to int32 (N,3) array",
      "Mentions post-collection steps: _scale_norm, ax.grid(False), update_datalim with tri x/y bounds, autoscale_view, and add_collection with autolim=False"
    ],
    "key_files": [
      "lib/matplotlib/tri/_tripcolor.py",
      "lib/matplotlib/collections.py",
      "lib/matplotlib/tri/_triangulation.py",
      "lib/matplotlib/axes/_axes.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_tripcolor.py] lib.matplotlib.tri._tripcolor.tripcolor\nCreate a pseudocolor plot of an unstructured triangular grid.\n\n    Call signatures::\n\n      tripcolor(triangulation, c, *, ...)\n      tripcolor(x, y, c, *, [triangles=triangles], [mask=mask], ...)\n\n    The triangular grid can be specified either by passing a `.Triangulation`\n    object as the first parameter, or by passing the points *x*, *y* and\n    optionally the *triangles* and a *mask*. See `.Triangulation` for an\n    explanation of these parameters.\n\n    It is possible to pass the triangles positionally, i.e.\n    ``tripcolor(x, y, triangles, c, ...)``. However, this is discouraged.\n    For more clarity, pass *triangles* via keyword argument.\n\n    If neither of *triangulation* or *triangles* are given, the triangulation\n    is calculated on the fly. In this case, it does not make sense to provide\n    colors at the triangle faces via *c* or *facecolors* because there are\n    multiple possible triangulations for a group of points and you don't know\n    which triangles will be constructed.\n\n    Parameters\n    ----------\n    triangulation : `.Triangulation`\n        An already created triangular grid.\n    x, y, triangles, mask\n        Parameters defining the triangular grid. See `.Triangulation`.\n        This is mutually exclusive with specifying *triangulation*.\n    c : array-like\n        The color values, either for the points or for the triangles. Which one\n        is automatically inferred from the length of *c*, i.e. does it match\n        the number of points or the number of triangles. If there are the same\n        number of points and triangles in the triangulation it is assumed that\n        color values are defined at points; to force the use of color values at\n        triangles use the keyword argument ``facecolors=c`` instead of just\n        ``c``.\n        This parameter is position-only.\n    facecolors : array-like, optional\n        Can be used alternatively to *c* to specify colors at the triangle\n        faces. This parameter takes precedence over *c*.\n    shading : {'flat', 'gouraud'}, default: 'flat'\n        If  'flat' and the color values *c* are defined at points, the color\n        values used for each triangle are from the mean c of the triangle's\n        three points. If *shading* is 'gouraud' then color values must be\n        defined at points.\n    %(cmap_doc)s\n\n    %(norm_doc)s\n\n    %(vmin_vmax_doc)s\n\n    %(colorizer_doc)s\n\n    Returns\n    -------\n    `~matplotlib.collections.PolyCollection` or `~matplotlib.collections.TriMesh`\n        The result depends on *shading*: For ``shading='flat'`` the result is a\n        `.PolyCollection`, for ``shading='gouraud'`` the result is a `.TriMesh`.\n\n    Other Parameters\n    ----------------\n    **kwargs : `~matplotlib.collections.Collection` properties\n\n        %(Collection:kwdoc)s",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed documentation-only descriptions ('pseudocolor plot', 'mimics pcolor') and trimmed unverifiable details about TriMesh internals and pyplot wrapper.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_44",
    "repo": "matplotlib",
    "question": "How does the `get_flat_tri_mask` method in `TriAnalyzer` iteratively identify and remove flat border triangles using its wavefront approach?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `TriAnalyzer` class in `lib/matplotlib/tri/_tritools.py` implements several algorithms for triangular mesh analysis:\n\n### Constructor and Validation\nThe `__init__` method uses `_api.check_isinstance(Triangulation, triangulation=triangulation)` to validate the input, ensuring only `Triangulation` objects are accepted. The triangulation is stored as `self._triangulation`.\n\n### `scale_factors` Property Algorithm\nComputes scaling factors to fit unmasked triangles into a unit square:\n1. Gets masked triangles via `self._triangulation.get_masked_triangles()` (which returns `triangles[~mask]` if mask exists, else all triangles).\n2. Uses `np.bincount` on the raveled triangle indices to determine which nodes are actually used (count != 0).\n3. Returns `(1/np.ptp(x[node_used]), 1/np.ptp(y[node_used]))` \u2014 the reciprocal of the peak-to-peak range for each coordinate.\n\n### `circle_ratios` Algorithm\nComputes incircle/circumcircle radius ratio for each triangle:\n1. Rescales coordinates by `scale_factors` if `rescale=True`, otherwise uses (1.0, 1.0).\n2. Stacks x*kx and y*ky into a (npoints, 2) array `pts`, then indexes by triangles to get `tri_pts` of shape (ntri, 3, 2).\n3. Computes the three side length vectors as differences between triangle vertices: `a = tri_pts[:, 1, :] - tri_pts[:, 0, :]`, etc., then converts to scalar lengths using `np.hypot`.\n4. Computes semi-perimeter `s = (a+b+c)*0.5`.\n5. Computes `prod = s*(a+b-s)*(a+c-s)*(b+c-s)` \u2014 related to Heron's formula for the triangle area.\n6. Handles degenerate (flat) triangles specially: if `prod == 0` for any triangle, sets `circum_radius` to `np.inf` for those, while computing `abc / (4*sqrt(prod))` for valid ones. In the non-pathologic case, directly computes `circum_radius = (a*b*c) / (4.0*np.sqrt(prod))`.\n7. Incircle radius is computed as `in_radius = (a*b*c) / (4.0*circum_radius*s)`.\n8. Final ratio is `in_radius / circum_radius`. Returns plain array if no mask, or `np.ma.array(circle_ratio, mask=mask)` if masked.\n\n### `get_flat_tri_mask` Wavefront Algorithm\nIteratively removes flat border triangles:\n1. Computes `mask_bad_ratio = self.circle_ratios(rescale) < min_circle_ratio` for all triangles.\n2. Copies `self._triangulation.neighbors` into `valid_neighbors` (shape ntri\u00d73, -1 means no neighbor).\n3. Creates a `renum_neighbors` identity array of shape (ntri,) with dtype `np.int32`.\n4. In a while loop (terminates when `nadd == 0`):\n   - Identifies the **wavefront**: unmasked triangles that have at least one neighbor equal to -1 (i.e., they're on the current border), computed as `(np.min(valid_neighbors, axis=1) == -1) & ~current_mask`.\n   - Marks triangles for masking: `added_mask = wavefront & mask_bad_ratio`.\n   - Updates `current_mask = added_mask | current_mask`.\n   - For newly masked triangles, sets their `valid_neighbors` rows to -1 and their `renum_neighbors` entries to -1.\n   - Propagates the neighbor invalidation: `valid_neighbors = np.where(valid_neighbors == -1, -1, renum_neighbors[valid_neighbors])` \u2014 this causes any triangle neighboring a newly-masked triangle to now see -1 in that neighbor slot, making it part of the new border.\n5. Returns `np.ma.filled(current_mask, True)` \u2014 meaning any masked values in the mask array itself are treated as True (masked).\n\n### `_get_compressed_triangulation` Algorithm\n1. Gets unmasked triangles via `get_masked_triangles()`.\n2. For triangle renumbering: if mask exists, calls `_total_to_compress_renum(~tri_mask)`, otherwise creates an identity arange.\n3. For node renumbering: uses `np.bincount` to find which nodes appear in unmasked triangles (`valid_node`), extracts only those x/y coordinates.\n4. Calls `_total_to_compress_renum(valid_node)` to build node renumbering.\n5. Applies node renumbering to the compressed triangles: `compressed_triangles = node_renum[compressed_triangles]`.\n6. Returns 5-tuple: (compressed_triangles, compressed_x, compressed_y, tri_renum, node_renum).\n\n### `_total_to_compress_renum` Static Method\nCreates a renumbering array: initializes with `np.full(np.size(valid), -1, dtype=np.int32)`, then sets `renum[valid] = np.arange(n_valid, dtype=np.int32)`. This maps valid indices to consecutive integers starting from 0, and invalid indices to -1.\n\n### Key Integration Point\n`_get_compressed_triangulation` is used by `CubicTriInterpolator.__init__` in `_triinterpolate.py` (line 396-398) to compress the triangulation before building a stiffness matrix. The compressed data is used to compute scale factors (`unit_x`, `unit_y`), triangle points, eccentricities, and DOF estimations for Hermite Clough-Tocher (HCT) element interpolation.\n\nThe `neighbors` array used in `get_flat_tri_mask` is lazily computed from the underlying C++ triangulation (`self.get_cpp_triangulation().get_neighbors()`) and is cleared (set to None) whenever `set_mask` is called on the `Triangulation` object, forcing recalculation.",
    "rubric": [
      "Explains that circle_ratios are computed and compared against min_circle_ratio to identify bad triangles",
      "Describes that the wavefront consists of unmasked triangles with at least one neighbor equal to -1 (i.e., on the current border)",
      "Explains that only wavefront triangles with bad circle ratios are masked in each iteration",
      "Describes how valid_neighbors is updated: newly masked triangles get their rows set to -1",
      "Explains how renum_neighbors propagates the masking: entries for masked triangles become -1, and valid_neighbors is updated via np.where so neighbors of newly masked triangles see -1 in the relevant slot",
      "Notes that the loop terminates when no new triangles are added to the mask (nadd == 0)",
      "Mentions that the final result uses np.ma.filled(current_mask, True) to treat any masked values as True"
    ],
    "key_files": [
      "lib/matplotlib/tri/_tritools.py",
      "lib/matplotlib/tri/_triangulation.py",
      "lib/matplotlib/tri/_triinterpolate.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/tri/_tritools.py] lib.matplotlib.tri._tritools.TriAnalyzer\nlib.matplotlib.tri._tritools.TriAnalyzer:\n  Define basic tools for triangular mesh analysis and improvement.\n\n    A TriAnalyzer encapsulates a `.Triangulation` object and provides basic\n    tools for mesh analysis and mesh improvement.\n\n    Attributes\n    ----------\n    scale_factors\n\n    Parameters\n    ----------\n    triangulation : `~matplotlib.tri.Triangulation`\n        The encapsulated triangulation to analyze.\n\nlib.matplotlib.tri._tritools.TriAnalyzer.scale_factors:\n  Factors to rescale the triangulation into a unit square.\n\n        Returns\n        -------\n        (float, float)\n            Scaling factors (kx, ky) so that the triangulation\n            ``[triangulation.x * kx, triangulation.y * ky]``\n            fits exactly inside a unit square.\n\nlib.matplotlib.tri._tritools.TriAnalyzer.circle_ratios:\n  Return a measure of the triangulation triangles flatness.\n\n        The ratio of the incircle radius over the circumcircle radius is a\n        widely used indicator of a triangle flatness.\n        It is always ``<= 0.5`` and ``== 0.5`` only for equilateral\n        triangles. Circle ratios below 0.01 denote very flat triangles.\n\n        To avoid unduly low values due to a difference of scale between the 2\n        axis, the triangular mesh can first be rescaled to fit inside a unit\n        square with `scale_factors` (Only if *rescale* is True, which is\n        its default value).\n\n        Parameters\n        ----------\n        rescale : bool, default: True\n            If True, internally rescale (based on `scale_factors`), so that the\n            (unmasked) triangles fit exactly inside a unit square mesh.\n\n        Returns\n        -------\n        masked array\n            Ratio of the incircle radius over the circumcircle radius, for\n            each 'rescaled' triangle of the encapsulated triangulation.\n            Values corresponding to masked triangles are masked out.\n\nlib.matplotlib.tri._tritools.TriAnalyzer.get_flat_tri_mask:\n  Eliminate excessively flat border triangles from the triangulation.\n\n        Returns a mask *new_mask* which allows to clean the encapsulated\n        triangulation from its border-located flat triangles\n        (according to their :meth:`circle_ratios`).\n        This mask is meant to be subsequently applied to the triangulation\n        using `.Triangulation.set_mask`.\n        *new_mask* is an extension of the initial triangulation mask\n        in the sense that an initially masked triangle will remain masked.\n\n        The *new_mask* array is computed recursively; at each step flat\n        triangles are removed only if they share a side with the current mesh\n        border. Thus, no new holes in the triangulated domain will be created.\n\n        Parameters\n        ----------\n        min_circle_ratio : float, default: 0.01\n            Border triangles with incircle/circumcircle radii ratio r/R will\n            be removed if r/R < *min_circle_ratio*.\n        rescale : bool, default: True\n            If True, first, internally rescale (based on `scale_factors`) so\n            that the (unmasked) triangles fit exactly inside a unit square\n            mesh.  This rescaling accounts for the difference of scale which\n            might exist between the 2 axis.\n\n        Returns\n        -------\n        array of bool\n            Mask to apply to encapsulated triangulation.\n            All the initially masked triangles remain masked in the\n            *new_mask*.\n\n        Notes\n        -----\n        The rationale behind this function is that a Delaunay\n        triangulation - of an unstructured set of points - sometimes contains\n        almost flat triangles at its border, leading to artifacts in plots\n        (especially for high-resolution contouring).\n        Masked with computed *new_mask*, the encapsulated\n        triangulation would contain no more unmasked border triangles\n        with a circle ratio below *min_circle_ratio*, thus improving the\n        mesh quality for subsequent plots or interpolation.\n\nlib.matplotlib.tri._tritools.TriAnalyzer._get_compressed_triangulation:\n  Compress (if masked) the encapsulated triangulation.\n\n        Returns minimal-length triangles array (*compressed_triangles*) and\n        coordinates arrays (*compressed_x*, *compressed_y*) that can still\n        describe the unmasked triangles of the encapsulated triangulation.\n\n        Returns\n        -------\n        compressed_triangles : array-like\n            the returned compressed triangulation triangles\n        compressed_x : array-like\n            the returned compressed triangulation 1st coordinate\n        compressed_y : array-like\n            the returned compressed triangulation 2nd coordinate\n        tri_renum : int array\n            renumbering table to translate the triangle numbers from the\n            encapsulated triangulation into the new (compressed) renumbering.\n            -1 for masked triangles (deleted from *compressed_triangles*).\n        node_renum : int array\n            renumbering table to translate the point numbers from the\n            encapsulated triangulation into the new (compressed) renumbering.\n            -1 for unused points (i.e. those deleted from *compressed_x* and\n            *compressed_y*).\n\nlib.matplotlib.tri._tritools.TriAnalyzer._total_to_compress_renum:\n  Parameters\n        ----------\n        valid : 1D bool array\n            Validity mask.\n\n        Returns\n        -------\n        int array\n            Array so that (`valid_array` being a compressed array\n            based on a `masked_array` with mask ~*valid*):\n\n            - For all i with valid[i] = True:\n              valid_array[renum[i]] = masked_array[i]\n            - For all i with valid[i] = False:\n              renum[i] = -1 (invalid value)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (specific numpy operations, variable names, loop structures, line numbers like 'line 396-398') that go far beyond what the documentation provides. These appear to be derived from reading the actual source code rather than the documentation chunk provided.",
      "The claim about `_api.check_isinstance(Triangulation, triangulation=triangulation)` in __init__ is not in the documentation \u2014 it's plausible but unverifiable from the doc.",
      "The detailed algorithm for circle_ratios (Heron's formula steps, handling of degenerate triangles with np.inf, etc.) is not documented \u2014 these are code-level details that could be correct but cannot be verified from the doc alone.",
      "The claim about CubicTriInterpolator.__init__ using _get_compressed_triangulation at 'line 396-398' is completely outside the scope of the provided documentation.",
      "The claim about neighbors being lazily computed from C++ triangulation and cleared on set_mask is not in the provided documentation.",
      "The get_flat_tri_mask algorithm description with renum_neighbors and wavefront propagation details goes beyond the doc, which only describes the recursive border-removal concept at a high level.",
      "The circle_ratios documentation says the ratio is 'always <= 0.5 and == 0.5 only for equilateral triangles' \u2014 the answer doesn't contradict this but doesn't mention it either.",
      "The scale_factors description using np.ptp and np.bincount is plausible implementation detail but not verifiable from doc.",
      "The answer's description of get_flat_tri_mask returning `np.ma.filled(current_mask, True)` is implementation detail not in the doc; the doc says it returns 'array of bool' as a mask."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_47",
    "repo": "matplotlib",
    "question": "Where is the Grid and ImageGrid layout machinery defined, and where do the axes classes, dividers, and colorbar axes come from?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Grid` class is defined in `lib/mpl_toolkits/axes_grid1/axes_grid.py`. It uses `SubplotDivider` (from `lib/mpl_toolkits/axes_grid1/axes_divider.py`) or `Divider` depending on the `rect` parameter type. `SubplotDivider` inherits from `Divider` and initializes with position `[0, 0, 1, 1]`, overriding `get_position()` to use `SubplotSpec.get_position()`. The Grid's `_defaultAxesClass` is `mpl_toolkits.axes_grid1.mpl_axes.Axes`, which itself inherits from `matplotlib.axes.Axes` and adds an `AxisDict` that stores `SimpleAxisArtist` instances keyed by 'bottom', 'top', 'left', 'right'. The `axis` property on `mpl_axes.Axes` is **not** a `MethodType` (it's a property returning `self._axislines`), so the `set_label_mode` method detects this with `isinstance(ax.axis, MethodType)` \u2014 if it's a MethodType (standard matplotlib Axes), it creates new `SimpleAxisArtist` instances; otherwise it accesses `ax.axis['bottom']` and `ax.axis['left']` from the AxisDict. The `ImageGrid` class inherits from `Grid` and overrides `_init_locators()`. Colorbar axes are created using `_cbaraxes_class_factory`, which is built via `cbook._make_class_factory(CbarAxesBase, 'Cbar{}')`. This factory is called as `_cbaraxes_class_factory(self._defaultAxesClass)(...)` to create colorbar axes, using the naming format string 'Cbar{}'. The `Grid._init_locators` method uses `Size.Scaled(1)` for each axes column/row and inserts the pad sizes between them, positioning axes at even indices (2*col, 2*(nrows-1-row)).",
    "rubric": [
      "Grid is defined in axes_grid.py (lib/mpl_toolkits/axes_grid1/axes_grid.py)",
      "Grid uses SubplotDivider or Divider depending on whether rect is a SubplotSpec/int or a 4-element tuple",
      "SubplotDivider inherits from Divider and initializes with position [0, 0, 1, 1], overriding get_position() to use SubplotSpec.get_position()",
      "Grid._defaultAxesClass is mpl_toolkits.axes_grid1.mpl_axes.Axes, which inherits from matplotlib.axes.Axes",
      "mpl_axes.Axes has an AxisDict storing SimpleAxisArtist instances keyed by 'bottom', 'top', 'left', 'right', exposed via a property called axis",
      "set_label_mode uses isinstance(ax.axis, MethodType) to distinguish standard matplotlib Axes (where axis is a method) from mpl_axes.Axes (where axis is a property returning AxisDict)",
      "ImageGrid inherits from Grid and overrides _init_locators() for colorbar support",
      "Colorbar axes are created via _cbaraxes_class_factory, built by cbook._make_class_factory(CbarAxesBase, 'Cbar{}')",
      "Grid._init_locators uses Size.Scaled(1) for axes and pads between them, placing axes at even indices (2*col, 2*(nrows-1-row))"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/axes_grid.py",
      "lib/mpl_toolkits/axes_grid1/mpl_axes.py",
      "lib/mpl_toolkits/axes_grid1/axes_divider.py",
      "lib/mpl_toolkits/axes_grid1/axes_size.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/axes_grid.py] lib.mpl_toolkits.axes_grid1.axes_grid.Grid\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid:\n  A grid of Axes.\n\n    In Matplotlib, the Axes location (and size) is specified in normalized\n    figure coordinates. This may not be ideal for images that needs to be\n    displayed with a given aspect ratio; for example, it is difficult to\n    display multiple images of a same size with some fixed padding between\n    them.  AxesGrid can be used in such case.\n\n    Attributes\n    ----------\n    axes_all : list of Axes\n        A flat list of Axes. Note that you can also access this directly\n        from the grid. The following is equivalent ::\n\n            grid[i] == grid.axes_all[i]\n            len(grid) == len(grid.axes_all)\n\n    axes_column : list of list of Axes\n        A 2D list of Axes where the first index is the column. This results\n        in the usage pattern ``grid.axes_column[col][row]``.\n    axes_row : list of list of Axes\n        A 2D list of Axes where the first index is the row. This results\n        in the usage pattern ``grid.axes_row[row][col]``.\n    axes_llc : Axes\n        The Axes in the lower left corner.\n    n_axes : int\n        Number of Axes in the grid.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.__init__:\n  Parameters\n        ----------\n        fig : `.Figure`\n            The parent figure.\n        rect : (float, float, float, float), (int, int, int), int, or     `~.SubplotSpec`\n            The axes position, as a ``(left, bottom, width, height)`` tuple,\n            as a three-digit subplot position code (e.g., ``(1, 2, 1)`` or\n            ``121``), or as a `~.SubplotSpec`.\n        nrows_ncols : (int, int)\n            Number of rows and columns in the grid.\n        n_axes : int, optional\n            If given, only the first *n_axes* axes in the grid are created.\n        direction : {\"row\", \"column\"}, default: \"row\"\n            Whether axes are created in row-major (\"row by row\") or\n            column-major order (\"column by column\").  This also affects the\n            order in which axes are accessed using indexing (``grid[index]``).\n        axes_pad : float or (float, float), default: 0.02\n            Padding or (horizontal padding, vertical padding) between axes, in\n            inches.\n        share_all : bool, default: False\n            Whether all axes share their x- and y-axis.  Overrides *share_x*\n            and *share_y*.\n        share_x : bool, default: True\n            Whether all axes of a column share their x-axis.\n        share_y : bool, default: True\n            Whether all axes of a row share their y-axis.\n        label_mode : {\"L\", \"1\", \"all\", \"keep\"}, default: \"L\"\n            Determines which axes will get tick labels:\n\n            - \"L\": All axes on the left column get vertical tick labels;\n              all axes on the bottom row get horizontal tick labels.\n            - \"1\": Only the bottom left axes is labelled.\n            - \"all\": All axes are labelled.\n            - \"keep\": Do not do anything.\n\n        axes_class : subclass of `matplotlib.axes.Axes`, default: `.mpl_axes.Axes`\n            The type of Axes to create.\n        aspect : bool, default: False\n            Whether the axes aspect ratio follows the aspect ratio of the data\n            limits.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.get_geometry:\n  Return the number of rows and columns of the grid as (nrows, ncols).\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.set_axes_pad:\n  Set the padding between the axes.\n\n        Parameters\n        ----------\n        axes_pad : (float, float)\n            The padding (horizontal pad, vertical pad) in inches.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.get_axes_pad:\n  Return the axes padding.\n\n        Returns\n        -------\n        hpad, vpad\n            Padding (horizontal pad, vertical pad) in inches.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.set_aspect:\n  Set the aspect of the SubplotDivider.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.get_aspect:\n  Return the aspect of the SubplotDivider.\n\nlib.mpl_toolkits.axes_grid1.axes_grid.Grid.set_label_mode:\n  Define which axes have tick labels.\n\n        Parameters\n        ----------\n        mode : {\"L\", \"1\", \"all\", \"keep\"}\n            The label mode:\n\n            - \"L\": All axes on the left column get vertical tick labels;\n              all axes on the bottom row get horizontal tick labels.\n            - \"1\": Only the bottom left axes is labelled.\n            - \"all\": All axes are labelled.\n            - \"keep\": Do not do anything.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many implementation details (SubplotDivider, Divider, AxisDict, SimpleAxisArtist, MethodType detection, _cbaraxes_class_factory, Size.Scaled, positioning formula) that are not present in the provided documentation. These cannot be verified from the doc chunk alone, but they are plausible code-level details.",
      "The answer states '_defaultAxesClass is mpl_toolkits.axes_grid1.mpl_axes.Axes' \u2014 the documentation says 'axes_class : subclass of matplotlib.axes.Axes, default: .mpl_axes.Axes', which is consistent.",
      "The answer mentions ImageGrid/AxesGrid inheriting from Grid \u2014 not mentioned in the documentation but is a plausible code-level detail.",
      "No claims in the answer directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; minor rewrites needed for the AxesGrid alias (not visible in truncated code) and internal details of _make_class_factory.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_46",
    "repo": "matplotlib",
    "question": "Why does HBoxDivider use `np.linalg.solve` instead of the simpler `_calc_k` formula used by the base Divider class?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `HBoxDivider` uses `np.linalg.solve` in its `_locate` helper function (module-level) rather than a simpler per-axis `_calc_k` calculation (used by the base `Divider` class) because it needs to simultaneously solve N+1 unknowns: N individual k-factors (one per axes column) plus a common height H. The constraint system requires that:\n\n1. All axes achieve the same height H (eq_rel_i * k_i + eq_abs_i = H for each i)\n2. The total width equals a fixed available width (sum of sm_rel_i * k_i + sm_abs_i = total_width)\n\nThis is a coupled system of linear equations that cannot be solved independently per-axis (unlike the base `Divider._calc_k` which uses a simple scalar formula `(total - abs_sum) / rel_sum` with a single shared k-factor). Using `np.linalg.solve` on the (N+1)\u00d7(N+1) matrix allows solving all constraints in one shot without iteration.\n\nThe matrix A is constructed as a diagonal with `eq_rels` values and a zero in the last position, with -1 in the last column of the first N rows (representing the shared height unknown H), and `sm_rels` in the last row (the total-width constraint).\n\nAdditionally, after solving, if the computed height exceeds `max_height` (the available vertical space), it falls back to a direct computation: `karray = (max_height - eq_abss) / eq_rels`, which caps the height and potentially makes the total width less than the available space, allowing proper anchoring within the bounding box.",
    "rubric": [
      "Explains that HBoxDivider needs to solve a coupled system where all axes share the same height H but have individual k-factors",
      "Mentions the system has N+1 unknowns: N per-axis k-factors plus a shared height H",
      "Describes the two types of constraints: equal height for all axes (eq_rel_i * k_i + eq_abs_i = H) and fixed total width (sum of sm_rel_i * k_i + sm_abs_i = total_width)",
      "Explains that `_calc_k` uses a simple scalar formula (total - abs_sum) / rel_sum with a single shared k-factor, which cannot handle per-axis k-factors",
      "Mentions the matrix structure: diagonal of eq_rels with -1 in last column for the first N rows, and sm_rels in the last row",
      "Notes the fallback behavior when height exceeds max_height: directly computes karray = (max_height - eq_abss) / eq_rels"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/axes_divider.py",
      "lib/mpl_toolkits/axes_grid1/axes_size.py",
      "lib/matplotlib/axes/_base.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/axes_divider.py] lib.mpl_toolkits.axes_grid1.axes_divider.HBoxDivider\nlib.mpl_toolkits.axes_grid1.axes_divider.HBoxDivider:\n  A `.SubplotDivider` for laying out axes horizontally, while ensuring that\n    they have equal heights.\n\n    Examples\n    --------\n    .. plot:: gallery/axes_grid1/demo_axes_hbox_divider.py\n\nlib.mpl_toolkits.axes_grid1.axes_divider.HBoxDivider.new_locator:\n  Create an axes locator callable for the specified cell.\n\n        Parameters\n        ----------\n        nx, nx1 : int\n            Integers specifying the column-position of the\n            cell. When *nx1* is None, a single *nx*-th column is\n            specified. Otherwise, location of columns spanning between *nx*\n            to *nx1* (but excluding *nx1*-th column) is specified.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided only describes HBoxDivider at a high level (laying out axes horizontally with equal heights) and documents the new_locator method's parameters. The generated answer discusses internal implementation details (np.linalg.solve, _locate helper function, matrix construction, _calc_k, max_height fallback, etc.) that are not present in the provided documentation chunk.",
      "While these implementation details are plausible and likely accurate based on the actual source code of axes_divider.py, they cannot be verified against the provided documentation excerpt alone.",
      "The core claim that HBoxDivider ensures equal heights is consistent with the documentation stating it is 'for laying out axes horizontally, while ensuring that they have equal heights.'"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Almost all claims are verifiable from the stripped code's algorithm and structure; only minor design rationale statements were removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_45",
    "repo": "matplotlib",
    "question": "What is the full class hierarchy and key dependencies of SpanSelector, including how events are connected and selection geometry is managed?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "SpanSelector depends on a four-level class hierarchy: `SpanSelector` \u2192 `_SelectorWidget` \u2192 `AxesWidget` \u2192 `Widget`, all defined in `lib/matplotlib/widgets.py`. Here are the key dependency details:\n\n**Class hierarchy and initialization chain:**\n- `Widget` is the base class with `drawon = True`, `eventson = True`, `_active = True` class attributes and an `active` property.\n- `AxesWidget` adds canvas event connection/disconnection (`connect_event`, `disconnect_events`), cursor setting (`_set_cursor`), and blit background save/load methods. The `canvas` attribute is a property that reads from `self.ax.get_figure(root=True).canvas`.\n- `_SelectorWidget` manages event state (`_eventpress`, `_eventrelease`, `_prev_event`, `_state`), button validation (`validButtons`), and state modifier keys. Its `connect_default_events()` connects 7 canvas events: `motion_notify_event`, `button_press_event`, `button_release_event`, `draw_event`, `key_press_event`, `key_release_event`, and `scroll_event`.\n- `SpanSelector` overrides `connect_default_events()` to add an 8th event (`motion_notify_event` \u2192 `self._hover`) when `interactive=True`, meaning there are two motion_notify callbacks in that case.\n\n**Key implementation details:**\n1. SpanSelector's `__init__` sets `state_modifier_keys` with `square='not-applicable'`, `center='not-applicable'`, and `rotate='not-applicable'`, disabling these states from the parent's defaults of `square='shift'`, `center='control'`, `rotate='r'`. The parent `_SelectorWidget` default for `move` is `' '` (spacebar).\n\n2. The selection shape is a `matplotlib.patches.Rectangle` created in `new_axes()`. For `direction='horizontal'`, it uses `ax.get_xaxis_transform()` with initial dimensions `(w=0, h=1)`. For `direction='vertical'`, it uses `ax.get_yaxis_transform()` with `(w=1, h=0)`.\n\n3. `_snap()` is a `@staticmethod` that computes `eps = np.min(np.abs(np.diff(snap_values))) * 1e-12`, then uses `np.abs(snap_values - v + np.sign(v) * eps).argmin()` to find the closest snap value.\n\n4. The `_call_with_reparented_event` decorator copies the event, sets `guiEvent = None`, reassigns `inaxes`, and uses `self.ax.transData.inverted().transform()` to recalculate `xdata`/`ydata` when `event.inaxes is not self.ax`. It's applied to `_release` and `_hover` on SpanSelector, but NOT to `_press`.\n\n5. Interactive handle lines are `ToolLineHandles`, which creates `axvline`/`axhline` artists. Its `closest()` method transforms handle positions to pixel space via `self.ax.transData.transform()` and computes pixel distance.\n\n6. `_contains` checks containment with `radius=0`: `self._selection_artist.contains(event, radius=0)[0]`.\n\n7. The `extents` property setter calls `_set_extents()` AND sets `_selection_completed = True`. The internal `_set_extents()` does not set `_selection_completed`.\n\n8. When `_active_handle == 'C'` (center), the widget is being dragged.\n\n9. `_SelectorWidget.useblit` is a property that returns `self._useblit and self.canvas.supports_blit`, but the `props['animated']` flag in SpanSelector.__init__ is set based on `self._useblit` alone.\n\n10. SpanSelector shares `_SelectorWidget` as parent with `RectangleSelector`, `LassoSelector`, and `PolygonSelector`.\n\n11. `_SelectorWidget._clean_event` clips event coordinates to axis bounds via `_get_data()`, which uses `np.clip(event.xdata, *self.ax.get_xbound())`.\n\n12. Cursor types come from `backend_tools.Cursors` (an `enum.IntEnum`): `RESIZE_HORIZONTAL` for horizontal spans, `RESIZE_VERTICAL` for vertical, and `POINTER` when disabled.",
    "rubric": [
      "Identifies the four-level class hierarchy: SpanSelector \u2192 _SelectorWidget \u2192 AxesWidget \u2192 Widget",
      "Mentions that _SelectorWidget.connect_default_events() connects 7 canvas events and SpanSelector adds an 8th (motion_notify for _hover) when interactive=True",
      "Explains that SpanSelector overrides state_modifier_keys to set square, center, and rotate as 'not-applicable'",
      "Describes how the selection shape is a Rectangle using get_xaxis_transform (horizontal) or get_yaxis_transform (vertical) with appropriate initial dimensions",
      "Notes that the extents property setter sets _selection_completed = True while _set_extents does not",
      "Mentions that _SelectorWidget shares its parent role with RectangleSelector, LassoSelector, and PolygonSelector",
      "Explains that AxesWidget provides canvas event connection/disconnection and the canvas property reads from ax.get_figure(root=True).canvas",
      "Describes that _call_with_reparented_event decorator is applied to _release and _hover but NOT _press",
      "Notes that interactive handle lines use ToolLineHandles which creates axvline/axhline artists",
      "Mentions that _SelectorWidget._clean_event clips coordinates to axis bounds via np.clip"
    ],
    "key_files": [
      "lib/matplotlib/widgets.py",
      "lib/matplotlib/backend_tools.py"
    ],
    "source_doc": "[docstring: lib/matplotlib/widgets.py] lib.matplotlib.widgets.SpanSelector\nlib.matplotlib.widgets.SpanSelector:\n  Visually select a min/max range on a single axis and call a function with\n    those values.\n\n    To guarantee that the selector remains responsive, keep a reference to it.\n\n    In order to turn off the SpanSelector, set ``span_selector.active`` to\n    False.  To turn it back on, set it to True.\n\n    Press and release events triggered at the same coordinates outside the\n    selection will clear the selector, except when\n    ``ignore_event_outside=True``.\n\n    Parameters\n    ----------\n    ax : `~matplotlib.axes.Axes`\n\n    onselect : callable with signature ``func(min: float, max: float)``\n        A callback function that is called after a release event and the\n        selection is created, changed or removed.\n\n    direction : {\"horizontal\", \"vertical\"}\n        The direction along which to draw the span selector.\n\n    minspan : float, default: 0\n        If selection is less than or equal to *minspan*, the selection is\n        removed (when already existing) or cancelled.\n\n    useblit : bool, default: False\n        If True, use the backend-dependent blitting features for faster\n        canvas updates. See the tutorial :ref:`blitting` for details.\n\n    props : dict, default: {'facecolor': 'red', 'alpha': 0.5}\n        Dictionary of `.Patch` properties.\n\n    onmove_callback : callable with signature ``func(min: float, max: float)``, optional\n        Called on mouse move while the span is being selected.\n\n    interactive : bool, default: False\n        Whether to draw a set of handles that allow interaction with the\n        widget after it is drawn.\n\n    button : `.MouseButton` or list of `.MouseButton`, default: all buttons\n        The mouse buttons which activate the span selector.\n\n    handle_props : dict, default: None\n        Properties of the handle lines at the edges of the span. Only used\n        when *interactive* is True. See `.Line2D` for valid properties.\n\n    grab_range : float, default: 10\n        Distance in pixels within which the interactive tool handles can be activated.\n\n    state_modifier_keys : dict, optional\n        Keyboard modifiers which affect the widget's behavior.  Values\n        amend the defaults, which are:\n\n        - \"clear\": Clear the current shape, default: \"escape\".\n\n    drag_from_anywhere : bool, default: False\n        If `True`, the widget can be moved by clicking anywhere within its bounds.\n\n    ignore_event_outside : bool, default: False\n        If `True`, the event triggered outside the span selector will be ignored.\n\n    snap_values : 1D array-like, optional\n        Snap the selector edges to the given values.\n\n    Examples\n    --------\n    >>> import matplotlib.pyplot as plt\n    >>> import matplotlib.widgets as mwidgets\n    >>> fig, ax = plt.subplots()\n    >>> ax.plot([1, 2, 3], [10, 50, 100])\n    >>> def onselect(vmin, vmax):\n    ...     print(vmin, vmax)\n    >>> span = mwidgets.SpanSelector(ax, onselect, 'horizontal',\n    ...                              props=dict(facecolor='blue', alpha=0.5))\n    >>> fig.show()\n\n    See also: :doc:`/gallery/widgets/span_selector`\n\nlib.matplotlib.widgets.SpanSelector.new_axes:\n  Set SpanSelector to operate on a new Axes.\n\nlib.matplotlib.widgets.SpanSelector._set_span_cursor:\n  Update the canvas cursor based on direction of the selector.\n\nlib.matplotlib.widgets.SpanSelector._press:\n  Button press event handler.\n\nlib.matplotlib.widgets.SpanSelector.direction:\n  Direction of the span selector: 'vertical' or 'horizontal'.\n\nlib.matplotlib.widgets.SpanSelector.direction:\n  Set the direction of the span selector.\n\nlib.matplotlib.widgets.SpanSelector._release:\n  Button release event handler.\n\nlib.matplotlib.widgets.SpanSelector._hover:\n  Update the canvas cursor if it's over a handle.\n\nlib.matplotlib.widgets.SpanSelector._onmove:\n  Motion notify event handler.\n\nlib.matplotlib.widgets.SpanSelector._set_active_handle:\n  Set active handle based on the location of the mouse event.\n\nlib.matplotlib.widgets.SpanSelector._contains:\n  Return True if event is within the patch.\n\nlib.matplotlib.widgets.SpanSelector._snap:\n  Snap values to a given array values (snap_values).\n\nlib.matplotlib.widgets.SpanSelector.extents:\n  (float, float)\n            The values, in data coordinates, for the start and end points of the current\n            selection. If there is no selection then the start and end values will be\n            the same.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in explanatory rationale about transforms, macOS backend requirement, and comment references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "matplotlib_gen_48",
    "repo": "matplotlib",
    "question": "How does the RGBAxes class use axes_divider to create its 4-panel layout and style the axes with white borders?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `RGBAxes` class in `lib/mpl_toolkits/axes_grid1/axes_rgb.py` creates a 4-panel image display layout using the `axes_divider` infrastructure. Here's how it works:\n\n**Initialization (`__init__`)**:\n1. It pops `axes_class` from kwargs (defaulting to `self._defaultAxesClass`, which is `mpl_toolkits.axes_grid1.mpl_axes.Axes`).\n2. Creates the main RGB axes by instantiating `axes_class(*args, **kwargs)` and adds it to the figure via `ax.get_figure().add_axes(ax)`.\n3. Delegates creation of R, G, B small axes to `make_rgb_axes(ax, pad=pad, axes_class=axes_class, **kwargs)`.\n4. For all four axes, it sets spine line colors to white (`\"w\"`) and tick mark edge colors to white, using the `AxisDict` and `SimpleChainedObjects` mechanism.\n\n**Layout via `make_rgb_axes`**:\n- Uses `make_axes_locatable(ax)` which creates an `AxesDivider` instance.\n- Horizontal layout has 3 elements: `[Size.AxesX(ax), pad_size, xsize]` where `xsize = ((1-2*pad)/3) * Size.AxesX(ax)` \u2014 so each small axes is 1/3 of (1-2*pad) times the main axes width.\n- Vertical layout has 5 elements: `[ysize, pad_size, ysize, pad_size, ysize]` where `ysize = ((1-2*pad)/3) * Size.AxesY(ax)` \u2014 three equal height small axes separated by padding.\n- The main RGB axes spans all vertical rows via `divider.new_locator(0, 0, ny1=-1)` (ny1=-1 is used as an index in `_locate`, meaning it spans to the last element).\n- R, G, B axes are placed at `nx=2` (the third column) at `ny=4, 2, 0` respectively (top to bottom: R, G, B).\n- All three small axes share both x and y axes with the main axes (`sharex=ax, sharey=ax`).\n\n**Axis handling distinction**:\nIn `__init__`, the code checks `isinstance(ax1.axis, MethodType)` to determine the axes type:\n- For standard `matplotlib.axes.Axes`, `.axis` is a method (bound `MethodType`), so the code creates a new `Axes.AxisDict` and populates it with `SimpleAxisArtist` wrappers for each spine.\n- For `mpl_toolkits.axes_grid1.mpl_axes.Axes`, `.axis` is a `@property` returning an `AxisDict` (set up in `clear()` as `self._axislines`), so it's used directly.\n\nThe `ad[:].line.set_color(\"w\")` call works because `AxisDict.__getitem__` with a slice `[:]` returns a `SimpleChainedObjects` wrapping all values, and attribute access chains through to all contained objects.\n\n**`imshow_rgb` method**:\n1. Validates that r, g, b arrays have identical shapes (raises `ValueError` with shape info if not).\n2. Stacks them into `RGB = np.dstack([r, g, b])`.\n3. Creates individual channel images as 3-channel arrays with zeros in non-relevant channels (e.g., `R[:, :, 0] = r` with zeros elsewhere), preserving the color appearance.\n4. Calls `imshow` on each of the four axes and returns all four results.\n\n**Tick label hiding in `make_rgb_axes`**:\nBoth standard tick labels and `SimpleAxisArtist.major_ticklabels` are hidden on the R, G, B axes. The code uses a try/except for the `axis.values()` approach since standard matplotlib Axes won't have that interface.",
    "rubric": [
      "Explains that make_axes_locatable creates an AxesDivider from the main axes",
      "Describes the horizontal layout: main axes width, a pad, and xsize = ((1-2*pad)/3) * AxesX(ax)",
      "Describes the vertical layout: 5 elements (3 ysize slots separated by 2 pads), where ysize = ((1-2*pad)/3) * AxesY(ax)",
      "Explains that the main RGB axes spans all vertical rows via ny1=-1 in new_locator",
      "Explains that R, G, B axes are placed at nx=2 (second column) and ny=4, 2, 0 (top to bottom)",
      "Mentions that R, G, B axes share x and y with the main axes (sharex=ax, sharey=ax)",
      "Explains the MethodType check on ax1.axis to distinguish standard matplotlib axes from mpl_toolkits axes",
      "Describes how AxisDict[:] returns a SimpleChainedObjects that propagates attribute access/calls to all contained axis artists",
      "Mentions that spine line colors and tick mark edge colors are set to white ('w') on all four axes"
    ],
    "key_files": [
      "lib/mpl_toolkits/axes_grid1/axes_rgb.py",
      "lib/mpl_toolkits/axes_grid1/mpl_axes.py",
      "lib/mpl_toolkits/axes_grid1/axes_divider.py",
      "lib/mpl_toolkits/axes_grid1/axes_size.py"
    ],
    "source_doc": "[docstring: lib/mpl_toolkits/axes_grid1/axes_rgb.py] lib.mpl_toolkits.axes_grid1.axes_rgb.RGBAxes\nlib.mpl_toolkits.axes_grid1.axes_rgb.RGBAxes:\n  4-panel `~.Axes.imshow` (RGB, R, G, B).\n\n    Layout::\n\n        \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2510\n        \u2502               \u2502  R  \u2502\n        \u2502               \u251c\u2500\u2500\u2500\u2500\u2500\u2524\n        \u2502      RGB      \u2502  G  \u2502\n        \u2502               \u251c\u2500\u2500\u2500\u2500\u2500\u2524\n        \u2502               \u2502  B  \u2502\n        \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2518\n\n    Subclasses can override the ``_defaultAxesClass`` attribute.\n    By default RGBAxes uses `.mpl_axes.Axes`.\n\n    Attributes\n    ----------\n    RGB : ``_defaultAxesClass``\n        The Axes object for the three-channel `~.Axes.imshow`.\n    R : ``_defaultAxesClass``\n        The Axes object for the red channel `~.Axes.imshow`.\n    G : ``_defaultAxesClass``\n        The Axes object for the green channel `~.Axes.imshow`.\n    B : ``_defaultAxesClass``\n        The Axes object for the blue channel `~.Axes.imshow`.\n\nlib.mpl_toolkits.axes_grid1.axes_rgb.RGBAxes.__init__:\n  Parameters\n        ----------\n        pad : float, default: 0\n            Fraction of the Axes height to put as padding.\n        axes_class : `~matplotlib.axes.Axes`\n            Axes class to use. If not provided, ``_defaultAxesClass`` is used.\n        *args\n            Forwarded to *axes_class* init for the RGB Axes\n        **kwargs\n            Forwarded to *axes_class* init for the RGB, R, G, and B Axes\n\nlib.mpl_toolkits.axes_grid1.axes_rgb.RGBAxes.imshow_rgb:\n  Create the four images {rgb, r, g, b}.\n\n        Parameters\n        ----------\n        r, g, b : array-like\n            The red, green, and blue arrays.\n        **kwargs\n            Forwarded to `~.Axes.imshow` calls for the four images.\n\n        Returns\n        -------\n        rgb : `~matplotlib.image.AxesImage`\n        r : `~matplotlib.image.AxesImage`\n        g : `~matplotlib.image.AxesImage`\n        b : `~matplotlib.image.AxesImage`",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped source code; only minor wording adjustments needed for two claims.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_03",
    "repo": "sympy",
    "question": "Where does `ipartfrac` get called, and how does its data flow through `migcdex`, `igcdex`, and `gcdext`?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `ipartfrac` function in `sympy/functions/elementary/_trigonometric_special.py` flows data through the following call chain:\n\n**Where `ipartfrac` is called:** It is called in `cos._eval_rewrite_as_sqrt()` in `sympy/functions/elementary/trigonometric.py`. This method is part of the `cos` class's ability to rewrite cosine expressions as square root expressions. The `sin` class's `_eval_rewrite_as_sqrt` does NOT call `ipartfrac` directly \u2014 instead it delegates through `self.rewrite(cos, **kwargs).rewrite(sqrt, **kwargs)`, so `ipartfrac` is invoked indirectly.\n\n**What `ipartfrac` calls internally:**\n1. It uses `functools.reduce` with a locally-defined `mul` function to compute the total product of denominators.\n2. It builds a list `a` of complements `denom // x` for each denominator.\n3. It calls `migcdex(*a)` and discards the gcd value (stored as `_`), returning only the tuple of coefficients `h`.\n\n**The `migcdex` function** recursively reduces the multi-argument extended GCD to two-argument cases using `igcdex` from `sympy.core.intfunc`. For the base case of 2 arguments, it calls `igcdex(x[0], x[1])` which returns `(u, v, h)`. For more than 2 arguments, it recurses on `x[1:]`, then combines the result with `x[0]` via another `igcdex` call, scaling the recursive coefficients by `v`.\n\n**The `igcdex` function** (in `sympy/core/intfunc.py`) is a thin wrapper that calls `gcdext` from `sympy.external.gmpy` and reorders the return values: `gcdext` returns `(g, x, y)` but `igcdex` returns `(x, y, g)`.\n\n**The `gcdext` function** (in `sympy/external/ntheory.py`) is a pure Python implementation of the extended Euclidean algorithm using an iterative approach with variables `x, r, y, s` tracking coefficients through the division loop.\n\n**How the result of `ipartfrac` is used in `cos._eval_rewrite_as_sqrt`:** The returned numerator tuple is zipped with the denominators to create rational partial fractions. These are multiplied by `pi_coeff.p` and combined with `numbered_symbols('z')` to form substitution pairs. The expression `cos(sum of symbolic angles)._eval_expand_trig().subs(substitutions)` decomposes the cosine into a sum-of-angles formula, where each angle is a rational multiple of \u03c0 that can be evaluated. If `fermat_coords` found Fermat prime factors (checked via `FC`), and there are more than one, the result is recursively rewritten as sqrt; otherwise, the partially expanded form `pcls` is returned directly.\n\n**Where the denominators come from:** If the denominator of the pi coefficient is factorable into distinct Fermat primes (3, 5, 17, 257, 65537 \u2014 checked by `fermat_coords`), those primes are used as denominators. Otherwise, `factorint(pi_coeff.q)` provides prime-power factors as `[b**e for b, e in factorint(...).items()]`.",
    "rubric": [
      "ipartfrac is called in cos._eval_rewrite_as_sqrt (not directly in sin's version, which delegates through cos rewrite)",
      "ipartfrac uses functools.reduce with a mul function to compute the product of all denominators",
      "ipartfrac builds complement values (denom // x for each x) and passes them to migcdex",
      "migcdex recursively reduces multi-argument extended GCD to two-argument igcdex calls, scaling coefficients by v",
      "igcdex is a wrapper around gcdext that reorders return values from (g, x, y) to (x, y, g)",
      "gcdext in sympy/external/ntheory.py implements the extended Euclidean algorithm iteratively",
      "The result of ipartfrac is used to create rational partial fractions that decompose the cosine via sum-of-angles expansion",
      "Denominators come from Fermat prime factors (via fermat_coords) or from factorint of the pi coefficient's denominator"
    ],
    "key_files": [
      "sympy/functions/elementary/_trigonometric_special.py",
      "sympy/functions/elementary/trigonometric.py",
      "sympy/core/intfunc.py",
      "sympy/external/ntheory.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/_trigonometric_special.py] sympy.functions.elementary._trigonometric_special.ipartfrac\nCompute the partial fraction decomposition.\n\n    Explanation\n    ===========\n\n    Given a rational number $\\frac{1}{q_1 \\cdots q_n}$ where all\n    $q_1, \\cdots, q_n$ are pairwise coprime,\n\n    A partial fraction decomposition is defined as\n\n    .. math::\n        \\frac{1}{q_1 \\cdots q_n} = \\frac{p_1}{q_1} + \\cdots + \\frac{p_n}{q_n}\n\n    And it can be derived from solving the following diophantine equation for\n    the $p_1, \\cdots, p_n$\n\n    .. math::\n        1 = p_1 \\prod_{i \\ne 1}q_i + \\cdots + p_n \\prod_{i \\ne n}q_i\n\n    Where $q_1, \\cdots, q_n$ being pairwise coprime implies\n    $\\gcd(\\prod_{i \\ne 1}q_i, \\cdots, \\prod_{i \\ne n}q_i) = 1$,\n    which guarantees the existence of the solution.\n\n    It is sufficient to compute partial fraction decomposition only\n    for numerator $1$ because partial fraction decomposition for any\n    $\\frac{n}{q_1 \\cdots q_n}$ can be easily computed by multiplying\n    the result by $n$ afterwards.\n\n    Parameters\n    ==========\n\n    denoms : int\n        The pairwise coprime integer denominators $q_i$ which defines the\n        rational number $\\frac{1}{q_1 \\cdots q_n}$\n\n    Returns\n    =======\n\n    tuple[int, ...]\n        The list of numerators which semantically corresponds to $p_i$ of the\n        partial fraction decomposition\n        $\\frac{1}{q_1 \\cdots q_n} = \\frac{p_1}{q_1} + \\cdots + \\frac{p_n}{q_n}$\n\n    Examples\n    ========\n\n    >>> from sympy import Rational, Mul\n    >>> from sympy.functions.elementary._trigonometric_special import ipartfrac\n\n    >>> denoms = 2, 3, 5\n    >>> numers = ipartfrac(2, 3, 5)\n    >>> numers\n    (1, 7, -14)\n\n    >>> Rational(1, Mul(*denoms))\n    1/30\n    >>> out = 0\n    >>> for n, d in zip(numers, denoms):\n    ...    out += Rational(n, d)\n    >>> out\n    1/30",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only line number references needed removal as they cannot be verified from stripped code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_02",
    "repo": "sympy",
    "question": "Why does `udivisor_sigma` use a single formula for all k values while `divisor_sigma` needs three separate code paths?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `udivisor_sigma` class uses a single unified formula `Mul(*[1+p**(k*e) for p, e in factorint(n).items()])` for all values of k, unlike the related `divisor_sigma` class which has three separate code paths (k=0, k is Integer, k is symbolic non-zero). This design difference exists because of a mathematical property of the formulas:\n\n**For `udivisor_sigma`**, the formula `1 + p^(k\u00b7e)` is always well-defined for all non-negative k\u2014when k=0, it simply becomes `1 + p^0 = 2`, which correctly yields `2^\u03c9(n)` (the unitary divisor count). No special casing is needed.\n\n**For `divisor_sigma`**, the geometric series formula `(p^(k(e+1)) - 1) / (p^k - 1)` produces a 0/0 indeterminate form when k=0, so the code must branch: it uses `Mul(*[e + 1 ...])` when k=0 (line 1726), delegates to the pure-Python `_divisor_sigma()` helper when both n and k are concrete integers for efficiency (line 1728), and uses `cancel()` to simplify the division when k is symbolic and non-zero (line 1730).\n\nAdditionally, `udivisor_sigma` does NOT have an `n is S.One` special case (unlike `divisor_sigma` which checks `if n is S.One: return S.One` at line 1722). This is because `factorint(1)` returns an empty dict, and `Mul(*[])` equals 1 in SymPy, so the general formula handles n=1 naturally.\n\nThere is also no `_udivisor_sigma` pure-Python helper function (analogous to `_divisor_sigma` in `ntheory/factor_.py`). The `divisor_sigma` class delegates to `_divisor_sigma` (which uses `math.prod` and integer division `//` for performance on concrete integers), but `udivisor_sigma` always constructs its result through SymPy's `Mul` with symbolic expressions. This means `udivisor_sigma` can handle symbolic k even when n is a concrete integer, while `divisor_sigma` only handles symbolic k through its `cancel()` path.\n\nOther design details:\n- Both classes inherit from `DefinedFunction` (in `sympy/core/function.py`), which inherits from `Function` and applies the `@cacheit` decorator on `__new__`, caching results.\n- Both set class attributes `is_integer = True` and `is_positive = True` as assumptions.\n- Both share identical validation: checking `n.is_integer`, `n.is_positive`, `k.is_integer`, `k.is_nonnegative`, and a prime shortcut returning `1 + n**k`.\n- The old location `sympy.ntheory.factor_.udivisor_sigma` was deprecated since version 1.13 and now simply imports and delegates to the combinatorial numbers version.\n- LaTeX printing uses `\\sigma^*` for `udivisor_sigma` versus `\\sigma` for `divisor_sigma`.",
    "rubric": [
      "Explains that udivisor_sigma's formula `1 + p^(k*e)` naturally handles k=0 (becomes 1 + 1 = 2) without indeterminate forms",
      "Explains that divisor_sigma's geometric series formula `(p^(k(e+1)) - 1)/(p^k - 1)` produces a 0/0 indeterminate form when k=0, requiring a separate path using `e + 1`",
      "Identifies that divisor_sigma has a concrete-integer optimization path delegating to `_divisor_sigma` (using integer arithmetic via math.prod and //), while udivisor_sigma has no such helper",
      "Identifies that divisor_sigma uses `cancel()` for the symbolic non-zero k case to handle the division simplification",
      "Notes that udivisor_sigma doesn't need a special n=1 check because factorint(1) returns an empty dict and Mul(*[]) naturally evaluates to 1"
    ],
    "key_files": [
      "sympy/functions/combinatorial/numbers.py",
      "sympy/ntheory/factor_.py",
      "sympy/core/function.py",
      "sympy/printing/latex.py"
    ],
    "source_doc": "[docstring: sympy/functions/combinatorial/numbers.py] sympy.functions.combinatorial.numbers.udivisor_sigma\nsympy.functions.combinatorial.numbers.udivisor_sigma:\n  Calculate the unitary divisor function `\\sigma_k^*(n)` for positive integer n\n\n    ``udivisor_sigma(n, k)`` is equal to ``sum([x**k for x in udivisors(n)])``\n\n    If n's prime factorization is:\n\n    .. math ::\n        n = \\prod_{i=1}^\\omega p_i^{m_i},\n\n    then\n\n    .. math ::\n        \\sigma_k^*(n) = \\prod_{i=1}^\\omega (1+ p_i^{m_ik}).\n\n    Parameters\n    ==========\n\n    k : power of divisors in the sum\n\n        for k = 0, 1:\n        ``udivisor_sigma(n, 0)`` is equal to ``udivisor_count(n)``\n        ``udivisor_sigma(n, 1)`` is equal to ``sum(udivisors(n))``\n\n        Default for k is 1.\n\n    Examples\n    ========\n\n    >>> from sympy.functions.combinatorial.numbers import udivisor_sigma\n    >>> udivisor_sigma(18, 0)\n    4\n    >>> udivisor_sigma(74, 1)\n    114\n    >>> udivisor_sigma(36, 3)\n    47450\n    >>> udivisor_sigma(111)\n    152\n\n    See Also\n    ========\n\n    sympy.ntheory.factor_.divisor_count, totient, sympy.ntheory.factor_.divisors,\n    sympy.ntheory.factor_.udivisors, sympy.ntheory.factor_.udivisor_count, divisor_sigma,\n    sympy.ntheory.factor_.factorint\n\n    References\n    ==========\n\n    .. [1] https://mathworld.wolfram.com/UnitaryDivisorFunction.html",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of claims in the answer are about implementation details (code paths, line numbers, class hierarchy, helper functions, caching, LaTeX printing, deprecation history) that are not present in the provided documentation. These cannot be verified from the doc alone.",
      "The mathematical claims about the formula \u03c3_k*(n) = \u220f(1 + p^(k\u00b7e)) are consistent with the documentation's formula \u220f(1 + p_i^{m_i\u00b7k}).",
      "The claim that udivisor_sigma(n, 0) equals 2^\u03c9(n) is consistent with the documentation stating udivisor_sigma(n, 0) equals udivisor_count(n), and the example udivisor_sigma(18, 0) = 4 (18 = 2\u00b73\u00b2, \u03c9=2, 2\u00b2=4).",
      "The claim about default k=1 is consistent with the documentation ('Default for k is 1').",
      "The specific line numbers (1722, 1726, 1728, 1730) and implementation details about divisor_sigma's code paths, _divisor_sigma helper, DefinedFunction inheritance, @cacheit decorator, etc. are not verifiable from this documentation and could be hallucinated or inaccurate.",
      "The answer makes many specific claims about code structure that go far beyond what the documentation describes, but none directly contradict the documentation."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code through examining class definitions, imports, control flow, and formula logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_04",
    "repo": "sympy",
    "question": "How does the `re` class's `eval` method decompose a complex expression into real parts, and how does it avoid infinite recursion?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `re` class in `sympy/functions/elementary/complexes.py` implements real part extraction through a multi-stage evaluation algorithm in its `eval` classmethod. Here is how it works in detail:\n\n**Class hierarchy:** `re` inherits from `DefinedFunction` (defined in `sympy/core/function.py`), which inherits from `Function`, which inherits from `Application` and `Expr`. The `DefinedFunction.__new__` method (decorated with `@cacheit`) delegates to `cls._new_()` defined on `Function`. The `Application.__new__` method calls `cls.eval(*args)` \u2014 if `eval` returns non-None, that result is returned directly; otherwise an unevaluated `re(arg)` instance is created.\n\n**Class-level attributes:** `re` sets `is_extended_real = True`, `unbranched = True`, and `_singularities = True`.\n\n**Evaluation algorithm in `re.eval(cls, arg)`:**\n\n1. **Special values:** If arg is `S.NaN` or `S.ComplexInfinity`, returns `S.NaN`.\n\n2. **Purely real:** If `arg.is_extended_real` is True, returns `arg` unchanged.\n\n3. **Purely imaginary:** If `arg.is_imaginary` or `(I*arg).is_extended_real`, returns `S.Zero`.\n\n4. **Matrix support:** If `arg.is_Matrix`, delegates to `arg.as_real_imag()[0]`.\n\n5. **Conjugate unwrapping:** If arg is a `conjugate` function instance (checked via `arg.is_Function and isinstance(arg, conjugate)`), returns `re(arg.args[0])`.\n\n6. **Additive decomposition (the main algorithm):** Splits the argument into additive terms using `Add.make_args(arg)`. Each term is classified into three lists:\n   - **`included`**: Terms that remain wrapped in `re()`.\n   - **`reverted`**: Terms of the form `coeff*I` where `coeff` is NOT extended real (partially known). For these, the real part is computed as `-im(coeff)` in the final result.\n   - **`excluded`**: Terms that are known-real and I-free, OR terms whose `as_real_imag()` expansion succeeded. These contribute directly to the real part.\n\n   For each term:\n   - `term.as_coefficient(I)` extracts the coefficient if the term is exactly `coeff * I`. If coeff is extended real, the term is dropped (not added to any list). If coeff is not extended real, it goes to `reverted`.\n   - If the term has no `I` and is extended real, it goes directly to `excluded`.\n   - Otherwise, `term.as_real_imag(ignore=arg)` is called. The `ignore` parameter serves as an anti-recursion mechanism: in `Symbol.as_real_imag`, if `hints.get('ignore') == self`, it returns `None` instead of calling `re(self)` again.\n\n   The final result is `cls(a) - im(b) + c` where `a` = sum of `included`, `b` = sum of `reverted`, `c` = sum of `excluded`. This is only returned when `len(args) != len(included)` (i.e., at least some progress was made).\n\n**Symmetry with `im`:** The `im.eval` method uses the same three-list decomposition pattern, but with key differences: for terms of the form `coeff*I` with real `coeff`, `im` puts `coeff` in `excluded`. Its final expression is `cls(a) + re(b) + c` (using `re(b)` instead of `-im(b)`).\n\n**Additional methods on `re`:**\n- `as_real_imag()` returns `(self, S.Zero)`.\n- `_eval_derivative(x)`: If x is real or arg is real, returns `re(d(arg)/dx)`. If x or arg is imaginary, returns `-I * im(d(arg)/dx)`.\n- `_eval_rewrite_as_im(arg)` returns `arg - I*im(arg)`.\n- `_eval_is_zero()` uses `fuzzy_or([self.args[0].is_imaginary, self.args[0].is_zero])`.\n- `_eval_is_algebraic()` returns `self.args[0].is_algebraic`.\n- `_eval_is_finite()` and `_eval_is_complex()` both return `True` when `self.args[0].is_finite`.",
    "rubric": [
      "Mentions that `re` inherits from `DefinedFunction` (and ultimately from `Function`/`Application`) and that `Application.__new__` calls `cls.eval()`, returning the result if non-None or creating an unevaluated instance otherwise.",
      "Identifies that special values (NaN, ComplexInfinity) return NaN, purely real args return arg directly, and purely imaginary args return S.Zero.",
      "Describes the three-list decomposition (included, reverted, excluded) applied to additive terms via `Add.make_args(arg)`.",
      "Explains how `as_coefficient(I)` classifies terms: real coefficient of I is dropped, non-real coefficient goes to `reverted`, terms with no I that are extended_real go to `excluded`.",
      "Explains the anti-recursion mechanism: `term.as_real_imag(ignore=arg)` is called, and the `ignore` parameter prevents re-entering `re()` for the same argument.",
      "States the final result formula: `cls(a) - im(b) + c` where a=included, b=reverted, c=excluded, only returned when progress was made (len(args) != len(included)).",
      "Mentions the symmetry/contrast with `im.eval`, which uses `cls(a) + re(b) + c` and puts real coefficients of I into `excluded` instead of dropping them."
    ],
    "key_files": [
      "sympy/functions/elementary/complexes.py",
      "sympy/core/function.py",
      "sympy/core/expr.py",
      "sympy/core/symbol.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/complexes.py] sympy.functions.elementary.complexes.re\nsympy.functions.elementary.complexes.re:\n  Returns real part of expression. This function performs only\n    elementary analysis and so it will fail to decompose properly\n    more complicated expressions. If completely simplified result\n    is needed then use ``Basic.as_real_imag()`` or perform complex\n    expansion on instance of this function.\n\n    Examples\n    ========\n\n    >>> from sympy import re, im, I, E, symbols\n    >>> x, y = symbols('x y', real=True)\n    >>> re(2*E)\n    2*E\n    >>> re(2*I + 17)\n    17\n    >>> re(2*I)\n    0\n    >>> re(im(x) + x*I + 2)\n    2\n    >>> re(5 + I + 2)\n    7\n\n    Parameters\n    ==========\n\n    arg : Expr\n        Real or complex expression.\n\n    Returns\n    =======\n\n    expr : Expr\n        Real part of expression.\n\n    See Also\n    ========\n\n    im\n\nsympy.functions.elementary.complexes.re.as_real_imag:\n  Returns the real number with a zero imaginary part.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in mathematical explanations and line number references; core algorithmic claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_01",
    "repo": "sympy",
    "question": "What is the full class hierarchy, memoization scheme, and computation strategy of the `bernoulli` function class?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `bernoulli` class in `sympy/functions/combinatorial/numbers.py` is a subclass of `DefinedFunction`, which itself is defined in `sympy/core/function.py` as a subclass of `Function`. The `DefinedFunction` class has a very minimal implementation \u2014 it overrides `__new__` with a `@cacheit` decorator that delegates to `cls._new_()`, which is inherited from `Function`. This caching means that repeated calls with the same arguments return the same object.\n\n**Architecture and class hierarchy:**\n- `bernoulli` \u2192 `DefinedFunction` \u2192 `Function` \u2192 `Application` (with `FunctionClass` metaclass) + `Expr`\n- The `Application.__new__` method calls `cls.eval(*args)`, and if `eval` returns a non-None value, that value is returned directly instead of creating a `bernoulli` instance.\n- The `nargs` for `bernoulli` is automatically determined from the `eval(cls, n, x=None)` signature via the `arity()` function in `function.py`, resulting in `nargs = (1, 2)` \u2014 meaning it accepts 1 or 2 arguments.\n\n**Memoization scheme:**\nThe class implements a custom modular memoization with two class-level dictionaries:\n- `_cache = {0: S.One, 1: Rational(1, 2), 2: Rational(1, 6), 4: Rational(-1, 30)}`\n- `_highest = {0: 0, 1: 1, 2: 2, 4: 4}` \u2014 tracks the highest computed index for each residue class mod 6, as seen from how `eval` uses `case = n % 6` and `highest_cached = cls._highest[case]`\n\nKeys in `_highest` are only 0, 1, 2, and 4 (not 3 or 5) because the code returns `S.Zero` for odd n > 1 (`n.is_odd and (n-1).is_positive`), and the computation loop increments in steps of 6 (`range(highest_cached + 6, n + 6, 6)`).\n\n**Computation strategies:**\n1. For n > 500, it uses `bernfrac(n)` (from `sympy.external.mpmath`) which returns a (p, q) pair that gets converted to `Rational(int(p), int(q))`.\n2. For n \u2264 500, it uses `_calc_bernoulli(n)`. The binomial coefficients in the summation are updated incrementally: instead of recomputing `binomial(n+3, n-6k)` from scratch, it multiplies/divides by products of consecutive integers via the `_product(a, b)` helper (which calls `math.prod(range(a, b+1))`).\n3. For Bernoulli polynomials (when x is provided and n is a number), it delegates to `bernoulli_poly(n, x)` from `sympy.polys.appellseqs`, which calls `named_poly(n, dup_bernoulli, QQ, ...)`. The `dup_bernoulli` function builds the polynomial iteratively starting from `p = [K.one, K(-1,2)]`, integrating at each step and adjusting the constant term at each even step.\n\n**Numerical evaluation:**\nThe `_eval_evalf(self, prec)` method handles four cases:\n- n == 0: returns mpf(1)\n- n == 1: returns `x - 0.5` (using `mp.fsub`)\n- Non-negative integer n with x == 1: uses `mp.bernoulli(n)`; with x \u2260 1: uses `mp.bernpoly(n, x)`\n- General complex s: computes `-s * zeta(1-s, x)` using mpmath's zeta function\n\nThe `local_workprec` context manager used in `_eval_evalf` is a custom SymPy class (defined in `sympy/external/mpmath.py`) that maintains a pool of `MPContext` objects (`_contexts: list[MPContext] = []`), creating or reusing separate contexts rather than modifying the global mpmath precision.\n\n**Rewrite capability:**\n`_eval_rewrite_as_zeta` rewrites as `Piecewise((1, Eq(n, 0)), (-n * zeta(1-n, x), True))`, importing the `zeta` class from `sympy.functions.special.zeta_functions`.",
    "rubric": [
      "Identifies the inheritance chain: bernoulli \u2192 DefinedFunction \u2192 Function \u2192 Application (with FunctionClass metaclass)",
      "Explains that DefinedFunction.__new__ uses @cacheit and delegates to cls._new_()",
      "Describes the modular memoization with _cache and _highest dictionaries keyed by residue classes mod 6 (keys 0, 1, 2, 4)",
      "Explains why keys 3 and 5 are absent: odd Bernoulli numbers > 1 are zero, so they are returned early without caching",
      "Describes that for n > 500, bernfrac from mpmath is used, returning a (p, q) pair converted to Rational",
      "Describes that for n \u2264 500, _calc_bernoulli uses Ramanujan's formula with incremental binomial coefficient updates via _product helper",
      "Explains that eval accepts 1 or 2 arguments (nargs = (1, 2)) and returns Bernoulli polynomials via bernoulli_poly when x is provided",
      "Describes _eval_evalf handling: uses mp.bernoulli/mp.bernpoly for integer n, and -s*zeta(1-s,x) for general complex s",
      "Mentions that local_workprec uses a pool of MPContext objects rather than modifying global precision",
      "Mentions the _eval_rewrite_as_zeta method that rewrites as Piecewise((1, Eq(n,0)), (-n*zeta(1-n,x), True))"
    ],
    "key_files": [
      "sympy/functions/combinatorial/numbers.py",
      "sympy/core/function.py",
      "sympy/polys/appellseqs.py",
      "sympy/external/mpmath.py"
    ],
    "source_doc": "[docstring: sympy/functions/combinatorial/numbers.py] sympy.functions.combinatorial.numbers.bernoulli\nsympy.functions.combinatorial.numbers.bernoulli:\n  Bernoulli numbers / Bernoulli polynomials / Bernoulli function\n\n    The Bernoulli numbers are a sequence of rational numbers\n    defined by `B_0 = 1` and the recursive relation (`n > 0`):\n\n    .. math :: n+1 = \\sum_{k=0}^n \\binom{n+1}{k} B_k\n\n    They are also commonly defined by their exponential generating\n    function, which is `\\frac{x}{1 - e^{-x}}`. For odd indices > 1,\n    the Bernoulli numbers are zero.\n\n    The Bernoulli polynomials satisfy the analogous formula:\n\n    .. math :: B_n(x) = \\sum_{k=0}^n (-1)^k \\binom{n}{k} B_k x^{n-k}\n\n    Bernoulli numbers and Bernoulli polynomials are related as\n    `B_n(1) = B_n`.\n\n    The generalized Bernoulli function `\\operatorname{B}(s, a)`\n    is defined for any complex `s` and `a`, except where `a` is a\n    nonpositive integer and `s` is not a nonnegative integer. It is\n    an entire function of `s` for fixed `a`, related to the Hurwitz\n    zeta function by\n\n    .. math:: \\operatorname{B}(s, a) = \\begin{cases}\n              -s \\zeta(1-s, a) & s \\ne 0 \\\\ 1 & s = 0 \\end{cases}\n\n    When `s` is a nonnegative integer this function reduces to the\n    Bernoulli polynomials: `\\operatorname{B}(n, x) = B_n(x)`. When\n    `a` is omitted it is assumed to be 1, yielding the (ordinary)\n    Bernoulli function which interpolates the Bernoulli numbers and is\n    related to the Riemann zeta function.\n\n    We compute Bernoulli numbers using Ramanujan's formula:\n\n    .. math :: B_n = \\frac{A(n) - S(n)}{\\binom{n+3}{n}}\n\n    where:\n\n    .. math :: A(n) = \\begin{cases} \\frac{n+3}{3} &\n        n \\equiv 0\\ \\text{or}\\ 2 \\pmod{6} \\\\\n        -\\frac{n+3}{6} & n \\equiv 4 \\pmod{6} \\end{cases}\n\n    and:\n\n    .. math :: S(n) = \\sum_{k=1}^{[n/6]} \\binom{n+3}{n-6k} B_{n-6k}\n\n    This formula is similar to the sum given in the definition, but\n    cuts `\\frac{2}{3}` of the terms. For Bernoulli polynomials, we use\n    Appell sequences.\n\n    For `n` a nonnegative integer and `s`, `a`, `x` arbitrary complex numbers,\n\n    * ``bernoulli(n)`` gives the nth Bernoulli number, `B_n`\n    * ``bernoulli(s)`` gives the Bernoulli function `\\operatorname{B}(s)`\n    * ``bernoulli(n, x)`` gives the nth Bernoulli polynomial in `x`, `B_n(x)`\n    * ``bernoulli(s, a)`` gives the generalized Bernoulli function\n      `\\operatorname{B}(s, a)`\n\n    .. versionchanged:: 1.12\n        ``bernoulli(1)`` gives `+\\frac{1}{2}` instead of `-\\frac{1}{2}`.\n        This choice of value confers several theoretical advantages [5]_,\n        including the extension to complex parameters described above\n        which this function now implements. The previous behavior, defined\n        only for nonnegative integers `n`, can be obtained with\n        ``(-1)**n*bernoulli(n)``.\n\n    Examples\n    ========\n\n    >>> from sympy import bernoulli\n    >>> from sympy.abc import x\n    >>> [bernoulli(n) for n in range(11)]\n    [1, 1/2, 1/6, 0, -1/30, 0, 1/42, 0, -1/30, 0, 5/66]\n    >>> bernoulli(1000001)\n    0\n    >>> bernoulli(3, x)\n    x**3 - 3*x**2/2 + x/2\n\n    See Also\n    ========\n\n    andre, bell, catalan, euler, fibonacci, harmonic, lucas, genocchi,\n    partition, tribonacci, sympy.polys.appellseqs.bernoulli_poly\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Bernoulli_number\n    .. [2] https://en.wikipedia.org/wiki/Bernoulli_polynomial\n    .. [3] https://mathworld.wolfram.com/BernoulliNumber.html\n    .. [4] https://mathworld.wolfram.com/BernoulliPolynomial.html\n    .. [5] Peter Luschny, \"The Bernoulli Manifesto\",\n           https://luschny.de/math/zeta/The-Bernoulli-Manifesto.html\n    .. [6] Peter Luschny, \"An introduction to the Bernoulli function\",\n           https://arxiv.org/abs/2009.06743",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed claims about genocchi class (truncated) and softened a few mathematical naming references (e.g., 'Ramanujan's formula', 'Appell sequence').",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_06",
    "repo": "sympy",
    "question": "Why does SymPy's `exp` class use a custom metaclass with `__instancecheck__`, and how does the dual representation of exponentials work internally?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `exp` class in SymPy uses the `ExpMeta` metaclass (which inherits from `FunctionClass`) to implement a custom `__instancecheck__` method. This allows `isinstance(x, exp)` to return `True` not only for actual `exp` function instances but also for `Pow` objects where the base is `S.Exp1` (Euler's number). SymPy has two internal representations of the exponential function \u2014 `exp(x)` as a Function instance, and `E**x` as a `Pow(S.Exp1, x)` instance \u2014 as evidenced by the `exp_is_pow` flag and the `ExpMeta.__instancecheck__` logic.\n\nThe `exp` class inherits from `ExpBase`, which in turn inherits from `DefinedFunction` (a subclass of `Function`). `DefinedFunction.__new__` is decorated with `@cacheit`, which means that creating `exp` objects is cached.\n\nThe `global_parameters.exp_is_pow` flag (defaulting to `False`) controls which representation is used: when `True`, `exp.eval()` returns `Pow(S.Exp1, arg)` instead of an `exp` object.\n\nThe `exp.eval()` classmethod contains careful logic to simplify expressions. It uses `logcombine` from `sympy.simplify.simplify` when the argument is a `Mul` to detect `exp(c*log(x))` patterns and simplify them to `x**c`. When the argument is of the form `coeff*pi*I` and `2*coeff` is an integer, it evaluates to `1`, `-1`, `I`, or `-I`. For rational multiples of `pi*I`, it normalizes the coefficient to the interval `(-pi, pi]` by reducing mod 2.\n\nThe `ExpBase` class sets `unbranched = True` and `_singularities = (S.ComplexInfinity,)`.\n\nThe `_eval_is_algebraic` method checks: if `self.exp / pi / I` is rational, the result is algebraic (returns `True`); if the exponent is non-zero and algebraic, the result is not algebraic (returns `False`); and if `self.exp / pi` is rational, the result is also not algebraic (returns `False`).",
    "rubric": [
      "Explains that `ExpMeta` metaclass (inheriting from `FunctionClass`) overrides `__instancecheck__` so `isinstance(x, exp)` returns True for both `exp` instances and `Pow` objects with base `S.Exp1` (Euler's number E)",
      "Identifies the dual representation: `exp(x)` as a Function instance vs `E**x` as a `Pow(S.Exp1, x)` instance",
      "Mentions the `global_parameters.exp_is_pow` flag (defaulting to False) and that when True, `exp.eval()` returns `Pow(S.Exp1, arg)` instead of an `exp` object",
      "Notes that `exp` inherits from `ExpBase`, which inherits from `DefinedFunction` (a subclass of `Function`), and `DefinedFunction.__new__` is decorated with `@cacheit`",
      "Describes key simplification logic in `exp.eval()`: handles `coeff*pi*I` patterns for integer multiples of pi/2, normalizes rational multiples to (-pi, pi], and uses `logcombine` to simplify `exp(c*log(x))` to `x**c`",
      "Mentions `_eval_is_algebraic` logic: returns True if exp/pi/I is rational, returns False if exponent is nonzero and algebraic, or if exp/pi is rational"
    ],
    "key_files": [
      "sympy/functions/elementary/exponential.py",
      "sympy/core/numbers.py",
      "sympy/core/parameters.py",
      "sympy/core/function.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/exponential.py] sympy.functions.elementary.exponential.exp\nsympy.functions.elementary.exponential.exp:\n  The exponential function, :math:`e^x`.\n\n    Examples\n    ========\n\n    >>> from sympy import exp, I, pi\n    >>> from sympy.abc import x\n    >>> exp(x)\n    exp(x)\n    >>> exp(x).diff(x)\n    exp(x)\n    >>> exp(I*pi)\n    -1\n\n    Parameters\n    ==========\n\n    arg : Expr\n\n    See Also\n    ========\n\n    sympy.functions.elementary.exponential.log\n\nsympy.functions.elementary.exponential.exp.fdiff:\n  Returns the first derivative of this function.\n\nsympy.functions.elementary.exponential.exp.base:\n  Returns the base of the exponential function.\n\nsympy.functions.elementary.exponential.exp.taylor_term:\n  Calculates the next term in the Taylor series expansion.\n\nsympy.functions.elementary.exponential.exp.as_real_imag:\n  Returns this function as a 2-tuple representing a complex number.\n\n        Examples\n        ========\n\n        >>> from sympy import exp, I\n        >>> from sympy.abc import x\n        >>> exp(x).as_real_imag()\n        (exp(re(x))*cos(im(x)), exp(re(x))*sin(im(x)))\n        >>> exp(1).as_real_imag()\n        (E, 0)\n        >>> exp(I).as_real_imag()\n        (cos(1), sin(1))\n        >>> exp(1+I).as_real_imag()\n        (E*cos(1), E*sin(1))\n\n        See Also\n        ========\n\n        sympy.functions.elementary.complexes.re\n        sympy.functions.elementary.complexes.im",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (ExpMeta metaclass, ExpBase class, DefinedFunction, @cacheit decorator, global_parameters.exp_is_pow flag, logcombine usage, Gruntz algorithm, Lindemann-Weierstrass theorem, etc.) that are not present in the provided documentation chunk. These claims cannot be verified from the given documentation alone.",
      "The documentation only describes exp as 'The exponential function, e^x' with basic examples, parameters, and methods like fdiff, base, taylor_term, and as_real_imag. The answer goes far beyond this scope.",
      "The claim about DefinedFunction being a superclass with @cacheit decorator is a specific implementation detail that could be plausible but cannot be verified from this documentation.",
      "The claim about ExpBase setting unbranched=True and _singularities is not mentioned in the documentation.",
      "While the answer's claims about as_real_imag behavior are consistent with the documentation examples, most other claims are extrapolations from source code not shown in the documentation chunk."
    ],
    "strip_verify_leakage": 0.27,
    "strip_verify_summary": "Several claims about design rationale (Gruntz algorithm comment, Lindemann-Weierstrass theorem name, unpolarify behavior) rely on stripped documentation; most structural and logic claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_05",
    "repo": "sympy",
    "question": "What is the full architecture of SymPy's `im` class \u2014 its inheritance chain, eval logic, assumption handlers, derivative rules, and integration with refine?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `im` class in SymPy is defined in `sympy/functions/elementary/complexes.py`. It is a subclass of `DefinedFunction`, which itself inherits from `Function` (defined in `sympy/core/function.py`). The `DefinedFunction` class uses `@cacheit` on `__new__` and delegates to `cls._new_()`. The inheritance chain is: `im` \u2192 `DefinedFunction` \u2192 `Function` \u2192 `Application` (with metaclass `FunctionClass`) + `Expr`.\n\nKey architectural details of `im`:\n\n1. **Class attributes**: `is_extended_real = True`, `unbranched = True`, and `_singularities = True` (the `Function.is_singular()` classmethod returns `True` directly when `_singularities is True`).\n\n2. **eval method logic**: The `eval` classmethod handles several cases:\n   - Returns `NaN` for `NaN` or `ComplexInfinity` inputs\n   - Returns `S.Zero` for extended real arguments\n   - Returns `-I * arg` for imaginary arguments (or when `I*arg` is extended real)\n   - Delegates to `arg.as_real_imag()[1]` for Matrix arguments\n   - Returns `-im(arg.args[0])` for `conjugate` arguments\n   - Otherwise decomposes `Add` terms: terms that are coefficients of `I` with extended_real coefficients go to `excluded`, non-real coefficients of `I` go to `reverted`, and remaining terms either go through `as_real_imag(ignore=arg)` expansion or stay as `included`. The final return is `cls(a) + re(b) + c` where a=included, b=reverted, c=excluded.\n\n3. **as_real_imag method**: Returns `(self, S.Zero)`.\n\n4. **Derivative handling**: `_eval_derivative` has two branches - if `x` is extended_real or arg is extended_real, it returns `im(Derivative(arg, x))`. If `x` is imaginary or arg is imaginary, it returns `-I * re(Derivative(arg, x))`.\n\n5. **Rewrite capability**: `_eval_rewrite_as_re` returns `-I*(self.args[0] - re(self.args[0]))`, establishing the identity `im(z) = -i*(z - re(z))`.\n\n6. **Assumption handlers**: `_eval_is_zero` returns `self.args[0].is_extended_real`, `_eval_is_algebraic` returns `self.args[0].is_algebraic`, `_eval_is_finite` and `_eval_is_complex` both return `True` if `self.args[0].is_finite`.\n\n7. **Sorting order**: In `Function.class_key()`, `im` has sort key `(4, 42, 'im')`, placing it after `conjugate` (40) and `re` (41) but before `arg` (43).\n\n8. **Refine integration**: In `sympy/assumptions/refine.py`, `refine_im` is registered in `handlers_dict` under key `'im'`. It returns `S.Zero` if the argument is known real, or `-S.ImaginaryUnit * arg` if the argument is known imaginary, otherwise falls back to `_refine_reim` which tries `expand(complex=True)`.\n\n9. **Connection to Expr.as_real_imag**: The default `Expr.as_real_imag()` method returns `(re(self), im(self))`, making `im` the canonical function for extracting imaginary parts in the general case.",
    "rubric": [
      "States that `im` inherits from `DefinedFunction`, which inherits from `Function`, and mentions `DefinedFunction` uses `@cacheit` on `__new__`",
      "Identifies class attributes: `is_extended_real = True`, `unbranched = True`, and `_singularities = True`",
      "Describes eval method handling NaN/ComplexInfinity (returns NaN), extended_real args (returns Zero), and imaginary args (returns -I*arg)",
      "Explains the Add decomposition logic in eval: splitting terms into excluded (real I-coefficients), reverted (non-real I-coefficients), and included, returning `cls(a) + re(b) + c`",
      "Mentions `as_real_imag` returns `(self, S.Zero)`",
      "Describes `_eval_derivative` branches: `im(Derivative(...))` for real x/arg, `-I*re(Derivative(...))` for imaginary x/arg",
      "Mentions `_eval_rewrite_as_re` returns `-I*(arg - re(arg))`",
      "Covers assumption handlers: `_eval_is_zero` checks `is_extended_real`, `_eval_is_algebraic` checks `is_algebraic`, `_eval_is_finite` and `_eval_is_complex` check `is_finite`",
      "Explains `refine_im` in the assumptions system returns Zero for real args, `-I*arg` for imaginary args, and falls back to `_refine_reim`",
      "Mentions the sort key `(4, 42, 'im')` placing it between `re` (41) and `arg` (43) in the Function.class_key ordering"
    ],
    "key_files": [
      "sympy/functions/elementary/complexes.py",
      "sympy/core/function.py",
      "sympy/assumptions/refine.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/complexes.py] sympy.functions.elementary.complexes.im\nsympy.functions.elementary.complexes.im:\n  Returns imaginary part of expression. This function performs only\n    elementary analysis and so it will fail to decompose properly more\n    complicated expressions. If completely simplified result is needed then\n    use ``Basic.as_real_imag()`` or perform complex expansion on instance of\n    this function.\n\n    Examples\n    ========\n\n    >>> from sympy import re, im, E, I\n    >>> from sympy.abc import x, y\n    >>> im(2*E)\n    0\n    >>> im(2*I + 17)\n    2\n    >>> im(x*I)\n    re(x)\n    >>> im(re(x) + y)\n    im(y)\n    >>> im(2 + 3*I)\n    3\n\n    Parameters\n    ==========\n\n    arg : Expr\n        Real or complex expression.\n\n    Returns\n    =======\n\n    expr : Expr\n        Imaginary part of expression.\n\n    See Also\n    ========\n\n    re\n\nsympy.functions.elementary.complexes.im.as_real_imag:\n  Return the imaginary part with a zero real part.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor documentation-style explanations about purpose/semantics were trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_08",
    "repo": "sympy",
    "question": "How does SymPy's `cbrt` function work internally, and why does `cbrt(x**3)` not simplify for general symbols while `cbrt(x)**3` does?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `cbrt` function in SymPy is implemented as a plain Python function (not a class) in `sympy/functions/elementary/miscellaneous.py`. It is a thin wrapper that delegates entirely to `Pow`:\n\n```python\ndef cbrt(arg, evaluate=None):\n    return Pow(arg, Rational(1, 3), evaluate=evaluate)\n```\n\nHere's the detailed algorithm and implementation flow:\n\n1. **Construction**: `cbrt(arg)` creates `Pow(arg, Rational(1, 3), evaluate=evaluate)`. This contrasts with `sqrt(arg)` which uses the singleton `S.Half` instead of `Rational(1, 2)`, and with `root(arg, n)` which uses `1/n` (Python division after `sympify(n)`).\n\n2. **Evaluation in `Pow.__new__`** (in `sympy/core/power.py`): When `evaluate` is `None`, it reads `global_parameters.evaluate`. If evaluating, the constructor performs multiple simplification checks (e.g., handling `S.Zero`, `S.One`, `S.NaN`, `S.Infinity` as exp, checking if base is negative with even/odd exp, recognizing `e` as base). After all special cases, it calls `base._eval_power(exp)` to attempt simplification.\n\n3. **Why `cbrt(x)**3` simplifies to `x`**: When computing `Pow(x, Rational(1,3))**3`, `Pow._eval_power` is called with `b=x`, `e=Rational(1,3)`, `expt=3`. Since `expt.is_integer` is `True`, `s` is set to `1` immediately, and the method returns `s * Pow(b, e*expt) = 1 * Pow(x, 1) = x`.\n\n4. **Why `cbrt(x**3)` does NOT simplify for general `x`**: When computing `Pow(Pow(x, 3), Rational(1,3))`, `Pow._eval_power` is called with `b=x`, `e=3`, `expt=Rational(1,3)`. Since `expt` is not an integer, it checks: `b.is_polar` (no), `e.is_extended_real` (True), then `(abs(e) < 1)` is False, `e != 1`, and `b.is_extended_nonnegative` is `None` for a general symbol. The `_half(expt)` helper returns `None` since `Rational(1,3)` doesn't have denominator 2. So `s` stays `None` and no simplification occurs. However, if `x` is declared positive, `b.is_extended_nonnegative` is `True`, so `s=1` and it returns `Pow(x, 3*Rational(1,3)) = x`.\n\n5. **`real_root` alternative**: The `real_root` function handles cube roots of negative numbers differently. When called without `n`, it uses a `Transform` (from `sympy.core.rules`) with `xreplace` to convert `(-n)**(1/odd)` patterns to `-n**(1/odd)`. The filter checks `x.is_Pow and x.base.is_negative and x.exp.is_Rational and x.exp.p == 1 and x.exp.q % 2`. When called with `n`, it returns a `Piecewise` that uses `sign(arg) * root(Abs(arg), n)` for real args with odd `n`.\n\n6. **Codegen `Cbrt` class**: There's a separate `Cbrt(Function)` class in `sympy/codegen/cfunctions.py` that preserves `cbrt` as an unevaluated function node. Its `fdiff` returns `Pow(self.args[0], Rational(-_Two/3))/3` (where `_Two = S(2)`), and it can be rewritten as `Pow` via `_eval_rewrite_as_Pow` which calls `_Cbrt(arg)` \u2192 `Pow(x, Rational(1, 3))`.\n\n7. **Integer base optimization**: When the base is an `Integer`, `Integer._eval_power` (in `numbers.py`) attempts factorization-based simplification.",
    "rubric": [
      "Explains that `cbrt` is a plain Python function (not a class) that delegates to `Pow(arg, Rational(1, 3), evaluate=evaluate)`",
      "Describes that `cbrt(x)**3` triggers `Pow._eval_power` where `expt.is_integer` is True, so `s=1` and the result simplifies to `Pow(x, 1) = x`",
      "Explains that `cbrt(x**3)` calls `Pow._eval_power` with `e=3` and `expt=Rational(1,3)`, where `expt` is not an integer, `abs(e) < 1` is False, `e != 1`, and `b.is_extended_nonnegative` is None for a general symbol, so `s` remains None and no simplification occurs",
      "Notes that if x is declared positive (nonnegative), `b.is_extended_nonnegative` becomes True, `s=1`, and `cbrt(x**3)` does simplify to `x`",
      "Mentions the codegen `Cbrt` class in cfunctions that preserves cbrt as an unevaluated function node, rewritable as Pow",
      "Mentions `real_root` as an alternative that handles cube roots of negative numbers using a Transform with xreplace or Piecewise with sign/Abs"
    ],
    "key_files": [
      "sympy/functions/elementary/miscellaneous.py",
      "sympy/core/power.py",
      "sympy/core/numbers.py",
      "sympy/codegen/cfunctions.py",
      "sympy/core/rules.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/miscellaneous.py] sympy.functions.elementary.miscellaneous.cbrt\nReturns the principal cube root.\n\n    Parameters\n    ==========\n\n    evaluate : bool, optional\n        The parameter determines if the expression should be evaluated.\n        If ``None``, its value is taken from\n        ``global_parameters.evaluate``.\n\n    Examples\n    ========\n\n    >>> from sympy import cbrt, Symbol\n    >>> x = Symbol('x')\n\n    >>> cbrt(x)\n    x**(1/3)\n\n    >>> cbrt(x)**3\n    x\n\n    Note that cbrt(x**3) does not simplify to x.\n\n    >>> cbrt(x**3)\n    (x**3)**(1/3)\n\n    This is because the two are not equal to each other in general.\n    For example, consider `x == -1`:\n\n    >>> from sympy import Eq\n    >>> Eq(cbrt(x**3), x).subs(x, -1)\n    False\n\n    This is because cbrt computes the principal cube root, this\n    identity does hold if `x` is positive:\n\n    >>> y = Symbol('y', positive=True)\n    >>> cbrt(y**3)\n    y\n\n    See Also\n    ========\n\n    sympy.polys.rootoftools.rootof, root, real_root\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Cube_root\n    .. [2] https://en.wikipedia.org/wiki/Principal_value",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed printer details and integer_nthroot specifics not visible in provided code, and rewrote line number references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_07",
    "repo": "sympy",
    "question": "Where is `sqrt` defined and how does it ultimately compute its result?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `sqrt` function is defined in `sympy/functions/elementary/miscellaneous.py` as a plain Python function (not a class). It is located in the same file as `cbrt`, `root`, `real_root`, `Min`, `Max`, and `IdentityFunction`. The function body is a single line: `return Pow(arg, S.Half, evaluate=evaluate)`, delegating entirely to the `Pow` class in `sympy/core/power.py`.\n\nKey architectural facts:\n\n1. **`sqrt` is not a class** - it's a plain function that returns a `Pow` instance. This means `sqrt(x).has(sqrt)` returns `False` and `func_name(sqrt(x))` returns `'Pow'`.\n\n2. **`Pow.__new__` is decorated with `@cacheit`**, meaning repeated calls to `sqrt` with the same argument will be cached.\n\n3. **Evaluation delegation**: When `evaluate=True` (the default from `global_parameters.evaluate`), `Pow.__new__` calls `base._eval_power(exp)`. For integer bases, this dispatches to `Integer._eval_power` in `sympy/core/numbers.py`, which:\n   - For negative integers with exponent `S.Half`, extracts `I`: `return S.ImaginaryUnit*Pow(-self, expt)`\n   - For perfect squares (e.g., sqrt(4)), uses `integer_nthroot(abs(self.p), expt.q)` from `sympy/core/intfunc.py` which in turn calls `iroot` from `sympy.external.gmpy`\n\n4. **Assumption propagation**: In `Pow._eval_is_extended_negative`, there's a special case: `if self.exp is S.Half: if self.base.is_complex or self.base.is_extended_real: return False` \u2014 meaning sqrt of any complex or real number is never considered extended_negative.\n\n5. **`S.Half` is a singleton** with `p = 1` and `q = 2`, used with identity checks (`is`) rather than equality checks.\n\n6. **The `global_parameters` instance** is created in `sympy/core/parameters.py` as `_global_parameters(evaluate=True, distribute=True, exp_is_pow=False)`, which is a thread-local (`local`) subclass.",
    "rubric": [
      "sqrt is defined in sympy/functions/elementary/miscellaneous.py as a plain function (not a class)",
      "It returns Pow(arg, S.Half, evaluate=evaluate), delegating to the Pow class",
      "Pow.__new__ is decorated with @cacheit for caching repeated calls",
      "For integer bases, Pow.__new__ calls base._eval_power(exp), dispatching to Integer._eval_power in sympy/core/numbers.py",
      "Integer._eval_power handles negative integers with exponent S.Half by extracting ImaginaryUnit",
      "For perfect squares, integer_nthroot is used to compute the exact root",
      "sqrt is in the same file as cbrt, root, real_root, Min, Max, and IdentityFunction",
      "Because sqrt is a function not a class, sqrt(x) is actually a Pow instance (sqrt(x).func is Pow)"
    ],
    "key_files": [
      "sympy/functions/elementary/miscellaneous.py",
      "sympy/core/power.py",
      "sympy/core/numbers.py",
      "sympy/printing/str.py",
      "sympy/core/intfunc.py",
      "sympy/core/parameters.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/miscellaneous.py] sympy.functions.elementary.miscellaneous.sqrt\nReturns the principal square root.\n\n    Parameters\n    ==========\n\n    evaluate : bool, optional\n        The parameter determines if the expression should be evaluated.\n        If ``None``, its value is taken from\n        ``global_parameters.evaluate``.\n\n    Examples\n    ========\n\n    >>> from sympy import sqrt, Symbol, S\n    >>> x = Symbol('x')\n\n    >>> sqrt(x)\n    sqrt(x)\n\n    >>> sqrt(x)**2\n    x\n\n    Note that sqrt(x**2) does not simplify to x.\n\n    >>> sqrt(x**2)\n    sqrt(x**2)\n\n    This is because the two are not equal to each other in general.\n    For example, consider x == -1:\n\n    >>> from sympy import Eq\n    >>> Eq(sqrt(x**2), x).subs(x, -1)\n    False\n\n    This is because sqrt computes the principal square root, so the square may\n    put the argument in a different branch.  This identity does hold if x is\n    positive:\n\n    >>> y = Symbol('y', positive=True)\n    >>> sqrt(y**2)\n    y\n\n    You can force this simplification by using the powdenest() function with\n    the force option set to True:\n\n    >>> from sympy import powdenest\n    >>> sqrt(x**2)\n    sqrt(x**2)\n    >>> powdenest(sqrt(x**2), force=True)\n    x\n\n    To get both branches of the square root you can use the rootof function:\n\n    >>> from sympy import rootof\n\n    >>> [rootof(x**2-3,i) for i in (0,1)]\n    [-sqrt(3), sqrt(3)]\n\n    Although ``sqrt`` is printed, there is no ``sqrt`` function so looking for\n    ``sqrt`` in an expression will fail:\n\n    >>> from sympy.utilities.misc import func_name\n    >>> func_name(sqrt(x))\n    'Pow'\n    >>> sqrt(x).has(sqrt)\n    False\n\n    To find ``sqrt`` look for ``Pow`` with an exponent of ``1/2``:\n\n    >>> (x + 1/sqrt(x)).find(lambda i: i.is_Pow and abs(i.exp) is S.Half)\n    {1/sqrt(x)}\n\n    See Also\n    ========\n\n    sympy.polys.rootoftools.rootof, root, real_root\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Square_root\n    .. [2] https://en.wikipedia.org/wiki/Principal_value",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed comment-dependent claim, unverifiable printer details, and line numbers throughout.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_09",
    "repo": "sympy",
    "question": "What are the key implementation details of `piecewise_exclusive`, including how it constructs exclusive conditions, handles NaN, and integrates with the expression tree?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `piecewise_exclusive` function (defined in `sympy/functions/elementary/piecewise.py`) transforms Piecewise expressions to use mutually exclusive conditions. Here are the key implementation details:\n\n1. **Internal mechanism**: It defines a nested function `make_exclusive(*pwargs)` that iterates through the Piecewise argument pairs. It maintains a cumulative condition (`cumcond`) initialized to `false` (the `BooleanFalse()` singleton from `sympy.logic.boolalg`). For each condition `cond_i`, it computes the exclusive condition as `And(cond_i, Not(cumcond)).simplify()` and updates `cumcond` as `Or(cond_i, cumcond).simplify()`.\n\n2. **Deferred simplification**: The implementation defers the simplification of `cumcond` for the last (nth) case. It only simplifies `Or(cond_n, cumcond)` for the final case when `skip_nan=False` to determine if a NaN clause is needed.\n\n3. **NaN fallback**: The `Undefined` constant used for the fallback case is defined at module level as `Undefined = S.NaN`. When `cumcond is not true` after processing all cases and `skip_nan=False`, it appends `(Undefined, Not(cumcond).simplify())` \u2014 meaning NaN is returned for any input not covered by the conditions.\n\n4. **The `deep` parameter and `replace`**: When `deep=True` (the default), it uses `expr.replace(Piecewise, make_exclusive)`, which leverages `Basic.replace()` from `sympy/core/basic.py`. The `replace` method, when called with a type and a callable, creates `_value = lambda expr, result: value(*expr.args)`, meaning `make_exclusive` receives the Piecewise's `args` (ExprCondPair tuples) as positional arguments. The `walk` function in `replace` traverses the expression tree bottom-up, applying the transformation to all nested `Piecewise` instances.\n\n5. **evaluate=False**: The final `Piecewise` is constructed with `evaluate=False` to prevent the normal evaluation logic from simplifying conditions back to non-exclusive form. Without this, `Piecewise.eval()` would call `_piecewise_collapse_arguments` which could merge or drop conditions.\n\n6. **Class hierarchy**: `Piecewise` inherits from `DefinedFunction`, which inherits from `Function`. `DefinedFunction` is a minimal class with only a `__new__` method decorated with `@cacheit`. `Piecewise` sets `nargs = None` which means it accepts any number of arguments (treated as `Naturals0` by the `FunctionClass` metaclass's `nargs` property, which returns `S.Naturals0` when `_nargs` is falsy).\n\n7. **Usage in Laplace transforms**: `piecewise_exclusive` is imported in `sympy/integrals/laplace.py`, where it is used in Piecewise-to-Heaviside conversion functionality.\n\n8. **`is_Piecewise` class attribute**: `Piecewise` defines `is_Piecewise = True`, and the `Basic` base class defines `is_Piecewise = False` as a default, enabling quick type checks throughout the codebase.",
    "rubric": [
      "Explains that a nested `make_exclusive` function iterates through Piecewise args, maintaining a cumulative condition (`cumcond`) initialized to `false` (BooleanFalse), computing exclusive conditions via `And(cond_i, Not(cumcond)).simplify()` and updating cumcond via `Or(cond_i, cumcond).simplify()`",
      "Notes that simplification of cumcond is deferred for the last case, only computing `Or(cond_n, cumcond)` for the final condition when `skip_nan=False`",
      "Identifies that `Undefined = S.NaN` is defined at module level and appended as `(Undefined, Not(cumcond).simplify())` when cumcond is not true and skip_nan=False",
      "Explains that when `deep=True`, `expr.replace(Piecewise, make_exclusive)` is used, where `replace` passes the Piecewise's args as positional arguments to `make_exclusive` and traverses the expression tree bottom-up",
      "Mentions that the final Piecewise is constructed with `evaluate=False` to prevent normal evaluation logic from simplifying conditions back",
      "Notes that Piecewise inherits from Function (or mentions the class hierarchy involving DefinedFunction) and sets `nargs = None`",
      "Mentions that `piecewise_exclusive` is used in the Laplace transform module for Piecewise-to-Heaviside conversion",
      "Notes that `Piecewise` defines `is_Piecewise = True` while `Basic` defines `is_Piecewise = False` as a default"
    ],
    "key_files": [
      "sympy/functions/elementary/piecewise.py",
      "sympy/core/basic.py",
      "sympy/core/function.py",
      "sympy/integrals/laplace.py"
    ],
    "source_doc": "[docstring: sympy/functions/elementary/piecewise.py] sympy.functions.elementary.piecewise.piecewise_exclusive\nRewrite :class:`Piecewise` with mutually exclusive conditions.\n\n    Explanation\n    ===========\n\n    SymPy represents the conditions of a :class:`Piecewise` in an\n    \"if-elif\"-fashion, allowing more than one condition to be simultaneously\n    True. The interpretation is that the first condition that is True is the\n    case that holds. While this is a useful representation computationally it\n    is not how a piecewise formula is typically shown in a mathematical text.\n    The :func:`piecewise_exclusive` function can be used to rewrite any\n    :class:`Piecewise` with more typical mutually exclusive conditions.\n\n    Note that further manipulation of the resulting :class:`Piecewise`, e.g.\n    simplifying it, will most likely make it non-exclusive. Hence, this is\n    primarily a function to be used in conjunction with printing the Piecewise\n    or if one would like to reorder the expression-condition pairs.\n\n    If it is not possible to determine that all possibilities are covered by\n    the different cases of the :class:`Piecewise` then a final\n    :class:`~sympy.core.numbers.NaN` case will be included explicitly. This\n    can be prevented by passing ``skip_nan=True``.\n\n    Examples\n    ========\n\n    >>> from sympy import piecewise_exclusive, Symbol, Piecewise, S\n    >>> x = Symbol('x', real=True)\n    >>> p = Piecewise((0, x < 0), (S.Half, x <= 0), (1, True))\n    >>> piecewise_exclusive(p)\n    Piecewise((0, x < 0), (1/2, Eq(x, 0)), (1, x > 0))\n    >>> piecewise_exclusive(Piecewise((2, x > 1)))\n    Piecewise((2, x > 1), (nan, x <= 1))\n    >>> piecewise_exclusive(Piecewise((2, x > 1)), skip_nan=True)\n    Piecewise((2, x > 1))\n\n    Parameters\n    ==========\n\n    expr: a SymPy expression.\n        Any :class:`Piecewise` in the expression will be rewritten.\n    skip_nan: ``bool`` (default ``False``)\n        If ``skip_nan`` is set to ``True`` then a final\n        :class:`~sympy.core.numbers.NaN` case will not be included.\n    deep:  ``bool`` (default ``True``)\n        If ``deep`` is ``True`` then :func:`piecewise_exclusive` will rewrite\n        any :class:`Piecewise` subexpressions in ``expr`` rather than just\n        rewriting ``expr`` itself.\n\n    Returns\n    =======\n\n    An expression equivalent to ``expr`` but where all :class:`Piecewise` have\n    been rewritten with mutually exclusive conditions.\n\n    See Also\n    ========\n\n    Piecewise\n    piecewise_fold",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals of line numbers, a quoted comment, and some design rationale descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_10",
    "repo": "sympy",
    "question": "Why is the \"scipy\" method faster than the \"sympy\" method in `jn_zeros` for computing spherical Bessel function zeros?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `jn_zeros` function provides two methods for computing zeros of the spherical Bessel function, with the \"scipy\" method being faster than the \"sympy\" method for several specific performance-related reasons:\n\n1. **Arbitrary precision vs. machine precision**: The \"sympy\" method uses `mpmath.besseljzero` with configurable precision (controlled by the `dps` parameter, default 15, converted to binary precision via `dps_to_prec`). This means it computes zeros at arbitrary precision using multiple-precision arithmetic. The \"scipy\" method uses hardware floating-point numbers (machine precision ~15-16 digits), which are inherently much faster to compute with.\n\n2. **Context creation overhead**: The \"sympy\" method uses `local_workprec`, which is a context manager that borrows from a pool of `MPContext` objects (class variable `_contexts: list[MPContext] = []`). If the pool is empty, it creates a new context via `_new_mpcontext()`, which instantiates both a `MPContext()` and `MPIntervalContext()`. While pooling helps, this is still more overhead than the scipy approach.\n\n3. **Conversion overhead in the sympy method**: Each iteration of the sympy method loop calls `S(n + 0.5)._to_mpmath(prec)` to convert the order to an mpmath number, and `Expr._from_mpmath(v, prec)` to convert the result back to a SymPy `Float` object. The `_from_mpmath` method goes through `Float._new` to create a Float from the mpf tuple.\n\n4. **Root-finding strategy for scipy**: The scipy method uses an intelligent initial guess strategy\u2014it approximates the first root position as `n + math.pi` and subsequent roots as `previous_root + math.pi` (exploiting the roughly \u03c0-spaced nature of Bessel zeros). Combined with SciPy's `newton` (Newton-Raphson), which converges quadratically from a good initial guess using hardware floats, this is very fast.\n\n5. **The sympy method iterates over zeros sequentially**: In the loop `for l in range(1, k + 1)`, it calls `ctx.besseljzero(S(n + 0.5)._to_mpmath(prec), l)` for each zero index `l`. The `besseljzero` function from mpmath computes the l-th zero directly using its own algorithm at the specified working precision, which involves more computation per zero than Newton's method with a good initial guess.\n\n6. **Fallback compatibility in scipy method**: The scipy method includes a try/except for importing `spherical_jn` vs. the older `sph_jn` API. When using the newer `spherical_jn`, the lambda `f = lambda x: spherical_jn(n, x)` is simpler and faster than the older `sph_jn(n, x)[0][-1]` which computed all orders up to n and took the last element.\n\nThe fundamental performance tradeoff is: the sympy method supports arbitrary precision (e.g., `dps=50` for 50 decimal digits) but is slower, while the scipy method is limited to ~15 significant digits but leverages optimized C-compiled SciPy routines and hardware floating point.",
    "rubric": [
      "Explains that the sympy method uses arbitrary/multiple-precision arithmetic via mpmath (with configurable dps), while scipy uses hardware/machine-precision floating point",
      "Mentions the overhead of creating or borrowing MPContext objects (via local_workprec and _new_mpcontext, which instantiates MPContext and MPIntervalContext)",
      "Describes the conversion overhead in the sympy path: converting order via S(n+0.5)._to_mpmath(prec) and converting results back via Expr._from_mpmath(v, prec)",
      "Explains the scipy method's initial guess strategy: first root approximated as n + pi, subsequent roots as previous_root + pi, exploiting the roughly \u03c0-spaced nature of Bessel zeros",
      "Notes that scipy uses Newton-Raphson (scipy.optimize.newton) which converges quickly from good initial guesses using fast hardware floats",
      "Mentions the fundamental tradeoff: sympy supports arbitrary precision (controlled by dps parameter) but is slower, while scipy is limited to ~15 digits but uses optimized routines"
    ],
    "key_files": [
      "sympy/functions/special/bessel.py",
      "sympy/external/mpmath.py",
      "sympy/core/expr.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/bessel.py] sympy.functions.special.bessel.jn_zeros\nZeros of the spherical Bessel function of the first kind.\n\n    Explanation\n    ===========\n\n    This returns an array of zeros of $jn$ up to the $k$-th zero.\n\n    * method = \"sympy\": uses `mpmath.besseljzero\n      <https://mpmath.org/doc/current/functions/bessel.html#mpmath.besseljzero>`_\n    * method = \"scipy\": uses the\n      `SciPy's sph_jn <https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.jn_zeros.html>`_\n      and\n      `newton <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.newton.html>`_\n      to find all\n      roots, which is faster than computing the zeros using a general\n      numerical solver, but it requires SciPy and only works with low\n      precision floating point numbers. (The function used with\n      method=\"sympy\" is a recent addition to mpmath; before that a general\n      solver was used.)\n\n    Examples\n    ========\n\n    >>> from sympy import jn_zeros\n    >>> jn_zeros(2, 4, dps=5)\n    [5.7635, 9.095, 12.323, 15.515]\n\n    See Also\n    ========\n\n    jn, yn, besselj, besselk, bessely\n\n    Parameters\n    ==========\n\n    n : integer\n        order of Bessel function\n\n    k : integer\n        number of zeros to return",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states the scipy method uses 'SciPy's sph_jn' and 'newton', while the answer mentions 'spherical_jn' as a newer API - this is plausible code-level detail but not directly stated in the documentation.",
      "Many specific implementation details (like `local_workprec`, `_contexts: list[MPContext]`, `S(n + 0.5)._to_mpmath(prec)`, `n + math.pi` initial guess strategy, `previous_root + math.pi`) are not in the provided documentation and appear to be extrapolated from source code knowledge - these could be accurate but cannot be verified from the given doc chunk alone.",
      "The documentation says the sympy method 'uses mpmath.besseljzero' while the answer says it's a 'recent addition to mpmath' - this is actually supported by the parenthetical note in the documentation.",
      "The answer's claim about default dps=15 is not stated in the documentation (the example uses dps=5), though it's a plausible default."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only a minor detail about mpf_norm in Float._new was partially unverifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_11",
    "repo": "sympy",
    "question": "Where is `bspline_basis` defined and how does it get exported, and what's the overall structure of the B-spline implementation?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `bspline_basis` function is defined in `sympy/functions/special/bsplines.py`. It is exported at the top level of sympy through the chain: `sympy/functions/special/bsplines.py` \u2192 `sympy/functions/__init__.py` \u2192 `sympy/__init__.py`. The function is decorated with `@lru_cache(maxsize=128)` from `functools`. It recursively calls itself to compute lower-degree B-splines, and combines them via the helper function `_add_splines` defined in the same file. A helper function `_ivl` extracts interval bounds from `And` conditions (checking for `And` with two args and accessing `.lts` and `.gts`). The `bspline_basis_set` function calls `bspline_basis` in a list comprehension for `i in range(n_splines)` where `n_splines = len(knots) - d - 1`. The `interpolating_spline` function uses `bspline_basis_set` internally, imports `linsolve` from `sympy.solvers.solveset` and `Matrix` from `sympy.matrices.dense` locally, and constructs the knot vector differently for odd vs even degree splines (using `d.is_odd` to branch). The base case (d==0) uses `Interval(knots[n], knots[n+1]).contains(x)` where `Interval` is imported from `sympy.sets.sets`. The `Interval.contains` call for a closed interval produces `And` expressions with `<=` inequalities, as evidenced by the test assertions. Inside `bspline_basis`, the user-supplied `x` is saved as `xvar` and replaced with a `Dummy()`, and the result is substituted back with `result.xreplace({x: xvar})`.",
    "rubric": [
      "bspline_basis is defined in sympy/functions/special/bsplines.py",
      "It is exported through sympy/functions/__init__.py and then sympy/__init__.py to top-level sympy",
      "bspline_basis is decorated with @lru_cache(maxsize=128) from functools",
      "It recursively calls itself for lower-degree B-splines and combines results via _add_splines helper",
      "A helper function _ivl extracts interval bounds from And conditions (checking for And with two args, accessing .lts and .gts)",
      "bspline_basis_set calls bspline_basis in a list comprehension with n_splines = len(knots) - d - 1",
      "interpolating_spline locally imports linsolve from sympy.solvers.solveset and Matrix from sympy.matrices.dense",
      "interpolating_spline constructs knot vectors differently for odd vs even degree (using d.is_odd to branch)",
      "The base case (d==0) uses Interval(knots[n], knots[n+1]).contains(x) where Interval is from sympy.sets.sets",
      "Inside bspline_basis, the user-supplied x is saved as xvar and replaced with a Dummy(), then substituted back with result.xreplace({x: xvar})"
    ],
    "key_files": [
      "sympy/functions/special/bsplines.py",
      "sympy/sets/sets.py",
      "sympy/functions/__init__.py",
      "sympy/functions/special/tests/test_bsplines.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/bsplines.py] sympy.functions.special.bsplines.bspline_basis\nThe $n$-th B-spline at $x$ of degree $d$ with knots.\n\n    Explanation\n    ===========\n\n    B-Splines are piecewise polynomials of degree $d$. They are defined on a\n    set of knots, which is a sequence of integers or floats.\n\n    Examples\n    ========\n\n    The 0th degree splines have a value of 1 on a single interval:\n\n        >>> from sympy import bspline_basis\n        >>> from sympy.abc import x\n        >>> d = 0\n        >>> knots = tuple(range(5))\n        >>> bspline_basis(d, knots, 0, x)\n        Piecewise((1, (x >= 0) & (x <= 1)), (0, True))\n\n    For a given ``(d, knots)`` there are ``len(knots)-d-1`` B-splines\n    defined, that are indexed by ``n`` (starting at 0).\n\n    Here is an example of a cubic B-spline:\n\n        >>> bspline_basis(3, tuple(range(5)), 0, x)\n        Piecewise((x**3/6, (x >= 0) & (x <= 1)),\n                  (-x**3/2 + 2*x**2 - 2*x + 2/3,\n                  (x >= 1) & (x <= 2)),\n                  (x**3/2 - 4*x**2 + 10*x - 22/3,\n                  (x >= 2) & (x <= 3)),\n                  (-x**3/6 + 2*x**2 - 8*x + 32/3,\n                  (x >= 3) & (x <= 4)),\n                  (0, True))\n\n    By repeating knot points, you can introduce discontinuities in the\n    B-splines and their derivatives:\n\n        >>> d = 1\n        >>> knots = (0, 0, 2, 3, 4)\n        >>> bspline_basis(d, knots, 0, x)\n        Piecewise((1 - x/2, (x >= 0) & (x <= 2)), (0, True))\n\n    It is quite time consuming to construct and evaluate B-splines. If\n    you need to evaluate a B-spline many times, it is best to lambdify them\n    first:\n\n        >>> from sympy import lambdify\n        >>> d = 3\n        >>> knots = tuple(range(10))\n        >>> b0 = bspline_basis(d, knots, 0, x)\n        >>> f = lambdify(x, b0)\n        >>> y = f(0.5)\n\n    Parameters\n    ==========\n\n    d : integer\n        degree of bspline\n\n    knots : list of integer values\n        list of knots points of bspline\n\n    n : integer\n        $n$-th B-spline\n\n    x : symbol\n\n    See Also\n    ========\n\n    bspline_basis_set\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/B-spline",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with exact line numbers and one design-rationale claim about why Dummy is used.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_12",
    "repo": "sympy",
    "question": "How does SymPy's DiracDelta class work end-to-end, from instantiation and evaluation through integration and expansion?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `DiracDelta` class in SymPy is implemented in `sympy/functions/special/delta_functions.py` and inherits from `DefinedFunction` (which inherits from `Function`). Here's how it works at the implementation level:\n\n**Class hierarchy and instantiation:**\n- `DiracDelta` \u2192 `DefinedFunction` \u2192 `Function` \u2192 `Application` (with `FunctionClass` metaclass)\n- `DefinedFunction.__new__` is decorated with `@cacheit` and delegates to `cls._new_()` from `Function`\n- The `_new_` method (in `Function`) validates argument count via `_valid_nargs`, then calls `Application.__new__` which calls `cls.eval()` when `evaluate=True`\n- The number of accepted arguments (`nargs`) is automatically determined by the `arity()` function which inspects the `eval` method's signature. Since `eval(cls, arg, k=S.Zero)` has 1 required parameter and 1 with default, `nargs` becomes `{1, 2}`.\n\n**eval() method mechanics:**\n- Returns `S.NaN` if arg is NaN, `S.Zero` if arg is nonzero, raises ValueError for complex arguments\n- For negative sign detection, uses `arg.args_cnc()` checking if `c[0] is S.NegativeOne`\n- When `k` is odd and leading factor is -1: returns `-cls(-arg, k)`\n- When `k` is even and leading factor is -1: returns `cls(-arg, k)` if k else `cls(-arg)`\n- When `k` is zero and no simplification possible: returns `cls(arg, evaluate=False)`\n\n**Integration support via `deltaintegrate`:**\n- For simple DiracDelta (degree 1 in x): integrates to `Heaviside(f.args[0])`\n- For DiracDelta derivatives: returns `DiracDelta(f.args[0], f.args[1] - 1) / f.args[0].as_poly().LC()`\n- For products `f(x)*DiracDelta(x-x0)`: uses repeated integration by parts with formula `(-1)^n * f^(n)(x0)`, iterating n from the derivative order down to 0, returning the first non-zero term as either `r*Heaviside(x - point)` (if m=0) or `r*DiracDelta(x, m-1)` (if m>0)\n\n**`_eval_expand_diracdelta` implementation:**\n- Uses `sympy.polys.polyroots.roots()` to find roots of the argument polynomial\n- Skips roots with multiplicity > 1\n- Skips roots where `is_real` is `False` (but accepts `None`/unknown)\n- Falls back to returning `self` on `PolynomialError`\n\n**`is_simple` implementation:**\n- Calls `self.args[0].as_poly(x)` and checks if `p.degree() == 1`\n- Returns `False` if `as_poly` returns a falsy value\n\n**Piecewise rewrite:**\n- Only works for zero-order DiracDelta (`len(args) == 1`); returns `None` implicitly for derivatives, leaving the expression unchanged",
    "rubric": [
      "Explains the class hierarchy: DiracDelta inherits from DefinedFunction, which inherits from Function, with DefinedFunction.__new__ decorated by @cacheit",
      "Explains that nargs is {1, 2} because eval(cls, arg, k=S.Zero) has 1 required and 1 optional parameter",
      "Describes eval() returning S.NaN for NaN, S.Zero for nonzero args, and raising ValueError for complex arguments",
      "Describes the negative-sign handling in eval() using args_cnc() to check if c[0] is S.NegativeOne, with different behavior for odd k (negate) vs even k",
      "Describes deltaintegrate: simple DiracDelta integrates to Heaviside, derivatives reduce order via division by leading coefficient",
      "Describes deltaintegrate's integration-by-parts loop for products f(x)*DiracDelta(x-x0, n), using (-1)^n * f^(n)(x0) and returning either r*Heaviside or r*DiracDelta(x, m-1)",
      "Describes _eval_expand_diracdelta using sympy.polys.polyroots.roots() to find roots, skipping roots with multiplicity > 1 or is_real is False",
      "Describes is_simple checking if the argument is a degree-1 polynomial in x via as_poly",
      "Mentions the Piecewise rewrite only works for zero-order DiracDelta (len(args) == 1)"
    ],
    "key_files": [
      "sympy/functions/special/delta_functions.py",
      "sympy/integrals/deltafunctions.py",
      "sympy/core/function.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/delta_functions.py] sympy.functions.special.delta_functions.DiracDelta\nsympy.functions.special.delta_functions.DiracDelta:\n  The DiracDelta function and its derivatives.\n\n    Explanation\n    ===========\n\n    DiracDelta is not an ordinary function. It can be rigorously defined either\n    as a distribution or as a measure.\n\n    DiracDelta only makes sense in definite integrals, and in particular,\n    integrals of the form ``Integral(f(x)*DiracDelta(x - x0), (x, a, b))``,\n    where it equals ``f(x0)`` if ``a <= x0 <= b`` and ``0`` otherwise. Formally,\n    DiracDelta acts in some ways like a function that is ``0`` everywhere except\n    at ``0``, but in many ways it also does not. It can often be useful to treat\n    DiracDelta in formal ways, building up and manipulating expressions with\n    delta functions (which may eventually be integrated), but care must be taken\n    to not treat it as a real function. SymPy's ``oo`` is similar. It only\n    truly makes sense formally in certain contexts (such as integration limits),\n    but SymPy allows its use everywhere, and it tries to be consistent with\n    operations on it (like ``1/oo``), but it is easy to get into trouble and get\n    wrong results if ``oo`` is treated too much like a number. Similarly, if\n    DiracDelta is treated too much like a function, it is easy to get wrong or\n    nonsensical results.\n\n    DiracDelta function has the following properties:\n\n    1) $\\frac{d}{d x} \\theta(x) = \\delta(x)$\n    2) $\\int_{-\\infty}^\\infty \\delta(x - a)f(x)\\, dx = f(a)$ and $\\int_{a-\n       \\epsilon}^{a+\\epsilon} \\delta(x - a)f(x)\\, dx = f(a)$\n    3) $\\delta(x) = 0$ for all $x \\neq 0$\n    4) $\\delta(g(x)) = \\sum_i \\frac{\\delta(x - x_i)}{\\|g'(x_i)\\|}$ where $x_i$\n       are the roots of $g$\n    5) $\\delta(-x) = \\delta(x)$\n\n    Derivatives of ``k``-th order of DiracDelta have the following properties:\n\n    6) $\\delta(x, k) = 0$ for all $x \\neq 0$\n    7) $\\delta(-x, k) = -\\delta(x, k)$ for odd $k$\n    8) $\\delta(-x, k) = \\delta(x, k)$ for even $k$\n\n    Examples\n    ========\n\n    >>> from sympy import DiracDelta, diff, pi\n    >>> from sympy.abc import x, y\n\n    >>> DiracDelta(x)\n    DiracDelta(x)\n    >>> DiracDelta(1)\n    0\n    >>> DiracDelta(-1)\n    0\n    >>> DiracDelta(pi)\n    0\n    >>> DiracDelta(x - 4).subs(x, 4)\n    DiracDelta(0)\n    >>> diff(DiracDelta(x))\n    DiracDelta(x, 1)\n    >>> diff(DiracDelta(x - 1), x, 2)\n    DiracDelta(x - 1, 2)\n    >>> diff(DiracDelta(x**2 - 1), x, 2)\n    2*(2*x**2*DiracDelta(x**2 - 1, 2) + DiracDelta(x**2 - 1, 1))\n    >>> DiracDelta(3*x).is_simple(x)\n    True\n    >>> DiracDelta(x**2).is_simple(x)\n    False\n    >>> DiracDelta((x**2 - 1)*y).expand(diracdelta=True, wrt=x)\n    DiracDelta(x - 1)/(2*Abs(y)) + DiracDelta(x + 1)/(2*Abs(y))\n\n    See Also\n    ========\n\n    Heaviside\n    sympy.simplify.simplify.simplify, is_simple\n    sympy.functions.special.tensor_functions.KroneckerDelta\n\n    References\n    ==========\n\n    .. [1] https://mathworld.wolfram.com/DeltaFunction.html\n\nsympy.functions.special.delta_functions.DiracDelta.fdiff:\n  Returns the first derivative of a DiracDelta Function.\n\n        Explanation\n        ===========\n\n        The difference between ``diff()`` and ``fdiff()`` is: ``diff()`` is the\n        user-level function and ``fdiff()`` is an object method. ``fdiff()`` is\n        a convenience method available in the ``Function`` class. It returns\n        the derivative of the function without considering the chain rule.\n        ``diff(function, x)`` calls ``Function._eval_derivative`` which in turn\n        calls ``fdiff()`` internally to compute the derivative of the function.\n\n        Examples\n        ========\n\n        >>> from sympy import DiracDelta, diff\n        >>> from sympy.abc import x\n\n        >>> DiracDelta(x).fdiff()\n        DiracDelta(x, 1)\n\n        >>> DiracDelta(x, 1).fdiff()\n        DiracDelta(x, 2)\n\n        >>> DiracDelta(x**2 - 1).fdiff()\n        DiracDelta(x**2 - 1, 1)\n\n        >>> diff(DiracDelta(x, 1)).fdiff()\n        DiracDelta(x, 3)\n\n        Parameters\n        ==========\n\n        argindex : integer\n            degree of derivative\n\nsympy.functions.special.delta_functions.DiracDelta.eval:\n  Returns a simplified form or a value of DiracDelta depending on the\n        argument passed by the DiracDelta object.\n\n        Explanation\n        ===========\n\n        The ``eval()`` method is automatically called when the ``DiracDelta``\n        class is about to be instantiated and it returns either some simplified\n        instance or the unevaluated instance depending on the argument passed.\n        In other words, ``eval()`` method is not needed to be called explicitly,\n        it is being called and evaluated once the object is called.\n\n        Examples\n        ========\n\n        >>> from sympy import DiracDelta, S\n        >>> from sympy.abc import x\n\n        >>> DiracDelta(x)\n        DiracDelta(x)\n\n        >>> DiracDelta(-x, 1)\n        -DiracDelta(x, 1)\n\n        >>> DiracDelta(1)\n        0\n\n        >>> DiracDelta(5, 1)\n        0\n\n        >>> DiracDelta(0)\n        DiracDelta(0)\n\n        >>> DiracDelta(-1)\n        0\n\n        >>> DiracDelta(S.NaN)\n        nan\n\n        >>> DiracDelta(x - 100).subs(x, 5)\n        0\n\n        >>> DiracDelta(x - 100).subs(x, 100)\n        DiracDelta(0)\n\n        Parameters\n        ==========\n\n        k : integer\n            order of derivative\n\n        arg : argument passed to DiracDelta\n\nsympy.functions.special.delta_functions.DiracDelta._eval_expand_diracdelta:\n  Compute a simplified representation of the function using\n        property number 4. Pass ``wrt`` as a hint to expand the expression\n        with respect to a particular variable.\n\n        Explanation\n        ===========\n\n        ``wrt`` is:\n\n        - a variable with respect to which a DiracDelta expression will\n        get expanded.\n\n        Examples\n        ========\n\n        >>> from sympy import DiracDelta\n        >>> from sympy.abc import x, y\n\n        >>> DiracDelta(x*y).expand(diracdelta=True, wrt=x)\n        DiracDelta(x)/Abs(y)\n        >>> DiracDelta(x*y).expand(diracdelta=True, wrt=y)\n        DiracDelta(y)/Abs(x)\n\n        >>> DiracDelta(x**2 + x - 2).expand(diracdelta=True, wrt=x)\n        DiracDelta(x - 1)/3 + DiracDelta(x + 2)/3\n\n        See Also\n        ========\n\n        is_simple, Diracdelta\n\nsympy.functions.special.delta_functions.DiracDelta.is_simple:\n  Tells whether the argument(args[0]) of DiracDelta is a linear\n        expression in *x*.\n\n        Examples\n        ========\n\n        >>> from sympy import DiracDelta, cos\n        >>> from sympy.abc import x, y\n\n        >>> DiracDelta(x*y).is_simple(x)\n        True\n        >>> DiracDelta(x*y).is_simple(y)\n        True\n\n        >>> DiracDelta(x**2 + x - 2).is_simple(x)\n        False\n\n        >>> DiracDelta(cos(x)).is_simple(x)\n        False\n\n        Parameters\n        ==========\n\n        x : can be a symbol\n\n        See Also\n        ========\n\n        sympy.simplify.simplify.simplify, DiracDelta\n\nsympy.functions.special.delta_functions.DiracDelta._eval_rewrite_as_Piecewise:\n  Represents DiracDelta in a piecewise form.\n\n        Examples\n        ========\n\n        >>> from sympy import DiracDelta, Piecewise, Symbol\n        >>> x = Symbol('x')\n\n        >>> DiracDelta(x).rewrite(Piecewise)\n        Piecewise((DiracDelta(0), Eq(x, 0)), (0, True))\n\n        >>> DiracDelta(x - 5).rewrite(Piecewise)\n        Piecewise((DiracDelta(0), Eq(x, 5)), (0, True))\n\n        >>> DiracDelta(x**2 - 5).rewrite(Piecewise)\n           Piecewise((DiracDelta(0), Eq(x**2, 5)), (0, True))\n\n        >>> DiracDelta(x - 5, 4).rewrite(Piecewise)\n        DiracDelta(x - 5, 4)\n\nsympy.functions.special.delta_functions.DiracDelta._eval_rewrite_as_SingularityFunction:\n  Returns the DiracDelta expression written in the form of Singularity\n        Functions.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that DiracDelta inherits from 'DefinedFunction' is not supported by the documentation - this appears to be a plausible but unverifiable implementation detail",
      "The claim about `DefinedFunction.__new__` being decorated with `@cacheit` is not in the documentation",
      "The detailed description of `eval()` internals (e.g., `arg.args_cnc()`, checking `c[0] is S.NegativeOne`) goes beyond what the documentation states but is plausible implementation detail",
      "The claim about `deltaintegrate` being called 'after trigintegrate but before singularityintegrate' is not supported by the documentation",
      "The detailed integration formulas and `deltaintegrate` implementation details are not covered in the provided documentation",
      "The claim that `_eval_expand_diracdelta` 'skips roots with multiplicity > 1' and 'skips roots where is_real is False' are implementation details not in the documentation",
      "The LaTeX printing details are not in the provided documentation",
      "The claim about `nargs` being determined by `arity()` function inspecting `eval` method's signature is not in the documentation",
      "The description of `is_simple` using `as_poly(x)` and checking `degree() == 1` is consistent with the documented behavior but the internal mechanism is not explicitly stated",
      "The eval() method description says 'returns S.Zero if arg is nonzero' which is consistent with the documentation showing DiracDelta(1)=0, DiracDelta(-1)=0, DiracDelta(pi)=0",
      "The property references (5, 7, 8) in eval() description are consistent with the documented properties"
    ],
    "strip_verify_leakage": 0.21,
    "strip_verify_summary": "Several claims about design rationale, property references, and printing behavior were removed or rewritten as they relied on documentation not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_15",
    "repo": "sympy",
    "question": "Where does `_prep_tuple` get called and how does its output flow into the construction of `hyper` and `meijerg` instances?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "`_prep_tuple(v)` is defined in `sympy/functions/special/hyper.py`. It takes an iterable `v`, applies `unpolarify(x)` to each element, and wraps the results into a `TupleArg` object (which is a subclass of `sympy.core.containers.Tuple`, itself a subclass of `Basic`).\n\nData flow:\n1. **Called by `hyper.__new__`**: After the constructor optionally removes common parameters between `ap` and `bq` (using `Counter` intersection), it calls `_prep_tuple(ap)` and `_prep_tuple(bq)` to create the stored `TupleArg` arguments.\n2. **Called by `meijerg.__new__`**: Inside a nested helper function `tr(p)`, which first sorts each sub-parameter list using `ordered(i)`, then calls `_prep_tuple` on each of the two sub-lists, wrapping the results in a nested `TupleArg(_prep_tuple(p[0]), _prep_tuple(p[1]))` structure.\n\nThe `unpolarify` function is imported from `sympy.functions.elementary.complexes`.\n\n`TupleArg` extends `Tuple` with two additional methods: `as_leading_term` and `limit`, both of which apply their respective operations element-wise and return a new `TupleArg`.\n\nThe parent class `TupleParametersBase`, from which both `hyper` and `meijerg` inherit, extends `DefinedFunction` and sets `is_commutative = True` explicitly.",
    "rubric": [
      "States that _prep_tuple is defined in the hyper module and converts an iterable by applying unpolarify to each element and wrapping results in a TupleArg",
      "Explains that hyper.__new__ calls _prep_tuple(ap) and _prep_tuple(bq) directly after optionally removing common parameters via Counter intersection",
      "Explains that meijerg.__new__ uses a nested helper function tr(p) that sorts sub-parameter lists with ordered(), then wraps two _prep_tuple calls in TupleArg(_prep_tuple(p[0]), _prep_tuple(p[1]))",
      "Mentions that TupleArg is a subclass of Tuple (from sympy.core.containers) and adds element-wise as_leading_term and limit methods",
      "Mentions that unpolarify is imported from sympy.functions.elementary.complexes",
      "Notes that TupleParametersBase (parent of both hyper and meijerg) inherits from DefinedFunction and sets is_commutative = True"
    ],
    "key_files": [
      "sympy/functions/special/hyper.py",
      "sympy/functions/elementary/complexes.py",
      "sympy/core/containers.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/hyper.py] sympy.functions.special.hyper._prep_tuple\nTurn an iterable argument *v* into a tuple and unpolarify, since both\n    hypergeometric and meijer g-functions are unbranched in their parameters.\n\n    Examples\n    ========\n\n    >>> from sympy.functions.special.hyper import _prep_tuple\n    >>> _prep_tuple([1, 2, 3])\n    (1, 2, 3)\n    >>> _prep_tuple((4, 5))\n    (4, 5)\n    >>> _prep_tuple((7, 8, 9))\n    (7, 8, 9)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about unpolarify internals and design rationale for TupleArg/TupleParametersBase rely on documentation not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_13",
    "repo": "sympy",
    "question": "What is the class hierarchy and key architectural design of Heaviside, including its relationship to DiracDelta, its rewrite methods, and cross-module usage?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `Heaviside` class in `sympy/functions/special/delta_functions.py` is a subclass of `DefinedFunction`, which itself inherits from `Function` (defined in `sympy/core/function.py`). `DefinedFunction` is a minimal base class that overrides `__new__` with a `@cacheit` decorator, calling `cls._new_()` for caching instances.\n\n**Key architectural details:**\n\n1. **`__new__` guard for nested Heaviside**: Heaviside overrides `__new__` with a special check: if `H0` is itself a `Heaviside` instance with exactly 1 argument, it silently resets `H0` to `S.Half`. It then delegates to `super(cls, cls).__new__()` which routes through `DefinedFunction.__new__` (cached via `@cacheit`).\n\n2. **`is_real = True`**: Both `DiracDelta` and `Heaviside` set the class attribute `is_real = True` directly on the class body, making all instances report as real-valued.\n\n3. **`pargs` property**: The `pargs` property returns `args[:1]` when `args[1] is S.Half`, suppressing the default second argument. The `_print_Heaviside` method in `str.py` uses `expr.pargs` for printing.\n\n4. **`_eval_rewrite_as_Piecewise` special cases**: The method has three branches: when `H0 == 0`, it simplifies to `Piecewise((0, arg <= 0), (1, True))` (using `<=`); when `H0 == 1`, it simplifies to `Piecewise((0, arg < 0), (1, True))` (using `<`); otherwise it uses a three-piece form with `Eq(arg, 0)`. Note the default parameter is `H0=None` (not `S.Half`).\n\n5. **`_eval_rewrite_as_sign` nested Piecewise logic**: The sign rewrite constructs a nested Piecewise: `pw1` handles the case where H0 \u2260 1/2, and `pw2` wraps it so that when `Heaviside(0, H0=H0) == S.Half`, it simplifies to just `(sign(arg) + 1)/2`. This only works when `arg.is_extended_real` is True; if arg is not known to be real, it returns `None` (unchanged).\n\n6. **Complex argument validation**: Both `eval()` and `DiracDelta.eval()` use `fuzzy_not(im(arg).is_zero)` to raise a `ValueError` for arguments known to have nonzero imaginary parts. This uses three-valued (fuzzy) logic from `sympy.core.logic`.\n\n7. **Cross-module usage**: `Max.fdiff()` returns `Heaviside(self.args[argindex] - self.args[1 - argindex])` for two-argument case, and `Min.fdiff()` returns `Heaviside(self.args[1-argindex] - self.args[argindex])`. `sign._eval_rewrite_as_Heaviside` returns `Heaviside(arg) * 2 - 1`.\n\n8. **`DiracDelta` shares the same parent class**: Both `DiracDelta` and `Heaviside` extend `DefinedFunction`. DiracDelta is the return value of `Heaviside.fdiff(argindex=1)` \u2014 specifically `DiracDelta(self.args[0])`.\n\n9. **`SingularityFunction` rewrite**: `Heaviside(0)` rewrites to `SingularityFunction(0, 0, 0)`, and for single-variable expressions, it solves for the root and creates `SingularityFunction(x, root, 0)`.",
    "rubric": [
      "Identifies that Heaviside and DiracDelta both subclass DefinedFunction, which inherits from Function, and that DefinedFunction uses @cacheit on __new__",
      "Explains the __new__ guard that resets H0 to S.Half when H0 is itself a Heaviside instance with exactly 1 argument",
      "Mentions that both DiracDelta and Heaviside set is_real = True as a class attribute",
      "Describes the pargs property returning args[:1] when the second argument is S.Half, used by printers to suppress the default value",
      "Explains _eval_rewrite_as_Piecewise with its three branches: H0==0 uses <=, H0==1 uses <, and the default uses a three-piece form with Eq(arg, 0)",
      "Describes _eval_rewrite_as_sign's nested Piecewise logic and that it returns None when arg is not known to be extended_real",
      "Notes that both eval() methods use fuzzy_not(im(arg).is_zero) to raise ValueError for known-complex arguments",
      "Identifies cross-module connections: Max.fdiff and Min.fdiff return Heaviside expressions, sign._eval_rewrite_as_Heaviside returns Heaviside(arg)*2-1",
      "Notes that Heaviside.fdiff returns DiracDelta(self.args[0])"
    ],
    "key_files": [
      "sympy/functions/special/delta_functions.py",
      "sympy/core/function.py",
      "sympy/functions/elementary/miscellaneous.py",
      "sympy/functions/elementary/complexes.py",
      "sympy/printing/latex.py",
      "sympy/printing/pretty/pretty.py",
      "sympy/printing/str.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/delta_functions.py] sympy.functions.special.delta_functions.Heaviside\nsympy.functions.special.delta_functions.Heaviside:\n  Heaviside step function.\n\n    Explanation\n    ===========\n\n    The Heaviside step function has the following properties:\n\n    1) $\\frac{d}{d x} \\theta(x) = \\delta(x)$\n    2) $\\theta(x) = \\begin{cases} 0 & \\text{for}\\: x < 0 \\\\ \\frac{1}{2} &\n       \\text{for}\\: x = 0 \\\\1 & \\text{for}\\: x > 0 \\end{cases}$\n    3) $\\frac{d}{d x} \\max(x, 0) = \\theta(x)$\n\n    Heaviside(x) is printed as $\\theta(x)$ with the SymPy LaTeX printer.\n\n    The value at 0 is set differently in different fields. SymPy uses 1/2,\n    which is a convention from electronics and signal processing, and is\n    consistent with solving improper integrals by Fourier transform and\n    convolution.\n\n    To specify a different value of Heaviside at ``x=0``, a second argument\n    can be given. Using ``Heaviside(x, nan)`` gives an expression that will\n    evaluate to nan for x=0.\n\n    .. versionchanged:: 1.9 ``Heaviside(0)`` now returns 1/2 (before: undefined)\n\n    Examples\n    ========\n\n    >>> from sympy import Heaviside, nan\n    >>> from sympy.abc import x\n    >>> Heaviside(9)\n    1\n    >>> Heaviside(-9)\n    0\n    >>> Heaviside(0)\n    1/2\n    >>> Heaviside(0, nan)\n    nan\n    >>> (Heaviside(x) + 1).replace(Heaviside(x), Heaviside(x, 1))\n    Heaviside(x, 1) + 1\n\n    See Also\n    ========\n\n    DiracDelta\n\n    References\n    ==========\n\n    .. [1] https://mathworld.wolfram.com/HeavisideStepFunction.html\n    .. [2] https://dlmf.nist.gov/1.16#iv\n\nsympy.functions.special.delta_functions.Heaviside.fdiff:\n  Returns the first derivative of a Heaviside Function.\n\n        Examples\n        ========\n\n        >>> from sympy import Heaviside, diff\n        >>> from sympy.abc import x\n\n        >>> Heaviside(x).fdiff()\n        DiracDelta(x)\n\n        >>> Heaviside(x**2 - 1).fdiff()\n        DiracDelta(x**2 - 1)\n\n        >>> diff(Heaviside(x)).fdiff()\n        DiracDelta(x, 1)\n\n        Parameters\n        ==========\n\n        argindex : integer\n            order of derivative\n\nsympy.functions.special.delta_functions.Heaviside.pargs:\n  Args without default S.Half\n\nsympy.functions.special.delta_functions.Heaviside.eval:\n  Returns a simplified form or a value of Heaviside depending on the\n        argument passed by the Heaviside object.\n\n        Explanation\n        ===========\n\n        The ``eval()`` method is automatically called when the ``Heaviside``\n        class is about to be instantiated and it returns either some simplified\n        instance or the unevaluated instance depending on the argument passed.\n        In other words, ``eval()`` method is not needed to be called explicitly,\n        it is being called and evaluated once the object is called.\n\n        Examples\n        ========\n\n        >>> from sympy import Heaviside, S\n        >>> from sympy.abc import x\n\n        >>> Heaviside(x)\n        Heaviside(x)\n\n        >>> Heaviside(19)\n        1\n\n        >>> Heaviside(0)\n        1/2\n\n        >>> Heaviside(0, 1)\n        1\n\n        >>> Heaviside(-5)\n        0\n\n        >>> Heaviside(S.NaN)\n        nan\n\n        >>> Heaviside(x - 100).subs(x, 5)\n        0\n\n        >>> Heaviside(x - 100).subs(x, 105)\n        1\n\n        Parameters\n        ==========\n\n        arg : argument passed by Heaviside object\n\n        H0 : value of Heaviside(0)\n\nsympy.functions.special.delta_functions.Heaviside._eval_rewrite_as_Piecewise:\n  Represents Heaviside in a Piecewise form.\n\n        Examples\n        ========\n\n        >>> from sympy import Heaviside, Piecewise, Symbol, nan\n        >>> x = Symbol('x')\n\n        >>> Heaviside(x).rewrite(Piecewise)\n        Piecewise((0, x < 0), (1/2, Eq(x, 0)), (1, True))\n\n        >>> Heaviside(x,nan).rewrite(Piecewise)\n        Piecewise((0, x < 0), (nan, Eq(x, 0)), (1, True))\n\n        >>> Heaviside(x - 5).rewrite(Piecewise)\n        Piecewise((0, x < 5), (1/2, Eq(x, 5)), (1, True))\n\n        >>> Heaviside(x**2 - 1).rewrite(Piecewise)\n        Piecewise((0, x**2 < 1), (1/2, Eq(x**2, 1)), (1, True))\n\nsympy.functions.special.delta_functions.Heaviside._eval_rewrite_as_sign:\n  Represents the Heaviside function in the form of sign function.\n\n        Explanation\n        ===========\n\n        The value of Heaviside(0) must be 1/2 for rewriting as sign to be\n        strictly equivalent. For easier usage, we also allow this rewriting\n        when Heaviside(0) is undefined.\n\n        Examples\n        ========\n\n        >>> from sympy import Heaviside, Symbol, sign, nan\n        >>> x = Symbol('x', real=True)\n        >>> y = Symbol('y')\n\n        >>> Heaviside(x).rewrite(sign)\n        sign(x)/2 + 1/2\n\n        >>> Heaviside(x, 0).rewrite(sign)\n        Piecewise((sign(x)/2 + 1/2, Ne(x, 0)), (0, True))\n\n        >>> Heaviside(x, nan).rewrite(sign)\n        Piecewise((sign(x)/2 + 1/2, Ne(x, 0)), (nan, True))\n\n        >>> Heaviside(x - 2).rewrite(sign)\n        sign(x - 2)/2 + 1/2\n\n        >>> Heaviside(x**2 - 2*x + 1).rewrite(sign)\n        sign(x**2 - 2*x + 1)/2 + 1/2\n\n        >>> Heaviside(y).rewrite(sign)\n        Heaviside(y)\n\n        >>> Heaviside(y**2 - 2*y + 1).rewrite(sign)\n        Heaviside(y**2 - 2*y + 1)\n\n        See Also\n        ========\n\n        sign\n\nsympy.functions.special.delta_functions.Heaviside._eval_rewrite_as_SingularityFunction:\n  Returns the Heaviside expression written in the form of Singularity\n        Functions.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims Heaviside is a subclass of 'DefinedFunction' - this is not mentioned in the documentation and cannot be verified from it, though it's plausible code-level detail.",
      "The answer mentions specific line numbers (e.g., line 483-486, line 560, etc.) that cannot be verified from the documentation alone - these are plausible but unverifiable.",
      "The answer states '_eval_rewrite_as_Piecewise' has 'default parameter H0=None (not S.Half)' - the documentation shows examples consistent with S.Half default behavior but doesn't explicitly state the parameter default.",
      "The answer claims complex argument validation raises ValueError for arguments with nonzero imaginary parts - not mentioned in documentation but plausible.",
      "The answer's claim about cross-module usage (Max.fdiff, Min.fdiff, Abs._eval_rewrite_as_Heaviside, sign._eval_rewrite_as_Heaviside, log) cannot be verified from this documentation chunk.",
      "The answer states 'is_real = True' is set as a class attribute - not mentioned in documentation but plausible.",
      "The documentation shows Heaviside(0) returns 1/2 which is consistent with the answer's claims about pargs suppressing S.Half default.",
      "The fdiff documentation confirms DiracDelta is returned, consistent with answer's claim #9.",
      "The _eval_rewrite_as_sign documentation confirms it works only for real arguments (x = Symbol('x', real=True) works, y = Symbol('y') doesn't), consistent with claim #6 about arg.is_extended_real."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed rationale-only explanations and claims about code not shown (PyTorch printer, log function).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_14",
    "repo": "sympy",
    "question": "Why does `multigamma`'s `eval` method always return a value, and how does that affect its `fdiff`, `_eval_is_real`, and `_eval_conjugate` methods?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `multigamma` class's `eval` classmethod always returns a non-None value (either raises ValueError for invalid inputs or returns the expanded Product expression with `.doit()` called). This means `multigamma` never persists as an unevaluated object during normal evaluation.\n\nThe key aspects of this design:\n\n1. **Always-evaluate pattern via `.doit()`**: The `eval` method constructs a `Product(gamma(x + (1-k)/2), (k, 1, p))` and immediately calls `.doit()` on it. When `p` is a concrete positive integer (e.g., with `dif < 100` in `Product._eval_product`), this expands the product into explicit gamma function multiplications via `_eval_product_direct`. When `p` is symbolic, `Product.doit()` returns the unevaluated `Product` object.\n\n2. **Dual-check in `_eval_is_real`**: The method uses two separate checks for integer-like values \u2014 `y.is_integer` (for symbolic assumptions on expressions) and `intlike(y)` (for numeric float values like `-1.0`). The `intlike` helper function uses `as_int(n, strict=False)` which can convert floats to ints. This covers cases where `2*x` is a float like `-2.0` that `intlike` can detect as integer-like.\n\n3. **`evaluate=False` support for assumptions**: Since `eval` always returns, the `_eval_is_real`, `_eval_conjugate`, and `fdiff` methods are only reachable when constructing with `evaluate=False`.\n\n4. **Apparent argindex bug in `fdiff`**: The `fdiff` method uses `argindex=2` as default and implements the derivative formula for that index, but the formula `self.func(x, p)*Sum(polygamma(0, x + (1-k)/2), (k, 1, p))` is actually the derivative with respect to `x` (the first argument), not `p` (the second). For `argindex=1`, it raises `ArgumentIndexError`, which causes `_eval_derivative` to fall back to the generic `Function.fdiff` creating a Subs/Derivative expression. Since `multigamma` always evaluates in practice, this apparent mismatch doesn't affect normal usage \u2014 diff always operates on the expanded gamma product form.\n\n5. **Inherits from `DefinedFunction`**: This base class (defined in `sympy/core/function.py`) applies `@cacheit` on `__new__` and routes directly to `cls._new_()`, skipping the `UndefinedFunction` routing logic in `Function.__new__`.",
    "rubric": [
      "Explains that eval always returns because it constructs a Product(gamma(x + (1-k)/2), (k, 1, p)) and calls .doit() on it, so it never returns None",
      "Notes that because eval never returns None, multigamma never persists as an unevaluated object in normal usage, so _eval_is_real, _eval_conjugate, and fdiff are only reachable via evaluate=False",
      "Identifies the dual check in _eval_is_real: y.is_integer for symbolic assumptions and intlike(y) for numeric float values, where intlike uses as_int with strict=False",
      "Points out the apparent argindex issue in fdiff: argindex=2 is the default but the derivative formula (involving polygamma) is actually with respect to x (the first argument), while argindex=1 raises ArgumentIndexError",
      "Mentions that multigamma inherits from DefinedFunction, which applies @cacheit and routes through cls._new_() bypassing UndefinedFunction logic"
    ],
    "key_files": [
      "sympy/functions/special/gamma_functions.py",
      "sympy/core/function.py",
      "sympy/concrete/products.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/gamma_functions.py] sympy.functions.special.gamma_functions.multigamma\nsympy.functions.special.gamma_functions.multigamma:\n  The multivariate gamma function is a generalization of the gamma function\n\n    .. math::\n        \\Gamma_p(z) = \\pi^{p(p-1)/4}\\prod_{k=1}^p \\Gamma[z + (1 - k)/2].\n\n    In a special case, ``multigamma(x, 1) = gamma(x)``.\n\n    Examples\n    ========\n\n    >>> from sympy import S, multigamma\n    >>> from sympy import Symbol\n    >>> x = Symbol('x')\n    >>> p = Symbol('p', positive=True, integer=True)\n\n    >>> multigamma(x, p)\n    pi**(p*(p - 1)/4)*Product(gamma(-_k/2 + x + 1/2), (_k, 1, p))\n\n    Several special values are known:\n\n    >>> multigamma(1, 1)\n    1\n    >>> multigamma(4, 1)\n    6\n    >>> multigamma(S(3)/2, 1)\n    sqrt(pi)/2\n\n    Writing ``multigamma`` in terms of the ``gamma`` function:\n\n    >>> multigamma(x, 1)\n    gamma(x)\n\n    >>> multigamma(x, 2)\n    sqrt(pi)*gamma(x)*gamma(x - 1/2)\n\n    >>> multigamma(x, 3)\n    pi**(3/2)*gamma(x)*gamma(x - 1)*gamma(x - 1/2)\n\n    Parameters\n    ==========\n\n    p : order or dimension of the multivariate gamma function\n\n    See Also\n    ========\n\n    gamma, lowergamma, uppergamma, polygamma, loggamma, digamma, trigamma,\n    sympy.functions.special.beta_functions.beta\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Multivariate_gamma_function",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer makes extensive claims about internal implementation details (eval classmethod behavior, _eval_is_real, fdiff, DefinedFunction inheritance, etc.) that are not present in the documentation and cannot be verified from it alone.",
      "The claim about 'apparent argindex bug in fdiff' is an implementation detail that cannot be verified from the documentation.",
      "The claim about 'DefinedFunction' base class with '@cacheit' on __new__ is not mentioned in the documentation.",
      "However, the claims that are verifiable against the documentation are consistent: multigamma(x, 1) = gamma(x), the formula matches, the examples match, and the special case behavior is correctly described."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed test references and softened design rationale language.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_17",
    "repo": "sympy",
    "question": "What is `jacobi_normalized` and how does its architecture differ from the `jacobi` class and other orthogonal polynomial classes?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`jacobi_normalized` is a plain Python function (not a class) defined in `sympy/functions/special/polynomials.py`. It computes the normalized Jacobi polynomial by dividing the unnormalized `jacobi(n, a, b, x)` by the square root of a normalization factor `nfactor`.\n\nThe normalization factor is computed as:\n```python\nnfactor = (S(2)**(a + b + 1) * (gamma(n + a + 1) * gamma(n + b + 1))\n           / (2*n + a + b + 1) / (factorial(n) * gamma(n + a + b + 1)))\n```\nand the function returns `jacobi(n, a, b, x) / sqrt(nfactor)`.\n\nKey architectural details:\n\n1. **`jacobi_normalized` is a regular function, not a class** \u2014 unlike `jacobi`, `gegenbauer`, `chebyshevt`, etc., which are all classes inheriting from `OrthogonalPolynomial`. This means `jacobi_normalized` does not participate in SymPy's expression evaluation framework (no `eval` classmethod, no `fdiff`, no rewrite rules).\n\n2. **`jacobi` class inherits from `OrthogonalPolynomial`, which inherits from `DefinedFunction`, which inherits from `Function`**. `DefinedFunction` (defined in `sympy/core/function.py`) applies the `@cacheit` decorator to its `__new__` method, meaning all `jacobi(...)` instances are cached.\n\n3. **Neither `jacobi` nor `gegenbauer` define `_ortho_poly`** \u2014 The base class `OrthogonalPolynomial._eval_at_order` references `cls._ortho_poly`, but `jacobi` and `gegenbauer` don't set this attribute. Only `chebyshevt`, `chebyshevu`, `legendre`, `hermite`, `hermite_prob`, and `laguerre` do. Instead, `jacobi` and `gegenbauer` handle numeric `n` evaluation directly in their `eval()` methods by calling `jacobi_poly(n, a, b, x)` and `gegenbauer_poly(n, a, x)` respectively.\n\n4. **`jacobi.eval()` simplifies to other orthogonal polynomials when `a == b`**: when `a == b == -1/2`, it returns in terms of `chebyshevt`; when `a == b == 0`, it returns `legendre`; when `a == b == 1/2`, it returns in terms of `chebyshevu`; otherwise `a == b` yields `gegenbauer`. When `b == -a`, it returns in terms of `assoc_legendre`.\n\n5. **`jacobi` overrides `_eval_conjugate`** from the base `OrthogonalPolynomial` class. The base version unpacks only 2 args `(self.args[0], self.args[1].conjugate())`, while `jacobi` unpacks 4 args `(n, a.conjugate(), b.conjugate(), x.conjugate())` to correctly conjugate all parameters.\n\n6. **The low-level polynomial generation** happens in `dup_jacobi` in `sympy/polys/orthopolys.py`, which uses a three-term recurrence relation. `jacobi_poly` calls `named_poly(n, dup_jacobi, None, \"Jacobi polynomial\", (x, a, b), polys)`, where `named_poly` (in `sympy/polys/polytools.py`) constructs the appropriate domain, generates coefficients via the low-level function, and wraps the result.\n\n7. **`jacobi.fdiff` supports differentiation with respect to all four arguments** (though `argindex=1` for `n` raises `ArgumentIndexError`). Differentiation with respect to `x` (argindex=4) yields `S.Half * (a + b + n + 1) * jacobi(n - 1, a + 1, b + 1, x)`. Differentiation with respect to `a` and `b` (argindex 2 and 3) uses `Sum` expressions involving Jacobi polynomials.",
    "rubric": [
      "Identifies that `jacobi_normalized` is a plain Python function (not a class), unlike `jacobi`, `gegenbauer`, `chebyshevt`, etc.",
      "Explains that it returns `jacobi(n, a, b, x) / sqrt(nfactor)` where nfactor involves `2**(a+b+1)`, gamma functions, factorial, and `(2*n + a + b + 1)`",
      "Notes that the orthogonal polynomial classes (`jacobi`, `gegenbauer`, etc.) inherit from `OrthogonalPolynomial`, which inherits from `DefinedFunction` (which inherits from `Function` and applies `@cacheit`)",
      "Explains that `jacobi` and `gegenbauer` do NOT define `_ortho_poly` (unlike `chebyshevt`, `chebyshevu`, `legendre`, `hermite`, `hermite_prob`, `laguerre`) and instead call `jacobi_poly`/`gegenbauer_poly` directly in their `eval()` methods",
      "Mentions that `jacobi.eval()` simplifies to other polynomial types based on the relationship between a and b (e.g., a==b==0 gives legendre, a==b==-1/2 gives chebyshevt, a==b==1/2 gives chebyshevu, a==b gives gegenbauer)",
      "Notes that `jacobi` overrides `_eval_conjugate` to conjugate all four args (n, a, b, x) rather than the base class version which only handles two args",
      "Mentions that `jacobi.fdiff` supports differentiation with respect to x (argindex=4), a (argindex=2), and b (argindex=3), with x-differentiation yielding `S.Half * (a + b + n + 1) * jacobi(n-1, a+1, b+1, x)`"
    ],
    "key_files": [
      "sympy/functions/special/polynomials.py",
      "sympy/polys/orthopolys.py",
      "sympy/core/function.py",
      "sympy/polys/polytools.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/polynomials.py] sympy.functions.special.polynomials.jacobi_normalized\nJacobi polynomial $P_n^{\\left(\\alpha, \\beta\\right)}(x)$.\n\n    Explanation\n    ===========\n\n    ``jacobi_normalized(n, alpha, beta, x)`` gives the $n$th\n    Jacobi polynomial in $x$, $P_n^{\\left(\\alpha, \\beta\\right)}(x)$.\n\n    The Jacobi polynomials are orthogonal on $[-1, 1]$ with respect\n    to the weight $\\left(1-x\\right)^\\alpha \\left(1+x\\right)^\\beta$.\n\n    This functions returns the polynomials normilzed:\n\n    .. math::\n\n        \\int_{-1}^{1}\n          P_m^{\\left(\\alpha, \\beta\\right)}(x)\n          P_n^{\\left(\\alpha, \\beta\\right)}(x)\n          (1-x)^{\\alpha} (1+x)^{\\beta} \\mathrm{d}x\n        = \\delta_{m,n}\n\n    Examples\n    ========\n\n    >>> from sympy import jacobi_normalized\n    >>> from sympy.abc import n,a,b,x\n\n    >>> jacobi_normalized(n, a, b, x)\n    jacobi(n, a, b, x)/sqrt(2**(a + b + 1)*gamma(a + n + 1)*gamma(b + n + 1)/((a + b + 2*n + 1)*factorial(n)*gamma(a + b + n + 1)))\n\n    Parameters\n    ==========\n\n    n : integer degree of polynomial\n\n    a : alpha value\n\n    b : beta value\n\n    x : symbol\n\n    See Also\n    ========\n\n    gegenbauer,\n    chebyshevt_root, chebyshevu, chebyshevu_root,\n    legendre, assoc_legendre,\n    hermite, hermite_prob,\n    laguerre, assoc_laguerre,\n    sympy.polys.orthopolys.jacobi_poly,\n    sympy.polys.orthopolys.gegenbauer_poly\n    sympy.polys.orthopolys.chebyshevt_poly\n    sympy.polys.orthopolys.chebyshevu_poly\n    sympy.polys.orthopolys.hermite_poly\n    sympy.polys.orthopolys.legendre_poly\n    sympy.polys.orthopolys.laguerre_poly\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Jacobi_polynomials\n    .. [2] https://mathworld.wolfram.com/JacobiPolynomial.html\n    .. [3] https://functions.wolfram.com/Polynomials/JacobiP/",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only minor line number references and one partial description of named_poly needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_18",
    "repo": "sympy",
    "question": "Why does the `laguerre` class handle negative indices via an identity transformation instead of raising an error like `assoc_laguerre` does?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `laguerre` class in `sympy/functions/special/polynomials.py` handles negative indices by returning `exp(x)*laguerre(-n - 1, -x)` in both symbolic and numeric evaluation paths, rather than raising an error. This contrasts with `assoc_laguerre` which raises a `ValueError` for negative numeric n.\n\nThe key implementation details:\n\n1. **Negative index support via identity transformation**: When `n` is a concrete negative number, `laguerre.eval()` returns `exp(x)*laguerre(-n - 1, -x)`, recursively calling itself with a transformed positive index. For symbolic negative `n`, it uses `n.could_extract_minus_sign()` with an additional guard `not(-n - 1).could_extract_minus_sign()` to avoid infinite recursion when the sign cannot be definitively determined.\n\n2. **Dummy variable substitution pattern for numeric evaluation**: The `_eval_at_order` method in `OrthogonalPolynomial` first evaluates the polynomial at a module-level `_x = Dummy('x')` and then substitutes the actual argument via `.subs(_x, x)`.\n\n3. **Separation between `laguerre` and `assoc_laguerre`**: When `assoc_laguerre` is called with `alpha=0`, it immediately delegates to `laguerre(n, x)`. However, `assoc_laguerre` does NOT define `_ortho_poly` and instead directly calls `laguerre_poly(n, x, alpha)` for numeric evaluation.\n\n4. **Caching via `DefinedFunction`**: The `laguerre` class inherits from `OrthogonalPolynomial \u2192 DefinedFunction`, whose `__new__` is decorated with `@cacheit`. This means repeated calls like `laguerre(3, x)` return cached results without re-evaluation.\n\n5. **Low-level recurrence in `dup_laguerre`**: The actual polynomial generation uses a three-term recurrence implemented with dense polynomial arithmetic (`dup_mul`, `dup_sub`). It starts with `m2=[K.zero], m1=[K.one]` and iterates: `a = dup_mul(m1, [-K.one/K(i), (alpha-K.one)/K(i) + K(2)], K)`, `b = dup_mul_ground(m2, (alpha-K.one)/K(i) + K.one, K)`.\n\n6. **`_eval_rewrite_as_polynomial`**: This method simply delegates to `_eval_rewrite_as_Sum`, which uses `RisingFactorial(-n, k) / factorial(k)**2 * x**k` as the summation kernel.",
    "rubric": [
      "Explains that laguerre uses the identity `exp(x)*laguerre(-n - 1, -x)` to transform negative n into a positive-index evaluation, rather than raising a ValueError",
      "Notes that assoc_laguerre raises a ValueError for negative numeric n instead of applying a similar transformation",
      "Describes the infinite-recursion guard for symbolic n: `n.could_extract_minus_sign() and not(-n - 1).could_extract_minus_sign()`",
      "Mentions that both the symbolic path (not n.is_Number) and the numeric path (n.is_negative) in laguerre.eval() apply the same exp(x)*laguerre(-n-1, -x) identity",
      "Notes that assoc_laguerre with alpha=0 delegates to laguerre(n, x), so the negative-index handling only lives in the laguerre class",
      "Mentions that numeric evaluation for non-negative n uses _eval_at_order which calls laguerre_poly via _ortho_poly, while assoc_laguerre calls laguerre_poly directly (no _ortho_poly attribute)"
    ],
    "key_files": [
      "sympy/functions/special/polynomials.py",
      "sympy/polys/orthopolys.py",
      "sympy/polys/polytools.py",
      "sympy/core/function.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/polynomials.py] sympy.functions.special.polynomials.laguerre\nsympy.functions.special.polynomials.laguerre:\n  Returns the $n$th Laguerre polynomial in $x$, $L_n(x)$.\n\n    Examples\n    ========\n\n    >>> from sympy import laguerre, diff\n    >>> from sympy.abc import x, n\n    >>> laguerre(0, x)\n    1\n    >>> laguerre(1, x)\n    1 - x\n    >>> laguerre(2, x)\n    x**2/2 - 2*x + 1\n    >>> laguerre(3, x)\n    -x**3/6 + 3*x**2/2 - 3*x + 1\n\n    >>> laguerre(n, x)\n    laguerre(n, x)\n\n    >>> diff(laguerre(n, x), x)\n    -assoc_laguerre(n - 1, 1, x)\n\n    Parameters\n    ==========\n\n    n : int\n        Degree of Laguerre polynomial. Must be `n \\ge 0`.\n\n    See Also\n    ========\n\n    jacobi, gegenbauer,\n    chebyshevt, chebyshevt_root, chebyshevu, chebyshevu_root,\n    legendre, assoc_legendre,\n    hermite, hermite_prob,\n    assoc_laguerre,\n    sympy.polys.orthopolys.jacobi_poly\n    sympy.polys.orthopolys.gegenbauer_poly\n    sympy.polys.orthopolys.chebyshevt_poly\n    sympy.polys.orthopolys.chebyshevu_poly\n    sympy.polys.orthopolys.hermite_poly\n    sympy.polys.orthopolys.hermite_prob_poly\n    sympy.polys.orthopolys.legendre_poly\n    sympy.polys.orthopolys.laguerre_poly\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Laguerre_polynomial\n    .. [2] https://mathworld.wolfram.com/LaguerrePolynomial.html\n    .. [3] https://functions.wolfram.com/Polynomials/LaguerreL/\n    .. [4] https://functions.wolfram.com/Polynomials/LaguerreL3/",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states the parameter n must satisfy n >= 0, but the answer claims laguerre handles negative indices through an identity L_{-n}(x) = exp(x) * L_{n-1}(-x). This contradicts the documented constraint, though it may reflect actual implementation behavior not covered in the docstring.",
      "Most of the answer's claims (about negative index handling, dummy variable substitution, caching, dup_laguerre recurrence, _eval_rewrite_as_polynomial, assoc_laguerre delegation) go far beyond what the documentation provides and appear to be based on source code inspection rather than the given documentation chunk.",
      "The answer's claim about diff(laguerre(n, x), x) returning -assoc_laguerre(n - 1, 1, x) is consistent with the documentation's example.",
      "The answer's description of laguerre(2, x) and laguerre(3, x) behavior is consistent with the documented examples."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale explanations and the 'backwards compatibility' claim rely on documentation/comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_19",
    "repo": "sympy",
    "question": "Where is the `Ynm` spherical harmonics class defined and used across the codebase, and how do other modules interact with it?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `Ynm` class (spherical harmonics) is defined in `sympy/functions/special/spherical_harmonics.py` and inherits from `DefinedFunction` (located in `sympy/core/function.py`), which itself inherits from `Function`. The `DefinedFunction.__new__` method uses `@cacheit` decorator and delegates to `cls._new_()` from the `Function` class.\n\n**Key locations where `Ynm` is used throughout the codebase:**\n\n1. **`sympy/physics/hydrogen.py`**: The `Y_lm(l, m, phi, theta)` function is a thin wrapper that calls `Ynm(l, m, theta, phi).expand(func=True)` \u2014 note the swapped argument order (phi/theta parameters are swapped between Y_lm's signature and the Ynm call). Similarly, `Z_lm(l, m, phi, theta)` calls `Znm(l, m, theta, phi).expand(func=True)`. The `Psi_nlm` function returns `R_nl(n, l, r, Z)*Ynm(l, m, theta, phi).expand(func=True)`.\n\n2. **`sympy/physics/wigner.py`**: The `dot_rot_grad_Ynm(j, p, l, m, theta, phi)` function returns a `Sum` expression involving `Ynm(k, m+p, theta, phi)` multiplied by an `alpha` coefficient that uses `Wigner3j` symbols.\n\n3. **`sympy/printing/latex.py`**: A `_print_Ynm` method formats Ynm in LaTeX notation.\n\n**Implementation details:**\n\n- The `_eval_expand_func` method performs a substitution `sqrt(-cos(theta)**2 + 1) -> sin(theta)` after expanding.\n- The `_eval_evalf` method uses mpmath's `ctx.spherharm(n, m, theta, phi)` via `local_workprec(prec)` context manager, then converts the result using `Expr._from_mpmath(res, prec)`, which handles both real (mpf) and complex (mpc) results.\n- `_eval_conjugate` returns `S.NegativeOne**m * self.func(n, -m, theta, phi)`, which then triggers `eval()` to transform the negative m back, producing the `(-1)**(2*m)*exp(-2*I*m*phi)*Ynm(n, m, theta, phi)` form.\n- The `as_real_imag` method splits the spherical harmonic into real and imaginary parts using `cos(m*phi)` and `sin(m*phi)` multiplied by the normalization factor and associated Legendre polynomial.\n- `Ynm_c` is a plain function (not a class), implemented simply as `return conjugate(Ynm(n, m, theta, phi))`.\n- The `Znm.eval` classmethod uses `Ynm` and `Ynm_c` differently based on the sign of m: `(Ynm + Ynm_c)/sqrt(2)` for positive m, plain `Ynm` for zero, and `(Ynm - Ynm_c)/(sqrt(2)*I)` for negative m.\n- The `_eval_expand_func` relies on `assoc_legendre` (from `sympy/functions/special/polynomials.py`), whose `_eval_at_order` method computes the associated Legendre polynomial by differentiating `legendre_poly(n, _x, polys=True)` m times and multiplying by `(-1)**m * (1 - x**2)**(m/2)`.\n- `Ynm` supports rewriting as `polynomial` (which calls `expand(func=True)`), as `sin` (which delegates to `rewrite(cos)`), and as `cos` (which uses both `simplify` and `trigsimp`).",
    "rubric": [
      "Identifies that Ynm is defined in the spherical_harmonics module and inherits from DefinedFunction",
      "Mentions that sympy/physics/hydrogen.py uses Ynm: Y_lm wraps Ynm with swapped phi/theta argument order, and Psi_nlm uses Ynm combined with R_nl",
      "Mentions that sympy/physics/wigner.py uses Ynm in dot_rot_grad_Ynm via a Sum expression with Wigner3j symbols",
      "Mentions the LaTeX printer has a _print_Ynm method for rendering Ynm in LaTeX notation",
      "Explains that Ynm_c is a plain function (not a class) that returns conjugate(Ynm(...))",
      "Explains that Znm.eval uses Ynm and Ynm_c differently based on the sign of m (positive, zero, or negative)",
      "Mentions key implementation details like _eval_expand_func using assoc_legendre or _eval_evalf using mpmath's spherharm",
      "Mentions _eval_conjugate returns (-1)**m * Ynm(n, -m, theta, phi)"
    ],
    "key_files": [
      "sympy/functions/special/spherical_harmonics.py",
      "sympy/core/function.py",
      "sympy/physics/hydrogen.py",
      "sympy/physics/wigner.py",
      "sympy/functions/special/polynomials.py",
      "sympy/printing/latex.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/spherical_harmonics.py] sympy.functions.special.spherical_harmonics.Ynm\nsympy.functions.special.spherical_harmonics.Ynm:\n  Spherical harmonics defined as\n\n    .. math::\n        Y_n^m(\\theta, \\varphi) := \\sqrt{\\frac{(2n+1)(n-m)!}{4\\pi(n+m)!}}\n                                  \\exp(i m \\varphi)\n                                  \\mathrm{P}_n^m\\left(\\cos(\\theta)\\right)\n\n    Explanation\n    ===========\n\n    ``Ynm()`` gives the spherical harmonic function of order $n$ and $m$\n    in $\\theta$ and $\\varphi$, $Y_n^m(\\theta, \\varphi)$. The four\n    parameters are as follows: $n \\geq 0$ an integer and $m$ an integer\n    such that $-n \\leq m \\leq n$ holds. The two angles are real-valued\n    with $\\theta \\in [0, \\pi]$ and $\\varphi \\in [0, 2\\pi]$.\n\n    Examples\n    ========\n\n    >>> from sympy import Ynm, Symbol, simplify\n    >>> from sympy.abc import n,m\n    >>> theta = Symbol(\"theta\")\n    >>> phi = Symbol(\"phi\")\n\n    >>> Ynm(n, m, theta, phi)\n    Ynm(n, m, theta, phi)\n\n    Several symmetries are known, for the order:\n\n    >>> Ynm(n, -m, theta, phi)\n    (-1)**m*exp(-2*I*m*phi)*Ynm(n, m, theta, phi)\n\n    As well as for the angles:\n\n    >>> Ynm(n, m, -theta, phi)\n    Ynm(n, m, theta, phi)\n\n    >>> Ynm(n, m, theta, -phi)\n    exp(-2*I*m*phi)*Ynm(n, m, theta, phi)\n\n    For specific integers $n$ and $m$ we can evaluate the harmonics\n    to more useful expressions:\n\n    >>> simplify(Ynm(0, 0, theta, phi).expand(func=True))\n    1/(2*sqrt(pi))\n\n    >>> simplify(Ynm(1, -1, theta, phi).expand(func=True))\n    sqrt(6)*exp(-I*phi)*sin(theta)/(4*sqrt(pi))\n\n    >>> simplify(Ynm(1, 0, theta, phi).expand(func=True))\n    sqrt(3)*cos(theta)/(2*sqrt(pi))\n\n    >>> simplify(Ynm(1, 1, theta, phi).expand(func=True))\n    -sqrt(6)*exp(I*phi)*sin(theta)/(4*sqrt(pi))\n\n    >>> simplify(Ynm(2, -2, theta, phi).expand(func=True))\n    sqrt(30)*exp(-2*I*phi)*sin(theta)**2/(8*sqrt(pi))\n\n    >>> simplify(Ynm(2, -1, theta, phi).expand(func=True))\n    sqrt(30)*exp(-I*phi)*sin(2*theta)/(8*sqrt(pi))\n\n    >>> simplify(Ynm(2, 0, theta, phi).expand(func=True))\n    sqrt(5)*(3*cos(theta)**2 - 1)/(4*sqrt(pi))\n\n    >>> simplify(Ynm(2, 1, theta, phi).expand(func=True))\n    -sqrt(30)*exp(I*phi)*sin(2*theta)/(8*sqrt(pi))\n\n    >>> simplify(Ynm(2, 2, theta, phi).expand(func=True))\n    sqrt(30)*exp(2*I*phi)*sin(theta)**2/(8*sqrt(pi))\n\n    We can differentiate the functions with respect\n    to both angles:\n\n    >>> from sympy import Ynm, Symbol, diff\n    >>> from sympy.abc import n,m\n    >>> theta = Symbol(\"theta\")\n    >>> phi = Symbol(\"phi\")\n\n    >>> diff(Ynm(n, m, theta, phi), theta)\n    m*cot(theta)*Ynm(n, m, theta, phi) + sqrt((-m + n)*(m + n + 1))*exp(-I*phi)*Ynm(n, m + 1, theta, phi)\n\n    >>> diff(Ynm(n, m, theta, phi), phi)\n    I*m*Ynm(n, m, theta, phi)\n\n    Further we can compute the complex conjugation:\n\n    >>> from sympy import Ynm, Symbol, conjugate\n    >>> from sympy.abc import n,m\n    >>> theta = Symbol(\"theta\")\n    >>> phi = Symbol(\"phi\")\n\n    >>> conjugate(Ynm(n, m, theta, phi))\n    (-1)**(2*m)*exp(-2*I*m*phi)*Ynm(n, m, theta, phi)\n\n    To get back the well known expressions in spherical\n    coordinates, we use full expansion:\n\n    >>> from sympy import Ynm, Symbol, expand_func\n    >>> from sympy.abc import n,m\n    >>> theta = Symbol(\"theta\")\n    >>> phi = Symbol(\"phi\")\n\n    >>> expand_func(Ynm(n, m, theta, phi))\n    sqrt((2*n + 1)*factorial(-m + n)/factorial(m + n))*exp(I*m*phi)*assoc_legendre(n, m, cos(theta))/(2*sqrt(pi))\n\n    See Also\n    ========\n\n    Ynm_c, Znm, sympy.physics.hydrogen.Y_lm, sympy.physics.hydrogen.Z_lm\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Spherical_harmonics\n    .. [2] https://mathworld.wolfram.com/SphericalHarmonic.html\n    .. [3] https://functions.wolfram.com/Polynomials/SphericalHarmonicY/\n    .. [4] https://dlmf.nist.gov/14.30",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `Ynm` inherits from `DefinedFunction` is not verifiable from the documentation and may be a hallucination \u2014 SymPy functions typically inherit from `Function` directly, and `DefinedFunction` is not a standard SymPy class.",
      "The claim about `DefinedFunction.__new__` using `@cacheit` and delegating to `cls._new_()` with specific line numbers cannot be verified from the documentation.",
      "The claim about `_eval_conjugate` returning `S.NegativeOne**m * self.func(n, -m, theta, phi)` is plausible but unverifiable from the doc \u2014 however the final result `(-1)**(2*m)*exp(-2*I*m*phi)*Ynm(n, m, theta, phi)` matches the documentation.",
      "The specific line numbers mentioned (line 823, line 96, line 154, line 212, etc.) cannot be verified from the documentation.",
      "The `Ynm(n, -m, theta, phi)` symmetry in the doc produces `(-1)**m*exp(-2*I*m*phi)*Ynm(n, m, theta, phi)`, which is consistent with the claim about `_eval_conjugate` producing this form.",
      "The claim about `Znm.eval` constructing real spherical harmonics is plausible but unverifiable from the provided documentation.",
      "The claim about `as_real_imag` method splitting using `cos(m*phi)` and `sin(m*phi)` is plausible but unverifiable.",
      "Most implementation details (evalf using mpmath, expand_func substitution, rewrite methods, benchmark file) are plausible extrapolations from a code-level perspective but cannot be verified from the documentation alone."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in descriptions of purpose/justification and one unverifiable benchmark file reference.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_16",
    "repo": "sympy",
    "question": "How does the `hyper` class implement the generalized hypergeometric function, including parameter cancellation, convergence, differentiation, and numerical evaluation?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `hyper` class implements the generalized hypergeometric function through a class hierarchy and several key mechanisms:\n\n**Class Hierarchy:** `hyper` \u2192 `TupleParametersBase` \u2192 `DefinedFunction` \u2192 `Function` \u2192 `Application` \u2192 `Basic`\n\n**Constructor (`__new__`) - Duplicate Parameter Cancellation:**\nWhen `evaluate=True` (default from `global_parameters.evaluate`), the constructor uses `collections.Counter` to find common elements between numerator parameters `ap` and denominator parameters `bq`, then removes them via Counter intersection (`ca & cb`). This is why `hyper((1, 2, 3), [3, 4], x)` simplifies to `hyper((1, 2), (4,), x)`. When `evaluate=False`, parameters are only sorted using `ordered()` but not cancelled.\n\n**Parameter Storage:** Parameters are stored as `TupleArg` objects (a subclass of `Tuple` from `sympy.core.containers`). The `_prep_tuple` function calls `unpolarify` on each parameter element.\n\n**The `eval` classmethod:** Only acts when `len(ap) <= len(bq)` or when `len(ap) == len(bq) + 1` AND `Abs(z) <= 1` evaluates to `True`. In those cases, it attempts to `unpolarify` the argument `z`, returning a new `hyper` only if `z` changed.\n\n**Differentiation (`fdiff`):** Differentiating with respect to `z` (argindex=3) produces `(product of ap)/(product of bq) * hyper(ap+1, bq+1, z)` \u2014 each parameter is incremented by 1. The parent class `TupleParametersBase._eval_derivative` handles differentiation with respect to symbols appearing in the tuple parameters, using `fdiff((1, i))` tuple-style argindex.\n\n**`radius_of_convergence`:** Implements a complex algorithm handling non-positive integer parameters. It checks if numerator non-positive integers can \"cancel\" denominator non-positive integers (a numerator `a` cancels a denominator `b` if `a >= b`, meaning it's larger/closer to zero). If uncancelled denominator non-positive integers remain, returns 0. If leftover numerator non-positive integers exist, returns \u221e. Otherwise returns 1 if `len(ap) == len(bq) + 1`, \u221e if `len(ap) <= len(bq)`, and 0 otherwise.\n\n**`convergence_statement`:** When radius is 1, uses three conditions based on `eta` (sum of ap minus sum of bq). The three conditions involve `re(eta) < 0` with `|z| \u2264 1`, `0 \u2264 re(eta) < 1` with `|z| \u2264 1` and `z \u2260 1`, and `re(eta) \u2265 1` with `|z| < 1`.\n\n**`_eval_expand_func`:** For 2F1(a,b;c;1), computes `gamma(c)*gamma(c - a - b)/gamma(c - a)/gamma(c - b)`. Falls back to `hyperexpand()` for other cases.\n\n**Numerical Evaluation:** `hyper` does NOT have a custom `_eval_evalf` (unlike `meijerg`). It relies on `Function._eval_evalf` which looks up `mpmath.hyper` by name (not in `MPMATH_TRANSLATIONS`). The `TupleArg._to_mpmath(prec)` method (inherited from `Tuple`) converts parameters to mpmath format by calling `_to_mpmath` on each element.\n\n**`meijerg._eval_evalf`:** Uses a special technique for polar arguments \u2014 it uses mpmath's optional `r` parameter, computing `z' = z^(1/n) * exp(I*branch/n)` where `n = ceiling(|branch/pi|) + 1`, then evaluates `G(z'^n)` via `ctx.meijerg(ap, bq, z, r)` with `r=1/n`.\n\n**`_eval_nseries`:** Only handles the simple case where `arg == x` and `x0 == 0` directly (computing RisingFactorial terms up to order n). Otherwise falls back to `hyperexpand(super()._eval_nseries(...))`.\n\n**`HyperRep` classes:** Multiple subclasses provide representations for specific hypergeometric functions, used by `hyperexpand()`. They implement `_expr_small`, `_expr_small_minus`, `_expr_big`, and `_expr_big_minus` for different branch regions, using `extract_branch_factor` to determine the branch.",
    "rubric": [
      "Explains the class hierarchy: hyper \u2192 TupleParametersBase \u2192 DefinedFunction \u2192 Function",
      "Describes the __new__ constructor's duplicate parameter cancellation using collections.Counter intersection (ca & cb) when evaluate=True, and just sorting with ordered() when evaluate=False",
      "Explains that parameters are stored as TupleArg objects and _prep_tuple calls unpolarify on each element",
      "Describes the eval classmethod's condition: only acts when len(ap) <= len(bq) or when len(ap) == len(bq)+1 and Abs(z) <= 1 evaluates to True, attempting to unpolarify z",
      "Explains fdiff: differentiating w.r.t. z produces (product of ap)/(product of bq) * hyper(ap+1, bq+1, z), incrementing each parameter by 1",
      "Describes TupleParametersBase._eval_derivative handling differentiation w.r.t. symbols in tuple parameters using fdiff((1, i)) tuple-style argindex",
      "Explains the radius_of_convergence algorithm for non-positive integer parameters: numerator a cancels denominator b if a >= b, uncancelled denominator non-positive integers give 0, leftover numerator ones give infinity",
      "Describes convergence_statement's three conditions when radius is 1, based on eta = sum(ap) - sum(bq): re(eta)<0 with |z|\u22641, 0\u2264re(eta)<1 with |z|\u22641 and z\u22601, re(eta)\u22651 with |z|<1",
      "Explains _eval_expand_func: for 2F1(a,b;c;1), computes gamma(c)*gamma(c-a-b)/gamma(c-a)/gamma(c-b); otherwise falls back to hyperexpand()",
      "Notes that hyper lacks a custom _eval_evalf (unlike meijerg), relying on the parent Function class to look up mpmath.hyper by name",
      "Describes meijerg._eval_evalf's technique for polar arguments using mpmath's r parameter with z' = z^(1/n)*exp(I*branch/n) where n = ceiling(|branch/pi|) + 1"
    ],
    "key_files": [
      "sympy/functions/special/hyper.py",
      "sympy/core/function.py",
      "sympy/core/containers.py",
      "sympy/simplify/hyperexpand.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/hyper.py] sympy.functions.special.hyper.hyper\nsympy.functions.special.hyper.hyper:\n  The generalized hypergeometric function is defined by a series where\n    the ratios of successive terms are a rational function of the summation\n    index. When convergent, it is continued analytically to the largest\n    possible domain.\n\n    Explanation\n    ===========\n\n    The hypergeometric function depends on two vectors of parameters, called\n    the numerator parameters $a_p$, and the denominator parameters\n    $b_q$. It also has an argument $z$. The series definition is\n\n    .. math ::\n        {}_pF_q\\left(\\begin{matrix} a_1, \\cdots, a_p \\\\ b_1, \\cdots, b_q \\end{matrix}\n                     \\middle| z \\right)\n        = \\sum_{n=0}^\\infty \\frac{(a_1)_n \\cdots (a_p)_n}{(b_1)_n \\cdots (b_q)_n}\n                            \\frac{z^n}{n!},\n\n    where $(a)_n = (a)(a+1)\\cdots(a+n-1)$ denotes the rising factorial.\n\n    If one of the $b_q$ is a non-positive integer then the series is\n    undefined unless one of the $a_p$ is a larger (i.e., smaller in\n    magnitude) non-positive integer. If none of the $b_q$ is a\n    non-positive integer and one of the $a_p$ is a non-positive\n    integer, then the series reduces to a polynomial. To simplify the\n    following discussion, we assume that none of the $a_p$ or\n    $b_q$ is a non-positive integer. For more details, see the\n    references.\n\n    The series converges for all $z$ if $p \\le q$, and thus\n    defines an entire single-valued function in this case. If $p =\n    q+1$ the series converges for $|z| < 1$, and can be continued\n    analytically into a half-plane. If $p > q+1$ the series is\n    divergent for all $z$.\n\n    Please note the hypergeometric function constructor currently does *not*\n    check if the parameters actually yield a well-defined function.\n\n    Examples\n    ========\n\n    The parameters $a_p$ and $b_q$ can be passed as arbitrary\n    iterables, for example:\n\n    >>> from sympy import hyper\n    >>> from sympy.abc import x, n, a\n    >>> h = hyper((1, 2, 3), [3, 4], x); h\n    hyper((1, 2), (4,), x)\n    >>> hyper((3, 1, 2), [3, 4], x, evaluate=False)  # don't remove duplicates\n    hyper((1, 2, 3), (3, 4), x)\n\n    There is also pretty printing (it looks better using Unicode):\n\n    >>> from sympy import pprint\n    >>> pprint(h, use_unicode=False)\n      _\n     |_  /1, 2 |  \\\n     |   |     | x|\n    2  1 \\  4  |  /\n\n    The parameters must always be iterables, even if they are vectors of\n    length one or zero:\n\n    >>> hyper((1, ), [], x)\n    hyper((1,), (), x)\n\n    But of course they may be variables (but if they depend on $x$ then you\n    should not expect much implemented functionality):\n\n    >>> hyper((n, a), (n**2,), x)\n    hyper((a, n), (n**2,), x)\n\n    The hypergeometric function generalizes many named special functions.\n    The function ``hyperexpand()`` tries to express a hypergeometric function\n    using named special functions. For example:\n\n    >>> from sympy import hyperexpand\n    >>> hyperexpand(hyper([], [], x))\n    exp(x)\n\n    You can also use ``expand_func()``:\n\n    >>> from sympy import expand_func\n    >>> expand_func(x*hyper([1, 1], [2], -x))\n    log(x + 1)\n\n    More examples:\n\n    >>> from sympy import S\n    >>> hyperexpand(hyper([], [S(1)/2], -x**2/4))\n    cos(x)\n    >>> hyperexpand(x*hyper([S(1)/2, S(1)/2], [S(3)/2], x**2))\n    asin(x)\n\n    We can also sometimes ``hyperexpand()`` parametric functions:\n\n    >>> from sympy.abc import a\n    >>> hyperexpand(hyper([-a], [], x))\n    (1 - x)**a\n\n    See Also\n    ========\n\n    sympy.simplify.hyperexpand\n    gamma\n    meijerg\n\n    References\n    ==========\n\n    .. [1] Luke, Y. L. (1969), The Special Functions and Their Approximations,\n           Volume 1\n    .. [2] https://en.wikipedia.org/wiki/Generalized_hypergeometric_function\n\nsympy.functions.special.hyper.hyper.argument:\n  Argument of the hypergeometric function.\n\nsympy.functions.special.hyper.hyper.ap:\n  Numerator parameters of the hypergeometric function.\n\nsympy.functions.special.hyper.hyper.bq:\n  Denominator parameters of the hypergeometric function.\n\nsympy.functions.special.hyper.hyper.eta:\n  A quantity related to the convergence of the series.\n\nsympy.functions.special.hyper.hyper.radius_of_convergence:\n  Compute the radius of convergence of the defining series.\n\n        Explanation\n        ===========\n\n        Note that even if this is not ``oo``, the function may still be\n        evaluated outside of the radius of convergence by analytic\n        continuation. But if this is zero, then the function is not actually\n        defined anywhere else.\n\n        Examples\n        ========\n\n        >>> from sympy import hyper\n        >>> from sympy.abc import z\n        >>> hyper((1, 2), [3], z).radius_of_convergence\n        1\n        >>> hyper((1, 2, 3), [4], z).radius_of_convergence\n        0\n        >>> hyper((1, 2), (3, 4), z).radius_of_convergence\n        oo\n\nsympy.functions.special.hyper.hyper.convergence_statement:\n  Return a condition on z under which the series converges.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (class hierarchy, __new__ internals, fdiff, _eval_expand_func, _eval_nseries, HyperRep classes, meijerg._eval_evalf, etc.) that go far beyond what the documentation covers. These are plausible code-level details but cannot be verified from the provided documentation alone.",
      "The answer's description of radius_of_convergence is consistent with the documentation's examples and the convergence rules described in the docstring.",
      "The answer's description of convergence_statement referencing eta and conditions is plausible but not explicitly detailed in the provided documentation (documentation only says 'Return a condition on z under which the series converges').",
      "The answer correctly describes the duplicate parameter cancellation behavior shown in the documentation examples (hyper((1, 2, 3), [3, 4], x) simplifying to hyper((1, 2), (4,), x)).",
      "The answer correctly describes the evaluate=False behavior shown in the documentation (hyper((3, 1, 2), [3, 4], x, evaluate=False) keeping all parameters).",
      "The answer's claim about convergence rules (p \u2264 q \u2192 entire, p = q+1 \u2192 |z| < 1, p > q+1 \u2192 divergent) is consistent with the documentation.",
      "The answer's description of hyperexpand examples is consistent with the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor documentation-only content includes references to named theorems, rationale comments, and the specific count of HyperRep subclasses.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_23",
    "repo": "sympy",
    "question": "Where is `is_matadd_valid` defined, is it actually used anywhere in the codebase, and how does shape validation for `MatAdd` really work?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `is_matadd_valid` function is defined in `sympy/matrices/expressions/_shape.py` at line 14. It is one of five shape-related functions in that module, alongside `is_matmul_valid`, `is_square`, `validate_matadd_integer`, and `validate_matmul_integer`.\n\nNotably, `is_matadd_valid` is **not directly called anywhere** in the codebase \u2014 it returns a `Boolean` expression (specifically an `And` of `Eq` relations). In contrast, the actual validation used during construction of `MatAdd` and `HadamardProduct` is done by `validate_matadd_integer`, which is a separate function that only checks integer-valued dimensions and raises `ShapeError` for mismatches.\n\nBoth `MatAdd` (in `matadd.py`, line 15) and `HadamardProduct` (in `hadamard.py`, line 10) import `validate_matadd_integer` aliased as `validate`, and call it in their `__new__` constructors. `HadamardPower.__new__` (line 379) also calls `validate(base, exp)` when both base and exp are `MatrixExpr` instances.\n\nThe `is_matadd_valid` implementation (lines 36-40) works by zipping together all argument shapes, then creating pairwise `Eq` constraints between consecutive rows and consecutive columns. For example, with args A(m,n) and B(p,q), it generates `Eq(m, p) & Eq(n, q)`. This is a sliding-window comparison (comparing consecutive pairs), not an all-to-first comparison.\n\nThe sibling function `is_matmul_valid` (line 43) differs: it filters out non-MatrixExpr args (scalar expressions), and only checks that each matrix's columns equal the next matrix's rows (`cols[:-1]` vs `rows[1:]`).\n\nThe `_shape.py` module is not exported in `sympy/matrices/expressions/__init__.py`'s `__all__`, meaning these functions must be imported directly from the private module path `sympy.matrices.expressions._shape`.",
    "rubric": [
      "Identifies that `is_matadd_valid` is defined in `sympy/matrices/expressions/_shape.py`",
      "Notes that `is_matadd_valid` is not called/imported anywhere outside its own module (unused in practice)",
      "Explains that `is_matadd_valid` returns a symbolic Boolean (And of Eq constraints) comparing consecutive argument shapes pairwise",
      "Identifies that `validate_matadd_integer` is the function actually used for validation, imported as `validate` in both `matadd.py` and `hadamard.py`",
      "Explains that `validate_matadd_integer` only checks integer-valued dimensions and raises `ShapeError` on mismatch",
      "Mentions that `HadamardPower.__new__` also calls `validate(base, exp)` when both args are MatrixExpr instances",
      "Notes the `_shape.py` module contains five shape-related functions: `is_matadd_valid`, `is_matmul_valid`, `is_square`, `validate_matadd_integer`, and `validate_matmul_integer`"
    ],
    "key_files": [
      "sympy/matrices/expressions/_shape.py",
      "sympy/matrices/expressions/matadd.py",
      "sympy/matrices/expressions/hadamard.py",
      "sympy/matrices/expressions/matmul.py",
      "sympy/matrices/expressions/__init__.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/_shape.py] sympy.matrices.expressions._shape.is_matadd_valid\nReturn the symbolic condition how ``MatAdd``, ``HadamardProduct``\n    makes sense.\n\n    Parameters\n    ==========\n\n    args\n        The list of arguments of matrices to be tested for.\n\n    Examples\n    ========\n\n    >>> from sympy import MatrixSymbol, symbols\n    >>> from sympy.matrices.expressions._shape import is_matadd_valid\n\n    >>> m, n, p, q = symbols('m n p q')\n    >>> A = MatrixSymbol('A', m, n)\n    >>> B = MatrixSymbol('B', p, q)\n    >>> is_matadd_valid(A, B)\n    Eq(m, p) & Eq(n, q)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor characterization of 'public symbolic API' was slightly trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_20",
    "repo": "sympy",
    "question": "How does KroneckerDelta's eval method decide whether to return zero, one, or swap its indices?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `KroneckerDelta` class in `sympy/functions/special/tensor_functions.py` is implemented as a subclass of `DefinedFunction` (which itself inherits from `Function` via the chain `DefinedFunction -> Function -> Application -> Basic`). Here's how the algorithm works:\n\n**Evaluation Algorithm (`eval` method):**\n1. If a `delta_range` tuple `(dinf, dsup)` is provided, it first checks if either index `i` or `j` falls outside the range `[dinf, dsup]`. If so, it returns `S.Zero`.\n2. Computes `diff = i - j` and uses `diff.is_zero` (SymPy's assumption system) to determine equality. If `diff.is_zero` is True, returns `S.One`. If `fuzzy_not(diff.is_zero)` is True (meaning the difference is definitively non-zero), returns `S.Zero`.\n3. Checks fermi-level assumptions: if one index has `below_fermi=True` and the other has `above_fermi=True`, returns `S.Zero`.\n4. **Canonicalization**: Uses `default_sort_key(j) < default_sort_key(i)` to enforce a canonical ordering of indices. If j sorts before i, it swaps them by returning `cls(j, i)` (or `cls(j, i, delta_range)` if a range is present).\n\n**Power handling (`_eval_power`):**\nFor any positive power exponent, `_eval_power` returns `self` unchanged. For negative exponents that are not exactly -1, it returns `1/self` (which creates a Pow object). This is called by `Pow.__new__` via `base._eval_power(exp)`.\n\n**nargs determination:**\nSince `eval(cls, i, j, delta_range=None)` has 2 required parameters and 1 optional parameter (excluding `cls`), the `arity()` function in `sympy/core/function.py` computes nargs as `(2, 3)` \u2014 meaning KroneckerDelta accepts 2 or 3 arguments.\n\n**Caching:**\n`DefinedFunction.__new__` is decorated with `@cacheit`, so repeated identical calls return cached results. This differs from the base `Function.__new__` which also uses `@cacheit` but goes through a different path for undefined functions.\n\n**Fermi-level index preference algorithm (`_get_preferred_index`):**\n1. If the delta cannot be non-zero above fermi (`not self.is_above_fermi`), it checks if `args[0]` has `below_fermi` assumption \u2014 if yes, return 0, else return 1.\n2. If the delta cannot be non-zero below fermi (`not self.is_below_fermi`), it checks if `args[0]` has `above_fermi` assumption \u2014 if yes, return 0, else return 1.\n3. Otherwise (both above and below fermi possible), returns 0 (first index is preferred by default).\n\n**Integration with summation system:**\nThe `deltasummation` function in `sympy/concrete/delta.py` solves `delta.args[0] - delta.args[1]` for the summation variable, substitutes the solution into the remaining expression, and wraps the result in a `Piecewise` to handle boundary conditions (unless `delta_range` confirms the solution is within bounds, in which case `no_piecewise=True` is set).",
    "rubric": [
      "Mentions checking delta_range bounds: if either index falls outside [dinf, dsup], returns zero",
      "Explains computing diff = i - j and using diff.is_zero to return S.One when true",
      "Explains using fuzzy_not(diff.is_zero) to return S.Zero when the difference is definitively non-zero",
      "Describes checking fermi-level assumptions: returns zero if one index is below_fermi and the other is above_fermi",
      "Explains canonicalization via default_sort_key comparison: if j sorts before i, returns cls(j, i) to swap the indices",
      "Mentions _eval_power returns self for positive exponents and 1/self for negative exponents (not -1)"
    ],
    "key_files": [
      "sympy/functions/special/tensor_functions.py",
      "sympy/core/function.py",
      "sympy/concrete/delta.py",
      "sympy/physics/secondquant.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/tensor_functions.py] sympy.functions.special.tensor_functions.KroneckerDelta\nsympy.functions.special.tensor_functions.KroneckerDelta:\n  The discrete, or Kronecker, delta function.\n\n    Explanation\n    ===========\n\n    A function that takes in two integers $i$ and $j$. It returns $0$ if $i$\n    and $j$ are not equal, or it returns $1$ if $i$ and $j$ are equal.\n\n    Examples\n    ========\n\n    An example with integer indices:\n\n        >>> from sympy import KroneckerDelta\n        >>> KroneckerDelta(1, 2)\n        0\n        >>> KroneckerDelta(3, 3)\n        1\n\n    Symbolic indices:\n\n        >>> from sympy.abc import i, j, k\n        >>> KroneckerDelta(i, j)\n        KroneckerDelta(i, j)\n        >>> KroneckerDelta(i, i)\n        1\n        >>> KroneckerDelta(i, i + 1)\n        0\n        >>> KroneckerDelta(i, i + 1 + k)\n        KroneckerDelta(i, i + k + 1)\n\n    Parameters\n    ==========\n\n    i : Number, Symbol\n        The first index of the delta function.\n    j : Number, Symbol\n        The second index of the delta function.\n\n    See Also\n    ========\n\n    eval\n    DiracDelta\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Kronecker_delta\n\nsympy.functions.special.tensor_functions.KroneckerDelta.eval:\n  Evaluates the discrete delta function.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta\n        >>> from sympy.abc import i, j, k\n\n        >>> KroneckerDelta(i, j)\n        KroneckerDelta(i, j)\n        >>> KroneckerDelta(i, i)\n        1\n        >>> KroneckerDelta(i, i + 1)\n        0\n        >>> KroneckerDelta(i, i + 1 + k)\n        KroneckerDelta(i, i + k + 1)\n\n        # indirect doctest\n\nsympy.functions.special.tensor_functions.KroneckerDelta.is_above_fermi:\n  True if Delta can be non-zero above fermi.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> q = Symbol('q')\n        >>> KroneckerDelta(p, a).is_above_fermi\n        True\n        >>> KroneckerDelta(p, i).is_above_fermi\n        False\n        >>> KroneckerDelta(p, q).is_above_fermi\n        True\n\n        See Also\n        ========\n\n        is_below_fermi, is_only_below_fermi, is_only_above_fermi\n\nsympy.functions.special.tensor_functions.KroneckerDelta.is_below_fermi:\n  True if Delta can be non-zero below fermi.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> q = Symbol('q')\n        >>> KroneckerDelta(p, a).is_below_fermi\n        False\n        >>> KroneckerDelta(p, i).is_below_fermi\n        True\n        >>> KroneckerDelta(p, q).is_below_fermi\n        True\n\n        See Also\n        ========\n\n        is_above_fermi, is_only_above_fermi, is_only_below_fermi\n\nsympy.functions.special.tensor_functions.KroneckerDelta.is_only_above_fermi:\n  True if Delta is restricted to above fermi.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> q = Symbol('q')\n        >>> KroneckerDelta(p, a).is_only_above_fermi\n        True\n        >>> KroneckerDelta(p, q).is_only_above_fermi\n        False\n        >>> KroneckerDelta(p, i).is_only_above_fermi\n        False\n\n        See Also\n        ========\n\n        is_above_fermi, is_below_fermi, is_only_below_fermi\n\nsympy.functions.special.tensor_functions.KroneckerDelta.is_only_below_fermi:\n  True if Delta is restricted to below fermi.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> q = Symbol('q')\n        >>> KroneckerDelta(p, i).is_only_below_fermi\n        True\n        >>> KroneckerDelta(p, q).is_only_below_fermi\n        False\n        >>> KroneckerDelta(p, a).is_only_below_fermi\n        False\n\n        See Also\n        ========\n\n        is_above_fermi, is_below_fermi, is_only_above_fermi\n\nsympy.functions.special.tensor_functions.KroneckerDelta.indices_contain_equal_information:\n  Returns True if indices are either both above or below fermi.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> q = Symbol('q')\n        >>> KroneckerDelta(p, q).indices_contain_equal_information\n        True\n        >>> KroneckerDelta(p, q+1).indices_contain_equal_information\n        True\n        >>> KroneckerDelta(i, p).indices_contain_equal_information\n        False\n\nsympy.functions.special.tensor_functions.KroneckerDelta.preferred_index:\n  Returns the index which is preferred to keep in the final expression.\n\n        Explanation\n        ===========\n\n        The preferred index is the index with more information regarding fermi\n        level. If indices contain the same information, 'a' is preferred before\n        'b'.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> j = Symbol('j', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> KroneckerDelta(p, i).preferred_index\n        i\n        >>> KroneckerDelta(p, a).preferred_index\n        a\n        >>> KroneckerDelta(i, j).preferred_index\n        i\n\n        See Also\n        ========\n\n        killable_index\n\nsympy.functions.special.tensor_functions.KroneckerDelta.killable_index:\n  Returns the index which is preferred to substitute in the final\n        expression.\n\n        Explanation\n        ===========\n\n        The index to substitute is the index with less information regarding\n        fermi level. If indices contain the same information, 'a' is preferred\n        before 'b'.\n\n        Examples\n        ========\n\n        >>> from sympy import KroneckerDelta, Symbol\n        >>> a = Symbol('a', above_fermi=True)\n        >>> i = Symbol('i', below_fermi=True)\n        >>> j = Symbol('j', below_fermi=True)\n        >>> p = Symbol('p')\n        >>> KroneckerDelta(p, i).killable_index\n        p\n        >>> KroneckerDelta(p, a).killable_index\n        p\n        >>> KroneckerDelta(i, j).killable_index\n        j\n\n        See Also\n        ========\n\n        preferred_index\n\nsympy.functions.special.tensor_functions.KroneckerDelta._get_preferred_index:\n  Returns the index which is preferred to keep in the final expression.\n\n        The preferred index is the index with more information regarding fermi\n        level. If indices contain the same information, index 0 is returned.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that KroneckerDelta is a subclass of 'DefinedFunction' is not supported by the documentation and may be incorrect - in modern SymPy, KroneckerDelta inherits from Function directly.",
      "The claim about 'DefinedFunction.__new__' being decorated with '@cacheit' is not supported by the documentation and likely refers to an older version of SymPy.",
      "The detailed line numbers (e.g., 'lines 148-201', 'lines 208-212', 'line 225 in sympy/core/power.py', 'lines 448-467', 'line 2147') are not verifiable from the documentation and may be fabricated.",
      "The claim about nargs being '(2, 3)' is not supported by the documentation but is plausible given the eval signature.",
      "The claim about LaTeX printing using space-separated vs comma-separated format is not supported by the documentation.",
      "The claim about deltasummation in sympy/concrete/delta.py is not supported by the documentation provided.",
      "The _eval_power behavior description is not supported by the documentation.",
      "The canonicalization using default_sort_key is not documented but is plausible implementation detail."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the LaTeX printer claim and minor rationale phrases needed removal/rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_22",
    "repo": "sympy",
    "question": "Why does the LRASolver's `check()` method have an `is_sat` fast path, and what are the different short-circuiting mechanisms that avoid running the simplex loop?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The LRASolver uses Bland's pivoting rule in the simplex-based `check()` method. Bland's rule selects the basic variable with the smallest `col_idx` among those violating their bounds: `xi = min(cand, key=lambda v: v.col_idx)`. However, there's an inconsistency in the pivot column selection: when `xi.assign < xi.lower`, the pivot column is chosen by `min(cand, key=str)`, while when `xi.assign > xi.upper`, it uses `min(cand, key=lambda v: v.col_idx)`.\n\nThe `is_sat` flag allows `check()` to bypass the simplex loop entirely, returning immediately with the current assignments. It works because for nonslack variables, `_assert_upper` and `_assert_lower` call `_update()` to immediately adjust assignments within bounds. The `_update` method iterates over all slack variables, adjusting their assignments based on the matrix coefficients. However, once `assert_lit` is called for a slack variable (detected via `if self.is_sat and sym not in self.slack_set`), the flag is set to False.\n\nThe `self.result` field provides a second level of short-circuiting: if `_assert_upper` or `_assert_lower` detects an immediate contradiction (new upper bound < existing lower bound or vice versa), it stores the conflict in `self.result`. Then `check()` returns this cached result immediately (`if self.result: return self.result`). In the DPLL(T) integration in dpll2.py, `assert_lit` is called for each literal in a loop that breaks early on conflict (`if res is not None: break`), followed by `check()`.\n\nThe current implementation resets all bounds via `reset_bounds()` after each `check()` call in the DPLL(T) loop (dpll2.py).\n\nThe `_pivot` method copies the matrix before modifying it (`A = M.copy()`). In testing_mode (`self.run_checks`), assertions verify M*X \u2248 0 after each bound assertion, and the `from_encoded_cnf` method sorts items by string representation when `testing_mode=True`.",
    "rubric": [
      "Explains that `is_sat` allows `check()` to return immediately with the current assignments without entering the simplex while loop",
      "Explains that `is_sat` remains True because `_assert_upper`/`_assert_lower` call `_update()` to keep nonslack variable assignments within bounds, maintaining feasibility without needing simplex",
      "Explains that `is_sat` is set to False when `assert_lit` is called on a slack variable (detected via `sym not in self.slack_set`) or when a conflict occurs",
      "Explains that `self.result` provides a second short-circuit: if `_assert_upper`/`_assert_lower` detects an immediate contradiction (upper < lower or vice versa), it caches the conflict so `check()` returns it directly",
      "Notes the inconsistency in Bland's rule: the pivot column selection uses `min(cand, key=str)` when `xi.assign < xi.lower` but `min(cand, key=lambda v: v.col_idx)` when `xi.assign > xi.upper`",
      "Mentions that in the DPLL(T) integration, `reset_bounds()` is called after every `check()` call, and `assert_lit` is called in a loop that breaks early on conflict"
    ],
    "key_files": [
      "sympy/logic/algorithms/lra_theory.py",
      "sympy/logic/algorithms/dpll2.py"
    ],
    "source_doc": "[docstring: sympy/logic/algorithms/lra_theory.py] sympy.logic.algorithms.lra_theory.LRASolver\nsympy.logic.algorithms.lra_theory.LRASolver:\n  Linear Arithmetic Solver for DPLL(T) implemented with an algorithm based on\n    the Dual Simplex method. Uses Bland's pivoting rule to avoid cycling.\n\n    References\n    ==========\n\n    .. [1] Dutertre, B., de Moura, L.:\n           A Fast Linear-Arithmetic Solver for DPLL(T)\n           https://link.springer.com/chapter/10.1007/11817963_11\n\nsympy.logic.algorithms.lra_theory.LRASolver.__init__:\n  Use the \"from_encoded_cnf\" method to create a new LRASolver.\n\nsympy.logic.algorithms.lra_theory.LRASolver.from_encoded_cnf:\n  Creates an LRASolver from an EncodedCNF object\n        and a list of conflict clauses for propositions\n        that can be simplified to True or False.\n\n        Parameters\n        ==========\n\n        encoded_cnf : EncodedCNF\n\n        testing_mode : bool\n            Setting testing_mode to True enables some slow assert statements\n            and sorting to reduce nonterministic behavior.\n\n        Returns\n        =======\n\n        (lra, conflicts)\n\n        lra : LRASolver\n\n        conflicts : list\n            Contains a one-literal conflict clause for each proposition\n            that can be simplified to True or False.\n\n        Example\n        =======\n\n        >>> from sympy.core.relational import Eq\n        >>> from sympy.assumptions.cnf import CNF, EncodedCNF\n        >>> from sympy.assumptions.ask import Q\n        >>> from sympy.logic.algorithms.lra_theory import LRASolver\n        >>> from sympy.abc import x, y, z\n        >>> phi = (x >= 0) & ((x + y <= 2) | (x + 2 * y - z >= 6))\n        >>> phi = phi & (Eq(x + y, 2) | (x + 2 * y - z > 4))\n        >>> phi = phi & Q.gt(2, 1)\n        >>> cnf = CNF.from_prop(phi)\n        >>> enc = EncodedCNF()\n        >>> enc.from_cnf(cnf)\n        >>> lra, conflicts = LRASolver.from_encoded_cnf(enc, testing_mode=True)\n        >>> lra #doctest: +SKIP\n        <sympy.logic.algorithms.lra_theory.LRASolver object at 0x7fdcb0e15b70>\n        >>> conflicts #doctest: +SKIP\n        [[4]]\n\nsympy.logic.algorithms.lra_theory.LRASolver.reset_bounds:\n  Resets the state of the LRASolver to before\n        anything was asserted.\n\nsympy.logic.algorithms.lra_theory.LRASolver.assert_lit:\n  Assert a literal representing a constraint\n        and update the internal state accordingly.\n\n        Note that due to peculiarities of this implementation\n        asserting ~(x > 0) will assert (x <= 0) but asserting\n        ~Eq(x, 0) will not do anything.\n\n        Parameters\n        ==========\n\n        enc_constraint : int\n            A mapping of encodings to constraints\n            can be found in `self.enc_to_boundary`.\n\n        Returns\n        =======\n\n        None or (False, explanation)\n\n        explanation : set of ints\n            A conflict clause that \"explains\" why\n            the literals asserted so far are unsatisfiable.\n\nsympy.logic.algorithms.lra_theory.LRASolver._assert_upper:\n  Adjusts the upper bound on variable xi if the new upper bound is\n        more limiting. The assignment of variable xi is adjusted to be\n        within the new bound if needed.\n\n        Also calls `self._update` to update the assignment for slack variables\n        to keep all equalities satisfied.\n\nsympy.logic.algorithms.lra_theory.LRASolver._assert_lower:\n  Adjusts the lower bound on variable xi if the new lower bound is\n        more limiting. The assignment of variable xi is adjusted to be\n        within the new bound if needed.\n\n        Also calls `self._update` to update the assignment for slack variables\n        to keep all equalities satisfied.\n\nsympy.logic.algorithms.lra_theory.LRASolver._update:\n  Updates all slack variables that have equations that contain\n        variable xi so that they stay satisfied given xi is equal to v.\n\nsympy.logic.algorithms.lra_theory.LRASolver.check:\n  Searches for an assignment that satisfies all constraints\n        or determines that no such assignment exists and gives\n        a minimal conflict clause that \"explains\" why the\n        constraints are unsatisfiable.\n\n        Returns\n        =======\n\n        (True, assignment) or (False, explanation)\n\n        assignment : dict of LRAVariables to values\n            Assigned values are tuples that represent a rational number\n            plus some infinatesimal delta.\n\n        explanation : set of ints\n\nsympy.logic.algorithms.lra_theory.LRASolver._pivot_and_update:\n  Pivots basic variable xi with nonbasic variable xj,\n        and sets value of xi to v and adjusts the values of all basic variables\n        to keep equations satisfied.\n\nsympy.logic.algorithms.lra_theory.LRASolver._pivot:\n  Performs a pivot operation about entry i, j of M by performing\n        a series of row operations on a copy of M and returning the result.\n        The original M is left unmodified.\n\n        Conceptually, M represents a system of equations and pivoting\n        can be thought of as rearranging equation i to be in terms of\n        variable j and then substituting in the rest of the equations\n        to get rid of other occurrences of variable j.\n\n        Example\n        =======\n\n        >>> from sympy.matrices.dense import Matrix\n        >>> from sympy.logic.algorithms.lra_theory import LRASolver\n        >>> from sympy import var\n        >>> Matrix(3, 3, var('a:i'))\n        Matrix([\n        [a, b, c],\n        [d, e, f],\n        [g, h, i]])\n\n        This matrix is equivalent to:\n        0 = a*x + b*y + c*z\n        0 = d*x + e*y + f*z\n        0 = g*x + h*y + i*z\n\n        >>> LRASolver._pivot(_, 1, 0)\n        Matrix([\n        [ 0, -a*e/d + b, -a*f/d + c],\n        [-1,       -e/d,       -f/d],\n        [ 0,  h - e*g/d,  i - f*g/d]])\n\n        We rearrange equation 1 in terms of variable 0 (x)\n        and substitute to remove x from the other equations.\n\n        0 = 0 + (-a*e/d + b)*y + (-a*f/d + c)*z\n        0 = -x + (-e/d)*y + (-f/d)*z\n        0 = 0 + (h - e*g/d)*y + (i - f*g/d)*z",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., line 587, 604, 623, 470-471, 512-513, 432, 554, 106, 694, 473-477, 254) that cannot be verified from the documentation provided. These may be accurate code references but are not supported by the given docs.",
      "The claim about an 'inconsistency' in pivot column selection where one case uses `min(cand, key=str)` and another uses `min(cand, key=lambda v: v.col_idx)` cannot be verified from the documentation. The documentation only states that Bland's pivoting rule is used to avoid cycling.",
      "The detailed description of `is_sat` flag behavior, `self.result` field, and `self.slack_set` are implementation details not mentioned in the documentation, making them unverifiable claims.",
      "References to dpll2.py integration (lines 231, 234) are not supported by the provided documentation.",
      "The claim that `_pivot` copies the matrix (line 694) is partially supported - the doc says 'performing a series of row operations on a copy of M and returning the result. The original M is left unmodified.'",
      "The documentation confirms testing_mode 'enables some slow assert statements and sorting to reduce nonterministic behavior', which aligns with the answer's claims about testing_mode, though the specific O(m*n) complexity claims are unverifiable.",
      "The answer's claim about Bland's rule being used in check() is consistent with the documentation stating 'Uses Bland's pivoting rule to avoid cycling.'"
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims rely on design rationale from comments/TODOs (cycling avoidance, efficiency comparisons, TODO items), but most algorithmic behavior claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_21",
    "repo": "sympy",
    "question": "What is the simplification architecture of `lerchphi` compared to `polylog` and `zeta`, and how does its `_eval_expand_func` work?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `lerchphi` class in `sympy/functions/special/zeta_functions.py` is a subclass of `DefinedFunction` (which itself inherits from `Function` in `sympy/core/function.py`). Unlike the related `polylog` and `zeta` classes in the same module, `lerchphi` does **not** define a `@classmethod eval` method \u2014 since `Application.__new__` calls `cls.eval` during construction, the absence of `eval` means no automatic simplification occurs. All simplification is deferred to `_eval_expand_func` (invoked via `expand_func()`).\n\nKey architectural details:\n\n1. **Parent class**: `DefinedFunction` (in `sympy/core/function.py`) is a thin wrapper around `Function` that decorates `__new__` with `@cacheit` and delegates to `cls._new_()`. This makes constructed instances cached.\n\n2. **`_eval_expand_func` has three branches**:\n   - If `z == 1`: returns `zeta(s, a)` directly\n   - If `s` is a non-positive integer: uses a `Poly` object and iterative differentiation to compute a rational function. It constructs `Poly((t + a)**(-s), t)` and builds up a result using `start = 1/(1-t)` with `start = t*start.diff(t)` at each step.\n   - If `a` is Rational: it first reduces `a` to the interval (0, 1], then expands in terms of `polylog` using `exp_polar(2*pi*I/n)` as a root of unity and `z**(1/n)` as the n-th root of z.\n   - If `z` is a root of unity (or exp with rational argument over pi*I): expands as a sum of Hurwitz zeta functions.\n\n3. **`fdiff` method**: Only argindex 1 (z) and 3 (a) are supported. For any other argindex (including argindex=2 for `s`), it raises `ArgumentIndexError`.\n\n4. **`_eval_rewrite_helper` pattern**: The rewrite methods (`_eval_rewrite_as_zeta` and `_eval_rewrite_as_polylog`) use a helper that first calls `_eval_expand_func()`, then checks whether the result contains the target function. If it does, the expanded result is returned; otherwise, `self` is returned unchanged.\n\n5. **Connection to hyperexpand**: The function `try_lerchphi` in `sympy/simplify/hyperexpand.py` is related to expressing hypergeometric functions in terms of Lerch transcendents (as evidenced by `lerchphi` being imported in that module).\n\n6. **`polylog` has an `eval` classmethod** that provides automatic simplification (e.g., polylog(s,0)=0, polylog(s,1)=zeta(s), polylog(s,-1)=-dirichlet_eta(s)), while `lerchphi` does not have an `eval` method and does not auto-simplify.\n\n7. **The `_dilogtable` function** is a `@cacheit`-decorated function returning a dictionary of 11 special values, used by `polylog.eval` when `s == 2`.",
    "rubric": [
      "Identifies that `lerchphi` is a subclass of `Function` (via `DefinedFunction`)",
      "Notes that `lerchphi` does NOT define a classmethod `eval`, unlike `polylog` and `zeta`",
      "Explains that all simplification is deferred to `_eval_expand_func` (called via `expand_func()`)",
      "Describes the z==1 branch returning `zeta(s, a)`",
      "Describes the non-positive integer `s` branch using polynomial differentiation",
      "Describes the rational `a` branch expanding in terms of `polylog` with roots of unity",
      "Mentions the `fdiff` method supports argindex 1 (z) and 3 (a) but raises ArgumentIndexError for argindex 2 (s)",
      "Explains the `_eval_rewrite_helper` pattern that checks if expand contains the target function",
      "Notes that `polylog.eval` provides automatic simplification (e.g., polylog(s,0)=0, polylog(s,1)=zeta(s))"
    ],
    "key_files": [
      "sympy/functions/special/zeta_functions.py",
      "sympy/core/function.py",
      "sympy/simplify/hyperexpand.py"
    ],
    "source_doc": "[docstring: sympy/functions/special/zeta_functions.py] sympy.functions.special.zeta_functions.lerchphi\nsympy.functions.special.zeta_functions.lerchphi:\n  Lerch transcendent (Lerch phi function).\n\n    Explanation\n    ===========\n\n    For $\\operatorname{Re}(a) > 0$, $|z| < 1$ and $s \\in \\mathbb{C}$, the\n    Lerch transcendent is defined as\n\n    .. math :: \\Phi(z, s, a) = \\sum_{n=0}^\\infty \\frac{z^n}{(n + a)^s},\n\n    where the standard branch of the argument is used for $n + a$,\n    and by analytic continuation for other values of the parameters.\n\n    A commonly used related function is the Lerch zeta function, defined by\n\n    .. math:: L(q, s, a) = \\Phi(e^{2\\pi i q}, s, a).\n\n    **Analytic Continuation and Branching Behavior**\n\n    It can be shown that\n\n    .. math:: \\Phi(z, s, a) = z\\Phi(z, s, a+1) + a^{-s}.\n\n    This provides the analytic continuation to $\\operatorname{Re}(a) \\le 0$.\n\n    Assume now $\\operatorname{Re}(a) > 0$. The integral representation\n\n    .. math:: \\Phi_0(z, s, a) = \\int_0^\\infty \\frac{t^{s-1} e^{-at}}{1 - ze^{-t}}\n                                \\frac{\\mathrm{d}t}{\\Gamma(s)}\n\n    provides an analytic continuation to $\\mathbb{C} - [1, \\infty)$.\n    Finally, for $x \\in (1, \\infty)$ we find\n\n    .. math:: \\lim_{\\epsilon \\to 0^+} \\Phi_0(x + i\\epsilon, s, a)\n             -\\lim_{\\epsilon \\to 0^+} \\Phi_0(x - i\\epsilon, s, a)\n             = \\frac{2\\pi i \\log^{s-1}{x}}{x^a \\Gamma(s)},\n\n    using the standard branch for both $\\log{x}$ and\n    $\\log{\\log{x}}$ (a branch of $\\log{\\log{x}}$ is needed to\n    evaluate $\\log{x}^{s-1}$).\n    This concludes the analytic continuation. The Lerch transcendent is thus\n    branched at $z \\in \\{0, 1, \\infty\\}$ and\n    $a \\in \\mathbb{Z}_{\\le 0}$. For fixed $z, a$ outside these\n    branch points, it is an entire function of $s$.\n\n    Examples\n    ========\n\n    The Lerch transcendent is a fairly general function, for this reason it does\n    not automatically evaluate to simpler functions. Use ``expand_func()`` to\n    achieve this.\n\n    If $z=1$, the Lerch transcendent reduces to the Hurwitz zeta function:\n\n    >>> from sympy import lerchphi, expand_func\n    >>> from sympy.abc import z, s, a\n    >>> expand_func(lerchphi(1, s, a))\n    zeta(s, a)\n\n    More generally, if $z$ is a root of unity, the Lerch transcendent\n    reduces to a sum of Hurwitz zeta functions:\n\n    >>> expand_func(lerchphi(-1, s, a))\n    zeta(s, a/2)/2**s - zeta(s, a/2 + 1/2)/2**s\n\n    If $a=1$, the Lerch transcendent reduces to the polylogarithm:\n\n    >>> expand_func(lerchphi(z, s, 1))\n    polylog(s, z)/z\n\n    More generally, if $a$ is rational, the Lerch transcendent reduces\n    to a sum of polylogarithms:\n\n    >>> from sympy import S\n    >>> expand_func(lerchphi(z, s, S(1)/2))\n    2**(s - 1)*(polylog(s, sqrt(z))/sqrt(z) -\n                polylog(s, sqrt(z)*exp_polar(I*pi))/sqrt(z))\n    >>> expand_func(lerchphi(z, s, S(3)/2))\n    -2**s/z + 2**(s - 1)*(polylog(s, sqrt(z))/sqrt(z) -\n                          polylog(s, sqrt(z)*exp_polar(I*pi))/sqrt(z))/z\n\n    The derivatives with respect to $z$ and $a$ can be computed in\n    closed form:\n\n    >>> lerchphi(z, s, a).diff(z)\n    (-a*lerchphi(z, s, a) + lerchphi(z, s - 1, a))/z\n    >>> lerchphi(z, s, a).diff(a)\n    -s*lerchphi(z, s + 1, a)\n\n    See Also\n    ========\n\n    polylog, zeta\n\n    References\n    ==========\n\n    .. [1] Bateman, H.; Erdelyi, A. (1953), Higher Transcendental Functions,\n           Vol. I, New York: McGraw-Hill. Section 1.11.\n    .. [2] https://dlmf.nist.gov/25.14\n    .. [3] https://en.wikipedia.org/wiki/Lerch_transcendent",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `lerchphi` is a subclass of `DefinedFunction` cannot be verified from the documentation - this is a code-level detail not mentioned in the doc.",
      "The claim that `lerchphi` does not define a `@classmethod eval` method cannot be verified from the documentation, though the doc does say 'it does not automatically evaluate to simpler functions'.",
      "The detailed description of `_eval_expand_func` branches is plausible but cannot be verified from the documentation alone - only the examples showing expand_func behavior are documented.",
      "The claim about `fdiff` raising `ArgumentIndexError` for argindex=2 is not contradicted by the doc (the doc only shows derivatives w.r.t. z and a, not s), but is not explicitly stated either.",
      "The `_eval_rewrite_helper` pattern, `try_lerchphi` in hyperexpand, printing details, `_dilogtable`, and `polylog.eval` classmethod details are all code-level claims that go beyond the documentation scope.",
      "The documentation confirms: expand_func(lerchphi(1,s,a)) = zeta(s,a), expand_func with roots of unity gives Hurwitz zeta sums, expand_func with a=1 gives polylog(s,z)/z, and the derivative formulas match exactly what the answer describes."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about paper references, printing behavior, and hyperexpand internals rely on documentation not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_24",
    "repo": "sympy",
    "question": "How does `is_matmul_valid` validate matrix multiplication shapes, and how does it differ from the validation done during `MatMul` construction?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `is_matmul_valid` function in `sympy/matrices/expressions/_shape.py` works by:\n\n1. **Filtering non-matrix arguments**: It uses a list comprehension with `isinstance(arg, MatrixExpr)` to filter out scalar expressions from the argument list, only considering matrix arguments for shape validation.\n\n2. **Extracting shapes**: It calls `.shape` on each matrix argument and uses `zip(*...)` to separate rows and columns into separate tuples.\n\n3. **Building constraint from adjacent column-row pairs**: It zips `cols[:-1]` with `rows[1:]` to pair each matrix's column dimension with the next matrix's row dimension, creating `Eq(col_i, row_{i+1})` conditions for each adjacent pair.\n\n4. **Returning a symbolic Boolean**: It wraps all `Eq` conditions in `And(*)`. When there's only one matrix (or the zip produces no pairs), `And()` with no arguments returns the identity because `LatticeOp.__new__` returns `sympify(cls.identity)` when `_args` is empty (in `sympy/core/operations.py`). When there's exactly one equality, `And` with a single-element frozenset returns that element directly via `set(_args).pop()`.\n\nThis is distinct from `validate_matmul_integer` (also in `_shape.py`), which is actually used during `MatMul.__new__` construction. `validate_matmul_integer` only raises a `ShapeError` when both dimensions being compared are concrete integers (`isinstance(i, (int, Integer))`) and they differ. It does NOT perform symbolic comparison - it silently passes when dimensions are symbolic. This means `MatMul` construction allows potentially invalid symbolic multiplications, while `is_matmul_valid` returns the symbolic `Eq` condition that must hold for the multiplication to be valid.\n\nThe companion function `is_matadd_valid` works similarly but checks that ALL rows are equal AND ALL columns are equal (comparing consecutive args), while `is_matmul_valid` only checks that adjacent matrices have matching inner dimensions.\n\n`MatMul.__new__` (in `matmul.py`) calls `validate` (aliased from `validate_matmul_integer`) on the matrices extracted via `as_coeff_matrices()`, which separates scalars from matrix arguments. The `check` parameter to `MatMul.__new__` is deprecated since version 1.11.",
    "rubric": [
      "Explains that is_matmul_valid filters out non-matrix (scalar) arguments using isinstance(arg, MatrixExpr) before checking shapes",
      "Explains that is_matmul_valid extracts shapes via .shape and uses zip to separate rows and columns, then pairs cols[:-1] with rows[1:] to check adjacent inner dimensions",
      "Explains that is_matmul_valid returns a symbolic Boolean by wrapping Eq conditions in And(), and that And() with no arguments returns its identity (True)",
      "Explains that MatMul.__new__ uses validate_matmul_integer (imported as validate), which only raises ShapeError when both compared dimensions are concrete integers (int or Integer) that differ",
      "Explains the key distinction: validate_matmul_integer silently passes on symbolic dimensions while is_matmul_valid returns the symbolic Eq condition that must hold"
    ],
    "key_files": [
      "sympy/matrices/expressions/_shape.py",
      "sympy/matrices/expressions/matmul.py",
      "sympy/core/operations.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/_shape.py] sympy.matrices.expressions._shape.is_matmul_valid\nReturn the symbolic condition how ``MatMul`` makes sense\n\n    Parameters\n    ==========\n\n    args\n        The list of arguments of matrices and scalar expressions to be tested\n        for.\n\n    Examples\n    ========\n\n    >>> from sympy import MatrixSymbol, symbols\n    >>> from sympy.matrices.expressions._shape import is_matmul_valid\n\n    >>> m, n, p, q = symbols('m n p q')\n    >>> A = MatrixSymbol('A', m, n)\n    >>> B = MatrixSymbol('B', p, q)\n    >>> is_matmul_valid(A, B)\n    Eq(n, p)",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most implementation details (line numbers, internal logic about And.identity, validate_matmul_integer, is_matadd_valid, MatMul.__new__ internals) go far beyond what the documentation provides and cannot be verified from it alone, but they are plausible code-level extrapolations.",
      "The core claim about what is_matmul_valid does \u2014 returning a symbolic condition like Eq(n, p) for adjacent matrix dimension matching \u2014 is directly supported by the documentation's example."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped source code; only minor line number references and one detail about And.identity=true (from unshown boolalg source) needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_25",
    "repo": "sympy",
    "question": "What is the overall architecture of BlockMatrix and BlockDiagMatrix, including how block_collapse, determinant, inverse, and the assumptions system interact with them?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `BlockMatrix` class in `sympy/matrices/expressions/blockmatrix.py` is a subclass of `MatrixExpr` (which itself inherits from `Expr` via `sympy.core.expr`).\n\n**Architecture & Class Hierarchy:**\n- `BlockMatrix` extends `MatrixExpr`, which extends `Expr` (from `sympy.core.expr`). `BlockMatrix.__new__` bypasses `MatrixExpr.__new__` entirely\u2014it directly calls `Basic.__new__(cls, mat)` after constructing an `ImmutableDenseMatrix` from the input rows.\n- `BlockDiagMatrix` is a subclass of `BlockMatrix` that also bypasses the parent `BlockMatrix.__new__`\u2014it calls `Basic.__new__(BlockDiagMatrix, *mats)` directly, storing individual matrices as `args` rather than a single `ImmutableDenseMatrix`.\n\n**Key Implementation Details:**\n1. The `blocks` property differs between the two classes: `BlockMatrix.blocks` simply returns `self.args[0]` (the stored `ImmutableDenseMatrix`), while `BlockDiagMatrix.blocks` dynamically constructs an `ImmutableDenseMatrix` with `ZeroMatrix` off-diagonal entries each time it's accessed.\n\n2. The `block_collapse` function uses the `sympy.strategies` framework for rule-based expression rewriting. It applies a `bottom_up` traversal with `exhaust` (repeated application until no change) using `expr_fns` (from `sympy/strategies/util.py`), which differs from `basic_fns` by using `lambda op, *args: op(*args)` as the `new` function instead of `Basic.__new__`. The `conditioned_rl` maps expression types to specific rule functions: `MatAdd` \u2192 `do_one(bc_matadd, bc_block_plus_ident)`, `MatMul` \u2192 `do_one(bc_matmul, bc_dist)`, `MatPow` \u2192 `bc_matmul`, `Transpose` \u2192 `bc_transpose`, `Inverse` \u2192 `bc_inverse`, `BlockMatrix` \u2192 `do_one(bc_unpack, deblock)`. All rules are only applied when the condition `hasbm` is true (the expression has a `BlockMatrix` somewhere).\n\n3. The `_eval_determinant` method for `BlockMatrix` uses the assumption system (`ask(Q.invertible(A))`) to decide which formula to use. For a 2x2 block matrix, if A is known invertible, it returns `det(A)*det(D - C*A.I*B)`; if not, it tries D. `BlockDiagMatrix` overrides this: if all blocks are square, it returns the product of individual determinants; if any block is non-square, it returns `S.Zero`.\n\n4. The `bc_inverse` function is mapped to `Inverse` in the `block_collapse` typed dispatch.\n\n5. The `LUdecomposition` method takes square roots of matrices: it computes `A = A**S.Half` and `Q = self.schur()**S.Half`, constructing L and U from these half-power matrices. This is different from the LDU decomposition which uses the full A and its inverse.\n\n6. `BlockMatrix` does not define `_eval_inverse`\u2014inverse handling is done entirely through `block_collapse` and its `bc_inverse` rule. `BlockDiagMatrix` does define `_eval_inverse`, returning `BlockDiagMatrix(*[mat.inverse() for mat in self.args])` if all blocks are square.\n\n7. The `_entry` method handles symbolic index access with a fallback: if it can't determine which block an index falls into (when the comparison `i < numrows` is neither True nor False), it returns an unevaluated `MatrixElement(self, orig_i, orig_j)` instead of raising an error.\n\n8. The `structurally_equal` method is used by `_blockadd` to determine if two BlockMatrices can be added block-wise. It checks shape, blockshape, rowblocksizes, and colblocksizes.\n\n9. In the assumptions system (`sympy/assumptions/handlers/matrices.py`), `InvertiblePredicate` has a dedicated handler for `BlockMatrix` that uses `reblock_2x2` and tests all four sub-matrices (A, B, C, D) for invertibility of the Schur complement. `BlockDiagMatrix` also has its own handler using `fuzzy_and` over the invertibility of diagonal blocks.\n\n10. Printing: The `str` printer's `_print_BlockMatrix` (in `sympy/printing/str.py`) delegates to printing `B.blocks`, which is the underlying `ImmutableDenseMatrix`.",
    "rubric": [
      "Explains that BlockMatrix extends MatrixExpr and its __new__ bypasses MatrixExpr.__new__, calling Basic.__new__ directly after constructing an ImmutableDenseMatrix",
      "Explains that BlockDiagMatrix subclasses BlockMatrix and also bypasses the parent __new__, calling Basic.__new__(BlockDiagMatrix, *mats) with individual matrices as args",
      "Describes how the blocks property differs: BlockMatrix.blocks returns self.args[0] while BlockDiagMatrix.blocks dynamically constructs an ImmutableDenseMatrix with ZeroMatrix off-diagonal entries",
      "Describes block_collapse's use of sympy.strategies (bottom_up traversal with exhaust) and its typed dispatch mapping (MatAdd, MatMul, MatPow, Transpose, Inverse, BlockMatrix) with the hasbm condition",
      "Notes that expr_fns differs from basic_fns by using lambda op, *args: op(*args) as the 'new' function instead of Basic.__new__",
      "Explains _eval_determinant for BlockMatrix uses ask(Q.invertible(A)) for the 2x2 case to choose between det(A)*det(D - C*A.I*B) and det(D)*det(A - B*D.I*C)",
      "Explains BlockDiagMatrix._eval_determinant returns product of individual determinants if all blocks are square, otherwise S.Zero",
      "Notes BlockMatrix has no _eval_inverse; inverse is handled through block_collapse's bc_inverse rule; BlockDiagMatrix defines _eval_inverse inverting each diagonal block",
      "Describes _entry's fallback behavior: returns unevaluated MatrixElement(self, orig_i, orig_j) when index comparison is indeterminate",
      "Describes InvertiblePredicate handlers: BlockMatrix handler uses reblock_2x2 and tests all four sub-matrices' Schur complements; BlockDiagMatrix handler uses fuzzy_and over diagonal blocks"
    ],
    "key_files": [
      "sympy/matrices/expressions/blockmatrix.py",
      "sympy/matrices/expressions/matexpr.py",
      "sympy/strategies/core.py",
      "sympy/strategies/util.py",
      "sympy/assumptions/handlers/matrices.py",
      "sympy/printing/str.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/blockmatrix.py] sympy.matrices.expressions.blockmatrix.BlockMatrix\nsympy.matrices.expressions.blockmatrix.BlockMatrix:\n  A BlockMatrix is a Matrix comprised of other matrices.\n\n    The submatrices are stored in a SymPy Matrix object but accessed as part of\n    a Matrix Expression\n\n    >>> from sympy import (MatrixSymbol, BlockMatrix, symbols,\n    ...     Identity, ZeroMatrix, block_collapse)\n    >>> n,m,l = symbols('n m l')\n    >>> X = MatrixSymbol('X', n, n)\n    >>> Y = MatrixSymbol('Y', m, m)\n    >>> Z = MatrixSymbol('Z', n, m)\n    >>> B = BlockMatrix([[X, Z], [ZeroMatrix(m,n), Y]])\n    >>> print(B)\n    Matrix([\n    [X, Z],\n    [0, Y]])\n\n    >>> C = BlockMatrix([[Identity(n), Z]])\n    >>> print(C)\n    Matrix([[I, Z]])\n\n    >>> print(block_collapse(C*B))\n    Matrix([[X, Z + Z*Y]])\n\n    Some matrices might be comprised of rows of blocks with\n    the matrices in each row having the same height and the\n    rows all having the same total number of columns but\n    not having the same number of columns for each matrix\n    in each row. In this case, the matrix is not a block\n    matrix and should be instantiated by Matrix.\n\n    >>> from sympy import ones, Matrix\n    >>> dat = [\n    ... [ones(3,2), ones(3,3)*2],\n    ... [ones(2,3)*3, ones(2,2)*4]]\n    ...\n    >>> BlockMatrix(dat)\n    Traceback (most recent call last):\n    ...\n    ValueError:\n    Although this matrix is comprised of blocks, the blocks do not fill\n    the matrix in a size-symmetric fashion. To create a full matrix from\n    these arguments, pass them directly to Matrix.\n    >>> Matrix(dat)\n    Matrix([\n    [1, 1, 2, 2, 2],\n    [1, 1, 2, 2, 2],\n    [1, 1, 2, 2, 2],\n    [3, 3, 3, 4, 4],\n    [3, 3, 3, 4, 4]])\n\n    See Also\n    ========\n    sympy.matrices.matrixbase.MatrixBase.irregular\n\nsympy.matrices.expressions.blockmatrix.BlockMatrix.transpose:\n  Return transpose of matrix.\n\n        Examples\n        ========\n\n        >>> from sympy import MatrixSymbol, BlockMatrix, ZeroMatrix\n        >>> from sympy.abc import m, n\n        >>> X = MatrixSymbol('X', n, n)\n        >>> Y = MatrixSymbol('Y', m, m)\n        >>> Z = MatrixSymbol('Z', n, m)\n        >>> B = BlockMatrix([[X, Z], [ZeroMatrix(m,n), Y]])\n        >>> B.transpose()\n        Matrix([\n        [X.T,  0],\n        [Z.T, Y.T]])\n        >>> _.transpose()\n        Matrix([\n        [X, Z],\n        [0, Y]])\n\nsympy.matrices.expressions.blockmatrix.BlockMatrix.schur:\n  Return the Schur Complement of the 2x2 BlockMatrix\n\n        Parameters\n        ==========\n\n        mat : String, optional\n            The matrix with respect to which the\n            Schur Complement is calculated. 'A' is\n            used by default\n\n        generalized : bool, optional\n            If True, returns the generalized Schur\n            Component which uses Moore-Penrose Inverse\n\n        Examples\n        ========\n\n        >>> from sympy import symbols, MatrixSymbol, BlockMatrix\n        >>> m, n = symbols('m n')\n        >>> A = MatrixSymbol('A', n, n)\n        >>> B = MatrixSymbol('B', n, m)\n        >>> C = MatrixSymbol('C', m, n)\n        >>> D = MatrixSymbol('D', m, m)\n        >>> X = BlockMatrix([[A, B], [C, D]])\n\n        The default Schur Complement is evaluated with \"A\"\n\n        >>> X.schur()\n        -C*A**(-1)*B + D\n        >>> X.schur('D')\n        A - B*D**(-1)*C\n\n        Schur complement with non-invertible matrices is not\n        defined. Instead, the generalized Schur complement can\n        be calculated which uses the Moore-Penrose Inverse. To\n        achieve this, `generalized` must be set to `True`\n\n        >>> X.schur('B', generalized=True)\n        C - D*(B.T*B)**(-1)*B.T*A\n        >>> X.schur('C', generalized=True)\n        -A*(C.T*C)**(-1)*C.T*D + B\n\n        Returns\n        =======\n\n        M : Matrix\n            The Schur Complement Matrix\n\n        Raises\n        ======\n\n        ShapeError\n            If the block matrix is not a 2x2 matrix\n\n        NonInvertibleMatrixError\n            If given matrix is non-invertible\n\n        References\n        ==========\n\n        .. [1] Wikipedia Article on Schur Component : https://en.wikipedia.org/wiki/Schur_complement\n\n        See Also\n        ========\n\n        sympy.matrices.matrixbase.MatrixBase.pinv\n\nsympy.matrices.expressions.blockmatrix.BlockMatrix.LDUdecomposition:\n  Returns the Block LDU decomposition of\n        a 2x2 Block Matrix\n\n        Returns\n        =======\n\n        (L, D, U) : Matrices\n            L : Lower Diagonal Matrix\n            D : Diagonal Matrix\n            U : Upper Diagonal Matrix\n\n        Examples\n        ========\n\n        >>> from sympy import symbols, MatrixSymbol, BlockMatrix, block_collapse\n        >>> m, n = symbols('m n')\n        >>> A = MatrixSymbol('A', n, n)\n        >>> B = MatrixSymbol('B', n, m)\n        >>> C = MatrixSymbol('C', m, n)\n        >>> D = MatrixSymbol('D', m, m)\n        >>> X = BlockMatrix([[A, B], [C, D]])\n        >>> L, D, U = X.LDUdecomposition()\n        >>> block_collapse(L*D*U)\n        Matrix([\n        [A, B],\n        [C, D]])\n\n        Raises\n        ======\n\n        ShapeError\n            If the block matrix is not a 2x2 matrix\n\n        NonInvertibleMatrixError\n            If the matrix \"A\" is non-invertible\n\n        See Also\n        ========\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.UDLdecomposition\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.LUdecomposition\n\nsympy.matrices.expressions.blockmatrix.BlockMatrix.UDLdecomposition:\n  Returns the Block UDL decomposition of\n        a 2x2 Block Matrix\n\n        Returns\n        =======\n\n        (U, D, L) : Matrices\n            U : Upper Diagonal Matrix\n            D : Diagonal Matrix\n            L : Lower Diagonal Matrix\n\n        Examples\n        ========\n\n        >>> from sympy import symbols, MatrixSymbol, BlockMatrix, block_collapse\n        >>> m, n = symbols('m n')\n        >>> A = MatrixSymbol('A', n, n)\n        >>> B = MatrixSymbol('B', n, m)\n        >>> C = MatrixSymbol('C', m, n)\n        >>> D = MatrixSymbol('D', m, m)\n        >>> X = BlockMatrix([[A, B], [C, D]])\n        >>> U, D, L = X.UDLdecomposition()\n        >>> block_collapse(U*D*L)\n        Matrix([\n        [A, B],\n        [C, D]])\n\n        Raises\n        ======\n\n        ShapeError\n            If the block matrix is not a 2x2 matrix\n\n        NonInvertibleMatrixError\n            If the matrix \"D\" is non-invertible\n\n        See Also\n        ========\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.LDUdecomposition\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.LUdecomposition\n\nsympy.matrices.expressions.blockmatrix.BlockMatrix.LUdecomposition:\n  Returns the Block LU decomposition of\n        a 2x2 Block Matrix\n\n        Returns\n        =======\n\n        (L, U) : Matrices\n            L : Lower Diagonal Matrix\n            U : Upper Diagonal Matrix\n\n        Examples\n        ========\n\n        >>> from sympy import symbols, MatrixSymbol, BlockMatrix, block_collapse\n        >>> m, n = symbols('m n')\n        >>> A = MatrixSymbol('A', n, n)\n        >>> B = MatrixSymbol('B', n, m)\n        >>> C = MatrixSymbol('C', m, n)\n        >>> D = MatrixSymbol('D', m, m)\n        >>> X = BlockMatrix([[A, B], [C, D]])\n        >>> L, U = X.LUdecomposition()\n        >>> block_collapse(L*U)\n        Matrix([\n        [A, B],\n        [C, D]])\n\n        Raises\n        ======\n\n        ShapeError\n            If the block matrix is not a 2x2 matrix\n\n        NonInvertibleMatrixError\n            If the matrix \"A\" is non-invertible\n\n        See Also\n        ========\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.UDLdecomposition\n        sympy.matrices.expressions.blockmatrix.BlockMatrix.LDUdecomposition",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 5 claims LUdecomposition computes A = A**S.Half and Q = self.schur()**S.Half, but the documentation shows LUdecomposition returns (L, U) where block_collapse(L*U) reconstructs the original matrix. The documentation does not describe half-power matrices. This claim cannot be verified from the provided documentation and may be a hallucination about implementation details.",
      "The answer claims BlockMatrix extends MatrixExpr at 'line 25' and various other line numbers - these are implementation details not verifiable from the documentation provided, though they are plausible code-level details.",
      "The answer states LDUdecomposition raises NonInvertibleMatrixError if matrix 'A' is non-invertible, which matches the documentation. However, the claim that LUdecomposition uses half-power matrices seems inconsistent with the documented behavior showing block_collapse(L*U) = original matrix.",
      "The documentation shows schur() returns expressions like '-C*A**(-1)*B + D', and the answer's claim about _eval_determinant using 'det(A)*det(D - C*A.I*B)' is consistent with the Schur complement formula, though not explicitly stated in the docs."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in purpose description, LaTeX/pretty printer claims not verifiable from shown code, and some truncated function details.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_26",
    "repo": "sympy",
    "question": "Why does the DFT matrix scale entries by 1/sqrt(n) instead of using an asymmetric normalization convention?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The DFT matrix is scaled with `1/sqrt(n)` (rather than being unscaled or using `1/n` scaling on one side), which makes it **unitary** \u2014 meaning `DFT * DFT^H = I`. This is evidenced by several implementation details:\n\n1. **Symmetric scaling between DFT and IDFT**: Both `DFT._entry` and `IDFT._entry` divide by `sqrt(self.n)`, making the forward and inverse transforms symmetric. The only difference is the sign of the exponent: DFT uses `w**(i*j)` while IDFT uses `w**(-i*j)` where `w = exp(-2*pi*I/n)`. This means both matrices independently have the `1/sqrt(n)` factor, ensuring `DFT * IDFT = I` (identity) rather than requiring an asymmetric `1/n` correction on one side.\n\n2. **Inverse relationship via `_eval_inverse`**: `DFT._eval_inverse()` directly returns `IDFT(self.n)` and vice versa, without any scaling correction. This is only valid because of the unitary normalization \u2014 if the scaling were different, the inverse would require additional scalar factors.\n\n3. **Cancellation mechanism in `combine_powers`**: When computing `DFT(n)*IDFT(n)`, the `combine_powers` rule in `matmul.py` calls `B_base.inverse()` on `IDFT(n)`, which returns `DFT(n)`. Since `A_base == B_base_inv` (both are `DFT(n)`), it computes `new_exp = 1 - 1 = 0`, and `MatPow(DFT(n), 0).doit(deep=False)` returns `Identity(base.rows)` (in `matpow.py`'s `doit` method where `exp == S.Zero`). This clean cancellation to identity is only correct because of the unitary scaling.\n\n4. **Validation in `_check_dim`**: The `_check_dim` classmethod inherited from `MatrixExpr` rejects float dimensions and negative integers by using `check_assumptions(dim, integer=True, nonnegative=True)`, which means DFT(0) is valid while `DFT(-1)` and `DFT(2.0)` raise `ValueError`.",
    "rubric": [
      "Explains that 1/sqrt(n) scaling makes the DFT matrix unitary (DFT * DFT^H = I)",
      "Notes that both DFT._entry and IDFT._entry divide by sqrt(n), making forward and inverse transforms symmetric (only the exponent sign differs)",
      "Mentions that _eval_inverse returns IDFT (or DFT) directly without any additional scaling correction, which is only valid because of unitary normalization",
      "Describes how combine_powers in matmul.py enables DFT(n)*IDFT(n) to cancel to Identity via the inverse relationship and MatPow(base, 0) returning Identity"
    ],
    "key_files": [
      "sympy/matrices/expressions/fourier.py",
      "sympy/matrices/expressions/matmul.py",
      "sympy/matrices/expressions/matpow.py",
      "sympy/matrices/expressions/matexpr.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/fourier.py] sympy.matrices.expressions.fourier.DFT\nsympy.matrices.expressions.fourier.DFT:\n  Returns a discrete Fourier transform matrix. The matrix is scaled\n    with :math:`\\frac{1}{\\sqrt{n}}` so that it is unitary.\n\n    Parameters\n    ==========\n\n    n : integer or Symbol\n        Size of the transform.\n\n    Examples\n    ========\n\n    >>> from sympy.abc import n\n    >>> from sympy.matrices.expressions.fourier import DFT\n    >>> DFT(3)\n    DFT(3)\n    >>> DFT(3).as_explicit()\n    Matrix([\n    [sqrt(3)/3,                sqrt(3)/3,                sqrt(3)/3],\n    [sqrt(3)/3, sqrt(3)*exp(-2*I*pi/3)/3,  sqrt(3)*exp(2*I*pi/3)/3],\n    [sqrt(3)/3,  sqrt(3)*exp(2*I*pi/3)/3, sqrt(3)*exp(-2*I*pi/3)/3]])\n    >>> DFT(n).shape\n    (n, n)\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/DFT_matrix",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claims 1-5 go far beyond what the documentation states, referencing internal implementation details like `_entry`, `_eval_inverse`, `combine_powers`, `_check_dim`, and test suite assertions that are not present in the provided documentation chunk.",
      "The core claim that DFT is scaled by 1/sqrt(n) to be unitary is directly supported by the documentation.",
      "Claim 4 about `DFT(0)` being valid and `DFT(-1)` raising ValueError, and `DFT(2.0)` raising ValueError, are implementation details not verifiable from the documentation and could be hallucinated.",
      "Claim 5 about `ask(Q.unitary(DFT(4)))` passing and `Abs(simplify(det(Matrix(DFT(4))))) == 1` being in the test suite cannot be verified from the documentation alone.",
      "None of the claims directly contradict the documentation; they are plausible code-level extrapolations but unverifiable from the given doc chunk."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only line number references and test suite behavior claims needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_28",
    "repo": "sympy",
    "question": "How does `FunctionMatrix` work internally \u2014 from construction and validation through entry access, powers, and trace evaluation?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `FunctionMatrix` class represents a matrix through a function/lambda, where entries are computed on access rather than stored. Here's how it works at the implementation level:\n\n**Class Hierarchy and Construction (`sympy/matrices/expressions/funcmatrix.py`):**\n- `FunctionMatrix` extends `MatrixExpr`, which extends `Expr` (from `sympy.core.expr`).\n- In `__new__`, rows and cols are `_sympify`'d (not `sympify`), then validated via `cls._check_dim()` (inherited from `MatrixExpr`). The lambda argument uses full `sympify()`, which is what enables string-based lambda parsing (e.g., `'lambda i, j: i + j'` gets parsed into a SymPy `Lambda`).\n- Validation checks: (1) the lambda must be a `FunctionClass` or `Lambda` instance after sympification; (2) `2 in lamda.nargs` must be True (checking that it accepts 2 arguments). For `FunctionClass` (like `KroneckerDelta` or undefined functions), `nargs` returns either a `FiniteSet` of allowed arities or `S.Naturals0` (the set of all natural numbers including 0), as visible in the `FunctionClass.nargs` property.\n- If the lambda is a `FunctionClass` (not a `Lambda` instance), it's automatically wrapped: two `Dummy` variables `i`, `j` are created and the function is converted to `Lambda((i, j), lamda(i, j))`. This means `KroneckerDelta` becomes `Lambda((i, j), KroneckerDelta(i, j))`.\n\n**Entry Access:**\n- `_entry(self, i, j, **kwargs)` simply calls `self.lamda(i, j)`.\n- `__getitem__` is inherited from `MatrixExpr` and calls `_entry` after sympifying indices and checking `valid_index`.\n\n**Lazy Evaluation of Powers:**\n- `MatrixExpr.__pow__` creates `MatPow(self, other).doit()`. In `MatPow.doit()`, for exponents > 1 that aren't -1, 0, or 1, it calls `base._eval_power(exp)`. `FunctionMatrix` does not override `_eval_power`, so the default `MatrixExpr._eval_power` returns `MatPow(self, exp)` \u2014 keeping it as an unevaluated expression.\n- When indexing into a `MatPow` (e.g., `(Y**2)[10,10]`), `MatPow._entry` calls `self.doit()` first. If it's still a `MatPow` with a positive integer exponent, it creates `MatMul(*[A.base for k in range(A.exp)])` and indexes into that. If the shape is not symbolic, it falls back to `_get_explicit_matrix()` which computes `self.base.as_explicit()**self.exp`.\n\n**Trace Evaluation (`_eval_trace`):**\n- FunctionMatrix overrides `_eval_trace` (the base `MatrixExpr` returns `None`). It creates `Trace(self).rewrite(Sum).doit()`. The `_eval_rewrite_as_Sum` method in `Trace` generates `Sum(self.arg[i, i], (i, 0, self.arg.rows - 1))` and calls `.doit()` on it. This accesses each diagonal entry via `_entry(i, i)` which evaluates the lambda.\n- Note: `Trace.doit()` calls `arg._eval_trace()`, and `FunctionMatrix._eval_trace` creates a new `Trace(self)` which it rewrites as a `Sum` \u2014 this works because `rewrite(Sum)` bypasses the `doit()` \u2192 `_eval_trace()` cycle.\n\n**Real/Imaginary Decomposition:**\n- `FunctionMatrix._eval_as_real_imag` overrides the base class version. The base `MatrixExpr._eval_as_real_imag` uses the formula `(self + conjugate(self))/2` and `(self - conjugate(self))/(2i)`. Instead, FunctionMatrix converts itself to an explicit `Matrix(self)` and then applies `re()` and `im()` elementwise.\n\n**Dimension Validation (`_check_dim` from `MatrixExpr`):**\n- Uses `check_assumptions(dim, integer=True, nonnegative=True)` from `sympy.core.assumptions`. Also explicitly rejects `Float` values (`not dim.is_Float`). Returns `None` (allowing construction) when assumptions can't be conclusively checked (e.g., a generic symbol without integer/nonnegative assumptions), and only raises `ValueError` when `ok is False` (definitively invalid).\n\n**Shape Property:**\n- `shape` returns `self.args[0:2]` (the first two args, rows and cols), while `lamda` returns `self.args[2]`.",
    "rubric": [
      "Explains that FunctionMatrix extends MatrixExpr, and that __new__ uses _sympify for rows/cols but full sympify for the lambda (enabling string-based lambda parsing into Lambda objects)",
      "Describes the validation: lambda must be FunctionClass or Lambda after sympification, and 2 must be in lamda.nargs",
      "Explains that FunctionClass arguments (like KroneckerDelta) are auto-wrapped into Lambda via Dummy variables i, j",
      "Describes _entry(i, j) simply calls self.lamda(i, j), and __getitem__ is inherited from MatrixExpr which sympifies indices and checks valid_index",
      "Explains lazy power evaluation: MatPow.doit() calls base._eval_power(exp), FunctionMatrix doesn't override it, so MatExpr._eval_power returns an unevaluated MatPow; indexing into MatPow._entry expands it to MatMul for positive integer exponents or falls back to _get_explicit_matrix",
      "Describes _eval_trace: creates Trace(self).rewrite(Sum).doit(), which triggers _eval_rewrite_as_Sum generating Sum(self.arg[i,i], (i, 0, rows-1))",
      "Mentions _eval_as_real_imag converts to explicit Matrix(self) and applies re/im elementwise, unlike the base class formula using conjugate",
      "Explains _check_dim uses check_assumptions(dim, integer=True, nonnegative=True) and rejects Float values, returning None (allowing construction) when assumptions are inconclusive"
    ],
    "key_files": [
      "sympy/matrices/expressions/funcmatrix.py",
      "sympy/matrices/expressions/matexpr.py",
      "sympy/matrices/expressions/matpow.py",
      "sympy/matrices/expressions/trace.py",
      "sympy/core/function.py",
      "sympy/matrices/expressions/tests/test_funcmatrix.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/funcmatrix.py] sympy.matrices.expressions.funcmatrix.FunctionMatrix\nsympy.matrices.expressions.funcmatrix.FunctionMatrix:\n  Represents a matrix using a function (``Lambda``) which gives\n    outputs according to the coordinates of each matrix entries.\n\n    Parameters\n    ==========\n\n    rows : nonnegative integer. Can be symbolic.\n\n    cols : nonnegative integer. Can be symbolic.\n\n    lamda : Function, Lambda or str\n        If it is a SymPy ``Function`` or ``Lambda`` instance,\n        it should be able to accept two arguments which represents the\n        matrix coordinates.\n\n        If it is a pure string containing Python ``lambda`` semantics,\n        it is interpreted by the SymPy parser and casted into a SymPy\n        ``Lambda`` instance.\n\n    Examples\n    ========\n\n    Creating a ``FunctionMatrix`` from ``Lambda``:\n\n    >>> from sympy import FunctionMatrix, symbols, Lambda, MatPow\n    >>> i, j, n, m = symbols('i,j,n,m')\n    >>> FunctionMatrix(n, m, Lambda((i, j), i + j))\n    FunctionMatrix(n, m, Lambda((i, j), i + j))\n\n    Creating a ``FunctionMatrix`` from a SymPy function:\n\n    >>> from sympy import KroneckerDelta\n    >>> X = FunctionMatrix(3, 3, KroneckerDelta)\n    >>> X.as_explicit()\n    Matrix([\n    [1, 0, 0],\n    [0, 1, 0],\n    [0, 0, 1]])\n\n    Creating a ``FunctionMatrix`` from a SymPy undefined function:\n\n    >>> from sympy import Function\n    >>> f = Function('f')\n    >>> X = FunctionMatrix(3, 3, f)\n    >>> X.as_explicit()\n    Matrix([\n    [f(0, 0), f(0, 1), f(0, 2)],\n    [f(1, 0), f(1, 1), f(1, 2)],\n    [f(2, 0), f(2, 1), f(2, 2)]])\n\n    Creating a ``FunctionMatrix`` from Python ``lambda``:\n\n    >>> FunctionMatrix(n, m, 'lambda i, j: i + j')\n    FunctionMatrix(n, m, Lambda((i, j), i + j))\n\n    Example of lazy evaluation of matrix product:\n\n    >>> Y = FunctionMatrix(1000, 1000, Lambda((i, j), i + j))\n    >>> isinstance(Y*Y, MatPow) # this is an expression object\n    True\n    >>> (Y**2)[10,10] # So this is evaluated lazily\n    342923500\n\n    Notes\n    =====\n\n    This class provides an alternative way to represent an extremely\n    dense matrix with entries in some form of a sequence, in a most\n    sparse way.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only printer-related claims and one minor design rationale needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_27",
    "repo": "sympy",
    "question": "Where does the data flow go when you multiply `IDFT(4)*DFT(4)` and how does it simplify to the identity matrix?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `IDFT` class in `sympy/matrices/expressions/fourier.py` is a subclass of `DFT`, which itself inherits from `MatrixExpr`. The data/control flow when computing `IDFT(4)*DFT(4)` proceeds as follows:\n\n1. `MatrixExpr.__mul__` creates `MatMul(IDFT(4), DFT(4)).doit()`\n2. `MatMul.doit()` calls `canonicalize(MatMul(...))`\n3. `canonicalize` is defined as `exhaust(typed({MatMul: do_one(*rules)}))` where rules include `combine_powers`\n4. `combine_powers` iterates through args; for A=IDFT(4), B=DFT(4), since they are not equal bases, it calls `B_base.inverse()` which routes to `DFT._eval_inverse()` returning `IDFT(self.n)`\n5. Since `A_base == B_base_inv` (both are IDFT(4)), `new_exp = A_exp - B_exp = 0`, and it creates `MatPow(IDFT(4), 0).doit(deep=False)`\n6. `MatPow.doit()` detects `exp == S.Zero` and returns `Identity(base.rows)` where `base.rows` comes from `MatrixExpr.rows` property which returns `self.shape[0]`\n\nThe `IDFT._entry(i, j)` method computes `w**(-i*j) / sqrt(self.n)` where `w = exp(-2*pi*I/n)`, while DFT's entry computes `w**(i*j) / sqrt(self.n)`. IDFT inherits `__new__`, `n`, and `shape` properties from DFT (which validates dimensions via `MatrixExpr._check_dim`). The `_check_dim` classmethod uses `check_assumptions(dim, integer=True, nonnegative=True)` and raises ValueError if the dimension is a Float, has negative value, or is known to be non-integer.",
    "rubric": [
      "Explains that MatrixExpr.__mul__ creates MatMul(IDFT(4), DFT(4)).doit()",
      "Mentions MatMul.doit() calls canonicalize which applies rules including combine_powers",
      "Explains that combine_powers calls B_base.inverse() (i.e., DFT._eval_inverse()) which returns IDFT(self.n), matching A_base",
      "Explains that since A_base == B_base_inv, new_exp becomes A_exp - B_exp = 0, creating MatPow(IDFT(4), 0).doit(deep=False)",
      "Explains that MatPow.doit() detects exp == S.Zero and returns Identity(base.rows)",
      "Mentions IDFT is a subclass of DFT which inherits from MatrixExpr",
      "Describes IDFT._entry computing w**(-i*j)/sqrt(n) vs DFT._entry computing w**(i*j)/sqrt(n)"
    ],
    "key_files": [
      "sympy/matrices/expressions/fourier.py",
      "sympy/matrices/expressions/matmul.py",
      "sympy/matrices/expressions/matpow.py",
      "sympy/matrices/expressions/matexpr.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/fourier.py] sympy.matrices.expressions.fourier.IDFT\nsympy.matrices.expressions.fourier.IDFT:\n  Returns an inverse discrete Fourier transform matrix. The matrix is scaled\n    with :math:`\\frac{1}{\\sqrt{n}}` so that it is unitary.\n\n    Parameters\n    ==========\n\n    n : integer or Symbol\n        Size of the transform\n\n    Examples\n    ========\n\n    >>> from sympy.matrices.expressions.fourier import DFT, IDFT\n    >>> IDFT(3)\n    IDFT(3)\n    >>> IDFT(4)*DFT(4)\n    I\n\n    See Also\n    ========\n\n    DFT",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.09,
    "strip_verify_summary": "Most claims are verifiable from stripped code; line numbers were removed and one claim about LaTeX printing was removed since that file wasn't provided.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_29",
    "repo": "sympy",
    "question": "What is `matrix_kronecker_product` and how does it relate to the `KroneckerProduct` class and its canonicalization rules?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`matrix_kronecker_product` is a function that computes the explicit (numerical) Kronecker product of a sequence of `MatrixBase` instances, distinct from the symbolic `KroneckerProduct` class. Here is what it is and how it fits into the broader architecture:\n\n**Core Algorithm**: The function works right-to-left through the input matrices (using `reversed(matrices[:-1])`), starting with the last matrix as `matrix_expansion`. For each preceding matrix, it iterates over that matrix's rows and columns, scaling `matrix_expansion` by each scalar element of that matrix and assembling the result using `row_join` (horizontal concatenation) and `col_join` (vertical concatenation). This builds up the Kronecker product incrementally.\n\n**Return type selection**: After computing the product, the function selects the output matrix class by finding the input matrix with the highest `_class_priority` attribute (`max(matrices, key=lambda M: M._class_priority).__class__`). If the result isn't already an instance of the highest-priority class, it gets converted.\n\n**Relationship to `KroneckerProduct` class**: `matrix_kronecker_product` is one of four canonicalization rules in the `rules` tuple: `(unpack, explicit_kronecker_product, flatten, extract_commutative)`. The `explicit_kronecker_product` rule is a wrapper that calls `matrix_kronecker_product` when all arguments to a `KroneckerProduct` are concrete `MatrixBase` instances. The `canonicalize` function applies these rules exhaustively using `exhaust(condition(lambda x: isinstance(x, KroneckerProduct), do_one(*rules)))` from the `sympy.strategies` framework.\n\n**`KroneckerProduct` class**: This is a symbolic `MatrixExpr` subclass that stores its arguments unevaluated. It has an `is_KroneckerProduct = True` class attribute. Its `__new__` method includes a special optimization: if all arguments are identity matrices, it returns `Identity(prod(a.rows for a in args))` \u2014 and if all args are also `MatrixBase` instances, it returns `ret.as_explicit()` (i.e., an explicit identity matrix). Its `_entry(i, j)` method computes individual elements using modular arithmetic (divmod) iterating through args in reverse.\n\n**`kronecker_product` vs `matrix_kronecker_product`**: The public `kronecker_product()` function is a higher-level dispatcher: it raises `TypeError` for empty input, returns the single matrix for length-1 input, and otherwise calls `KroneckerProduct(*matrices).doit()`, which triggers canonicalization that eventually calls `matrix_kronecker_product` for concrete matrices. `matrix_kronecker_product` is lower-level and requires all inputs to be `MatrixBase` instances (raises `TypeError` otherwise).\n\n**Combination rules**: The module also provides `combine_kronecker()` which uses the `sympy.strategies` framework with `bottom_up` traversal to simplify expressions involving `KroneckerProduct` \u2014 it handles `MatAdd` (via `kronecker_mat_add` using `_kronecker_add`), `MatMul` (via `kronecker_mat_mul` using `_kronecker_mul`), and `MatPow` (via `kronecker_mat_pow`, which requires all factors to be square).",
    "rubric": [
      "Explains that `matrix_kronecker_product` computes an explicit/numerical Kronecker product of `MatrixBase` instances, as opposed to the symbolic `KroneckerProduct` class",
      "Describes the right-to-left algorithm: starts with the last matrix, iterates over each preceding matrix's elements, scaling and assembling via `row_join` and `col_join`",
      "Mentions that the return type is determined by selecting the class with the highest `_class_priority` among the input matrices",
      "Describes the canonicalization pipeline: the `rules` tuple contains `unpack`, `explicit_kronecker_product`, `flatten`, and `extract_commutative`, applied via `exhaust(condition(..., do_one(*rules)))`",
      "Explains that `explicit_kronecker_product` is the bridge rule that calls `matrix_kronecker_product` when all args are `MatrixBase` instances",
      "Explains `KroneckerProduct.__new__` identity optimization: returns `Identity(prod(a.rows for a in args))` when all args are identity matrices, and `.as_explicit()` if they're also `MatrixBase`",
      "Describes the public `kronecker_product()` function as a higher-level dispatcher that calls `KroneckerProduct(*matrices).doit()` which triggers canonicalization",
      "Mentions `combine_kronecker` and its handling of `MatAdd`, `MatMul`, and `MatPow` via `bottom_up` traversal using the strategies framework"
    ],
    "key_files": [
      "sympy/matrices/expressions/kronecker.py",
      "sympy/strategies/core.py",
      "sympy/strategies/rl.py",
      "sympy/strategies/tools.py",
      "sympy/matrices/expressions/special.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/kronecker.py] sympy.matrices.expressions.kronecker.matrix_kronecker_product\nCompute the Kronecker product of a sequence of SymPy Matrices.\n\n    This is the standard Kronecker product of matrices [1].\n\n    Parameters\n    ==========\n\n    matrices : tuple of MatrixBase instances\n        The matrices to take the Kronecker product of.\n\n    Returns\n    =======\n\n    matrix : MatrixBase\n        The Kronecker product matrix.\n\n    Examples\n    ========\n\n    >>> from sympy import Matrix\n    >>> from sympy.matrices.expressions.kronecker import (\n    ... matrix_kronecker_product)\n\n    >>> m1 = Matrix([[1,2],[3,4]])\n    >>> m2 = Matrix([[1,0],[0,1]])\n    >>> matrix_kronecker_product(m1, m2)\n    Matrix([\n    [1, 0, 2, 0],\n    [0, 1, 0, 2],\n    [3, 0, 4, 0],\n    [0, 3, 0, 4]])\n    >>> matrix_kronecker_product(m2, m1)\n    Matrix([\n    [1, 2, 0, 0],\n    [3, 4, 0, 0],\n    [0, 0, 1, 2],\n    [0, 0, 3, 4]])\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Kronecker_product",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a docstring for matrix_kronecker_product showing its parameters, return type, and examples. The generated answer includes extensive implementation details (algorithm internals, class priorities with specific numbers, canonicalization rules, KroneckerProduct class details, combine_kronecker function) that are not present in the provided documentation chunk.",
      "The specific class priority numbers (MatrixBase=3, DenseMatrix=4, ImmutableDenseMatrix=8, ImmutableSparseMatrix=9) are not in the documentation and cannot be verified from it.",
      "The claim about the algorithm working 'right-to-left through the input matrices (using reversed(matrices[:-1]))' is not verifiable from the provided documentation.",
      "Details about explicit_kronecker_product, canonicalize, combine_kronecker, kronecker_mat_add, kronecker_mat_mul, kronecker_mat_pow are all beyond the scope of the provided documentation.",
      "The claim about KroneckerProduct's __new__ returning Identity for all-identity inputs is not verifiable from the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: specific _class_priority numbers and the 'mixed-product property' description are not verifiable from stripped code; all other claims are directly verifiable from code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_30",
    "repo": "sympy",
    "question": "Why does PermutationMatrix unify transpose, adjoint, and inverse into the same method, and how do its other methods handle identity results and symbolic indices?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `PermutationMatrix` class assigns `_eval_transpose = _eval_adjoint = _eval_inverse` in permutation.py.\n\nThe `_eval_inverse` method specifically computes the inverse by raising the underlying permutation to the power -1: `PermutationMatrix(self.args[0] ** -1)`. By assigning the same method to `_eval_transpose` and `_eval_adjoint`, the class ensures that transpose and adjoint operations automatically return the same result as inversion.\n\nThe `_eval_power` method calls `.doit()` after constructing the powered PermutationMatrix, which checks if the result is the identity permutation and, if so, returns an `Identity(self.rows)` matrix instead. When a permutation raised to some power yields the identity, the result becomes the symbolic Identity matrix rather than remaining as a PermutationMatrix wrapping an identity permutation.\n\nThe `_entry` method uses `KroneckerDelta(perm.apply(i), j)` to compute individual matrix entries. When `i` is a symbolic (non-integer) index, `perm.apply(i)` returns an `AppliedPermutation` object (defined in sympy/combinatorics/permutations.py).\n\nThe `combine_permutations` function in matmul.py is included in the `rules` tuple used by the `canonicalize` exhaust strategy for `MatMul`. It combines adjacent PermutationMatrix instances in a product by multiplying their underlying permutation cycles (`cycle_1 * cycle_2`), reducing chains of permutation matrix multiplications to a single PermutationMatrix.",
    "rubric": [
      "Explains that `_eval_transpose` and `_eval_adjoint` are assigned to `_eval_inverse`, so transpose and adjoint return the same result as inversion (the permutation raised to -1).",
      "Explains that `_eval_inverse` computes the inverse by raising the underlying permutation to the power -1: `PermutationMatrix(self.args[0] ** -1)`.",
      "Explains that `_eval_power` calls `.doit()` which checks `is_Identity` and returns `Identity(self.rows)` when the powered permutation is the identity permutation.",
      "Explains that `_entry` uses `KroneckerDelta(perm.apply(i), j)` and that for symbolic (non-integer) indices, `perm.apply(i)` returns an `AppliedPermutation` object.",
      "Explains that `combine_permutations` in the MatMul canonicalization rules merges adjacent PermutationMatrix instances by multiplying their underlying permutations (`cycle_1 * cycle_2`) into a single PermutationMatrix."
    ],
    "key_files": [
      "sympy/matrices/expressions/permutation.py",
      "sympy/matrices/expressions/matmul.py",
      "sympy/combinatorics/permutations.py",
      "sympy/matrices/expressions/matexpr.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/permutation.py] sympy.matrices.expressions.permutation.PermutationMatrix\nsympy.matrices.expressions.permutation.PermutationMatrix:\n  A Permutation Matrix\n\n    Parameters\n    ==========\n\n    perm : Permutation\n        The permutation the matrix uses.\n\n        The size of the permutation determines the matrix size.\n\n        See the documentation of\n        :class:`sympy.combinatorics.permutations.Permutation` for\n        the further information of how to create a permutation object.\n\n    Examples\n    ========\n\n    >>> from sympy import Matrix, PermutationMatrix\n    >>> from sympy.combinatorics import Permutation\n\n    Creating a permutation matrix:\n\n    >>> p = Permutation(1, 2, 0)\n    >>> P = PermutationMatrix(p)\n    >>> P = P.as_explicit()\n    >>> P\n    Matrix([\n    [0, 1, 0],\n    [0, 0, 1],\n    [1, 0, 0]])\n\n    Permuting a matrix row and column:\n\n    >>> M = Matrix([0, 1, 2])\n    >>> Matrix(P*M)\n    Matrix([\n    [1],\n    [2],\n    [0]])\n\n    >>> Matrix(M.T*P)\n    Matrix([[2, 0, 1]])\n\n    See Also\n    ========\n\n    sympy.combinatorics.permutations.Permutation",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided only covers the basic class description, parameters, and usage examples of PermutationMatrix. The generated answer makes extensive claims about internal implementation details (line numbers, specific method implementations like _eval_transpose, _eval_adjoint, _eval_inverse, _eval_power, _entry, combine_permutations, etc.) that are not present in the provided documentation chunk.",
      "While the claims about permutation matrices being orthogonal (P^T = P^{-1}) are mathematically correct and plausible for a SymPy implementation, they cannot be verified from the provided documentation alone.",
      "Specific line number references (line 94 of permutation.py, line 3088 of permutations.py, line 386 of matmul.py) cannot be verified from the provided documentation.",
      "The claim about AppliedPermutation, combine_permutations function, and canonicalize exhaust strategy are implementation details not covered in the documentation."
    ],
    "strip_verify_leakage": 0.3,
    "strip_verify_summary": "Several claims about mathematical rationale (orthogonal matrix, conjugation trivial) and line numbers are doc-leakage; core code behavior claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_33",
    "repo": "sympy",
    "question": "What are the key architectural components and dependencies of the `parse_latex` function and its two backends (ANTLR and Lark)?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `parse_latex` function in `sympy/parsing/latex/__init__.py` serves as a unified entry point that dispatches LaTeX string parsing to one of two backends (ANTLR or Lark). Here are the key dependency and architectural details discovered from the source code:\n\n**Pre-parsing validation:** Before dispatching to any backend, `parse_latex` always calls two validation functions in sequence: `check_cases_env(s)` which raises `LaTeXParsingError` if the input contains `\\begin{cases}`, and `check_matrix_delimiters(s)` which validates matched matrix delimiter pairs using regex-based detection. The `MATRIX_DELIMS` dictionary contains 15 entries mapping begin-delimiter regex patterns to end-delimiter regex patterns, covering `pmatrix`, `bmatrix`, `vmatrix`, `\\begin{matrix}` with various parentheses/brackets/pipes, and `\\begin{array}` variants.\n\n**ANTLR backend (`_parse_latex_antlr.py`):**\n- Requires `antlr4-python3-runtime` version starting with `4.11` specifically \u2014 checked via `importlib.metadata.version('antlr4-python3-runtime').startswith('4.11')`\n- The `MathErrorListener` class extends `antlr4.error.ErrorListener.ErrorListener` and provides custom syntax error messages. It categorizes errors into \"missing\", \"no viable\", and \"mismatched\" cases.\n- The ANTLR `parse_latex` function has a parameter `sympy` that shadows the module name. It calls `parser.math().relation()` to get the parse tree, then `convert_relation()` to convert it. In strict mode, it checks `relation.start.start != 0 or relation.stop.stop != len(sympy) - 1`.\n- Uses `sympy.printing.str.StrPrinter` to format subscripts in `convert_atom` (e.g., `sname += '_{' + StrPrinter().doprint(subscript) + '}'`)\n- Imports `Bra` and `Ket` from `sympy.physics.quantum.state` for Dirac notation support\n- Handles `\\lg` with base 10 and both `\\ln` and `\\log` default to base `sympy.E`\n\n**Lark backend (`sympy/parsing/latex/lark/`):**\n- `LarkLaTeXParser` uses Lark's `\"earley\"` parser with `ambiguity=\"explicit\"`, start rule `\"latex_string\"`, and `lexer=\"auto\"`\n- A module-level singleton `_lark_latex_parser` is created at import time (if lark is installed), so `parse_latex_lark` reuses this single instance\n- The `TransformToSymPyExpr` class extends `lark.Transformer` and has a fallback: if the `lark` module is not installed, a stub `Transformer` class with a no-op `transform` method is defined\n- The Lark grammar file (`latex.lark`) defines `CMD_FRAC` to match `\\frac`, `\\dfrac`, `\\tfrac`, and `\\nicefrac`; `CMD_BINOM` matches `\\binom`, `\\dbinom`, `\\tbinom`\n- The Lark grammar ignores LaTeX spacing commands like `\\,`, `\\:`, `\\;`, `\\quad`, `\\qquad`, as well as `\\left`, `\\right`, `\\limits`, `\\nolimits`, and `\\displaystyle`\n- Greek symbols are defined in a separate imported grammar file `greek_symbols.lark` with 20 Greek letters (alpha through omega); notably, `pi` and `sigma` are NOT included in the grammar\n- The Lark `TransformToSymPyExpr.adjacent_expressions` method handles implicit multiplication, but has special cases: a `Ket` followed by `Bra` becomes an `OuterProduct`, and a leading `d` symbol is treated as a differential operator\n- The `SYMBOL` token in the Lark grammar matches `/[a-zA-Z]'*/`, meaning only single Latin characters (optionally with primes) are matched as bare symbols\n- The Lark transformer's `superscript` method handles matrix transposition (`^T`), adjoint (`^H`), primes, and star notation for matrices\n\n**Error handling:** `LaTeXParsingError` is a simple subclass of `Exception` with no additional methods (defined in `errors.py`).\n\n**Decorator:** The `parse_latex` function is decorated with `@doctest_depends_on(modules=('antlr4', 'lark'))`.",
    "rubric": [
      "Mentions that `parse_latex` dispatches to either an ANTLR or Lark backend based on the `backend` parameter",
      "Mentions that `check_cases_env` and `check_matrix_delimiters` are called as pre-parsing validation before dispatching",
      "Mentions that `check_cases_env` raises LaTeXParsingError if \\begin{cases} is found in the input",
      "Mentions that the MATRIX_DELIMS dictionary has 15 entries mapping begin-delimiter regex patterns to end-delimiter regex patterns",
      "Mentions that the ANTLR backend requires antlr4-python3-runtime version starting with 4.11",
      "Mentions that `MathErrorListener` extends `ErrorListener` and categorizes syntax errors into missing, no viable, and mismatched cases",
      "Mentions that the ANTLR `parse_latex` function's `sympy` parameter shadows the module name, and strict mode checks start/stop positions",
      "Mentions that the Lark backend uses Lark with earley parser, ambiguity='explicit', and start rule 'latex_string'",
      "Mentions that a module-level singleton `_lark_latex_parser` is created at import time and reused by `parse_latex_lark`",
      "Mentions that `TransformToSymPyExpr` extends `lark.Transformer` with a stub fallback if lark is not installed",
      "Mentions that CMD_FRAC matches \\frac, \\dfrac, \\tfrac, and \\nicefrac in the Lark grammar",
      "Mentions that the Lark grammar ignores spacing commands (\\,, \\:, \\;, \\quad, \\qquad) and \\left, \\right, \\limits, \\displaystyle",
      "Mentions that the Greek symbols grammar defines 20 letters (alpha through omega) but notably excludes pi and sigma",
      "Mentions that adjacent_expressions handles Ket followed by Bra as OuterProduct and leading 'd' as differential",
      "Mentions that the ANTLR backend handles \\lg as base 10 and \\ln/\\log default to base E (sympy.E)"
    ],
    "key_files": [
      "sympy/parsing/latex/__init__.py",
      "sympy/parsing/latex/_parse_latex_antlr.py",
      "sympy/parsing/latex/lark/latex_parser.py",
      "sympy/parsing/latex/lark/transformer.py",
      "sympy/parsing/latex/lark/grammar/latex.lark",
      "sympy/parsing/latex/lark/grammar/greek_symbols.lark",
      "sympy/parsing/latex/errors.py"
    ],
    "source_doc": "[docstring: sympy/parsing/latex/__init__.py] sympy.parsing.latex.parse_latex\nConverts the input LaTeX string ``s`` to a SymPy ``Expr``.\n\n    Parameters\n    ==========\n\n    s : str\n        The LaTeX string to parse. In Python source containing LaTeX,\n        *raw strings* (denoted with ``r\"``, like this one) are preferred,\n        as LaTeX makes liberal use of the ``\\`` character, which would\n        trigger escaping in normal Python strings.\n    backend : str, optional\n        Currently, there are two backends supported: ANTLR, and Lark.\n        The default setting is to use the ANTLR backend, which can be\n        changed to Lark if preferred.\n\n        Use ``backend=\"antlr\"`` for the ANTLR-based parser, and\n        ``backend=\"lark\"`` for the Lark-based parser.\n\n        The ``backend`` option is case-sensitive, and must be in\n        all lowercase.\n    strict : bool, optional\n        This option is only available with the ANTLR backend.\n\n        If True, raise an exception if the string cannot be parsed as\n        valid LaTeX. If False, try to recover gracefully from common\n        mistakes.\n\n    Examples\n    ========\n\n    >>> from sympy.parsing.latex import parse_latex\n    >>> expr = parse_latex(r\"\\frac {1 + \\sqrt {\\a}} {\\b}\")\n    >>> expr\n    (sqrt(a) + 1)/b\n    >>> expr.evalf(4, subs=dict(a=5, b=2))\n    1.618\n    >>> func = parse_latex(r\"\\int_1^\\alpha \\dfrac{\\mathrm{d}t}{t}\", backend=\"lark\")\n    >>> func.evalf(subs={\"alpha\": 2})\n    0.693147180559945",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from attribution comment (latex2sympy), ISO standard reference, TODO comments, and line count claim; most claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_31",
    "repo": "sympy",
    "question": "Where is `MatrixPermute` defined and how does its data flow work from construction through entry computation to explicit matrix generation?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `MatrixPermute` class is defined in `sympy/matrices/expressions/permutation.py` and inherits from `MatrixExpr` (defined in `sympy/matrices/expressions/matexpr.py`). It stores three args: `mat`, `perm`, and `axis`. The key data flow works as follows:\n\n1. **Construction (`__new__`)**: Located in `sympy/matrices/expressions/permutation.py`. It validates the matrix (must have `is_Matrix` True), accepts either a `Permutation` or `PermutationMatrix` (extracting the permutation from `args[0]` of the latter), validates the axis (must be 0 or 1), and calls `perm.resize(mat_size)` if the permutation size doesn't match the matrix dimension along the given axis.\n\n2. **Entry computation (`_entry`)**: For axis=0 (rows), it returns `mat[perm.apply(i), j]`. For axis=1 (columns), it returns `mat[i, perm.apply(j)]`. The `perm.apply()` method (in `sympy/combinatorics/permutations.py`) returns `Integer(self._array_form[i])` for integer indices, or an `AppliedPermutation` symbolic expression for symbolic indices.\n\n3. **`as_explicit()` call chain**: `MatrixPermute` inherits `as_explicit()` from `MatrixExpr`, which constructs an `ImmutableDenseMatrix` by calling `self[i, j]` for each entry. The `__getitem__` method dispatches to `self._entry(i, j)`, completing the chain.\n\n4. **`doit()` simplifications**: Returns the original matrix if perm is identity; returns `PermutationMatrix(perm)` if mat is Identity and axis=0, or `PermutationMatrix(perm**-1)` if axis=1; returns the matrix unchanged for `ZeroMatrix` or `OneMatrix`; and composes nested `MatrixPermute` with the same axis by multiplying the permutations (`perm * mat.args[1]`).\n\n5. **Rewrite to MatMul** (`_eval_rewrite_as_MatMul`): For axis=0, returns `MatMul(PermutationMatrix(perm), mat)`. For axis=1, returns `MatMul(mat, PermutationMatrix(perm**-1))` \u2014 note the inverse for column permutation.\n\n6. **Permutation resize** (in `sympy/combinatorics/permutations.py`): Called via `perm.resize(mat_size)` when the permutation size doesn't match the matrix dimension.",
    "rubric": [
      "Identifies that MatrixPermute is defined in sympy/matrices/expressions/permutation.py and inherits from MatrixExpr",
      "Explains that __new__ validates the matrix (is_Matrix), accepts Permutation or PermutationMatrix, validates axis (0 or 1), and resizes the permutation if needed",
      "Describes _entry behavior: axis=0 returns mat[perm.apply(i), j], axis=1 returns mat[i, perm.apply(j)]",
      "Explains that as_explicit() is inherited from MatrixExpr and constructs an ImmutableDenseMatrix by calling self[i, j] which dispatches to _entry",
      "Describes doit() simplifications: identity perm returns mat, Identity mat returns PermutationMatrix (with inverse for axis=1), ZeroMatrix/OneMatrix unchanged, nested MatrixPermute composes permutations",
      "Explains _eval_rewrite_as_MatMul: axis=0 gives MatMul(PermutationMatrix(perm), mat), axis=1 gives MatMul(mat, PermutationMatrix(perm**-1))"
    ],
    "key_files": [
      "sympy/matrices/expressions/permutation.py",
      "sympy/matrices/expressions/matexpr.py",
      "sympy/combinatorics/permutations.py"
    ],
    "source_doc": "[docstring: sympy/matrices/expressions/permutation.py] sympy.matrices.expressions.permutation.MatrixPermute\nsympy.matrices.expressions.permutation.MatrixPermute:\n  Symbolic representation for permuting matrix rows or columns.\n\n    Parameters\n    ==========\n\n    perm : Permutation, PermutationMatrix\n        The permutation to use for permuting the matrix.\n        The permutation can be resized to the suitable one,\n\n    axis : 0 or 1\n        The axis to permute alongside.\n        If `0`, it will permute the matrix rows.\n        If `1`, it will permute the matrix columns.\n\n    Notes\n    =====\n\n    This follows the same notation used in\n    :meth:`sympy.matrices.matrixbase.MatrixBase.permute`.\n\n    Examples\n    ========\n\n    >>> from sympy import Matrix, MatrixPermute\n    >>> from sympy.combinatorics import Permutation\n\n    Permuting the matrix rows:\n\n    >>> p = Permutation(1, 2, 0)\n    >>> A = Matrix([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n    >>> B = MatrixPermute(A, p, axis=0)\n    >>> B.as_explicit()\n    Matrix([\n    [4, 5, 6],\n    [7, 8, 9],\n    [1, 2, 3]])\n\n    Permuting the matrix columns:\n\n    >>> B = MatrixPermute(A, p, axis=1)\n    >>> B.as_explicit()\n    Matrix([\n    [2, 3, 1],\n    [5, 6, 4],\n    [8, 9, 7]])\n\n    See Also\n    ========\n\n    sympy.matrices.matrixbase.MatrixBase.permute",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only line numbers and some details about the truncated `resize` method needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_32",
    "repo": "sympy",
    "question": "How does `parse_autolev` translate Autolev code into SymPy code, from input handling through lexing, tree walking, and code generation?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `parse_autolev` function implements Autolev-to-SymPy code translation using a multi-stage pipeline based on ANTLR4 parser infrastructure:\n\n**Overall Algorithm:**\n\n1. **Entry point** (`sympy/parsing/autolev/__init__.py`): The public `parse_autolev(autolev_code, include_numeric=False)` function delegates to an internal module `sympy.parsing.autolev._parse_autolev_antlr` after importing it via `import_module`.\n\n2. **Input handling** (`_parse_autolev_antlr.py`): If `autolev_code` is not a string, it calls `\"\".join(autolev_code.readlines())` to convert it. It also checks that the `antlr4-python3-runtime` package is version `4.11` (using `importlib.metadata.version`), raising an `ImportError` if not.\n\n3. **Lexing and Parsing**: The code creates an `antlr4.InputStream`, feeds it to `AutolevLexer` to produce tokens, wraps them in a `CommonTokenStream`, passes to `AutolevParser`, and calls `parser.prog()` to produce the parse tree.\n\n4. **Tree walking with listener pattern**: An `antlr4.ParseTreeWalker` walks the tree with a `MyListener` instance (from `_listener_autolev_antlr.py`). The listener has enter/exit methods corresponding to parse tree nodes.\n\n5. **MyListener class**: This is the core code generation engine. It inherits from `AutolevListener` (generated ANTLR listener). Key implementation details:\n   - It uses a `tree_property` dict for tree annotation\n   - It maintains `symbol_table` and `symbol_table2` (OrderedDicts) mapping Autolev names to SymPy names\n   - It classifies expressions into `matrix_expr`, `numeric_expr`, `vector_expr`, and `fr_expr` lists\n   - Vector multiplication of two vectors produces `_me.outer(...)` (dyadic product)\n   - The `^` operator in Autolev is translated to `**` for Python exponentiation\n   - The `output_code` list accumulates generated code strings, always starting with 4 import statements: `sympy.physics.mechanics as _me`, `sympy as _sm`, `math as m`, `numpy as _np`\n\n6. **Variable handling**: The `strfunc(z)` helper generates derivative suffixes: 0\u2192\"\", 1\u2192\"_d\", 2\u2192\"_dd\", etc. Constants with `+` sign get `nonnegative=True`, with `-` get `nonpositive=True`. The `settings[\"complex\"]` flag controls whether `real=True` is added to `_sm.symbols()` calls (default is `real=True`).\n\n7. **Kane's method assembly**: When `FR()` or `FRSTAR()` is encountered, the listener assembles kinematic differential equations, constructs force lists, and generates `_me.KanesMethod(...)` code. If no kd_eqs are explicitly supplied, it creates auxiliary `u_` variables for motion variables automatically.\n\n8. **Numeric code generation** (`exitCodegen`): When `CODE DYNAMICS()` is encountered and `include_numeric=True`, it generates PyDy `System` integration code using `_np.linspace(0.0, tfinal, tfinal/integstp)` for time discretization. It reads `tfinal` and `integstp` from `self.inputs[\"tfinal\"]` and `self.inputs[\"integstp\"]`.\n\n9. **Grammar** (`Autolev.g4`): The grammar is case-insensitive for keywords (using character alternatives like `('M'|'m')('A'|'a')...`). Comments use `%` prefix and are skipped. The `&` character is included in the whitespace rule and skipped. Vectors are identified by trailing `>` characters.\n\nThe final output is produced by `\"\".join(my_listener.output_code)` - joining all accumulated code strings.",
    "rubric": [
      "Explains that the public parse_autolev function delegates to an internal _parse_autolev_antlr module via import_module",
      "Describes input handling: non-string input is converted via readlines(), and antlr4 version 4.11 is validated",
      "Explains the ANTLR4 pipeline: InputStream \u2192 AutolevLexer \u2192 CommonTokenStream \u2192 AutolevParser \u2192 parser.prog() parse tree",
      "Describes the ParseTreeWalker walking the tree with a MyListener instance that has enter/exit methods",
      "Explains that MyListener inherits from AutolevListener and uses tree_property dict for tree annotation and symbol_table/symbol_table2 OrderedDicts for name mapping",
      "Mentions expression classification into matrix_expr, numeric_expr, vector_expr, and fr_expr lists",
      "Explains that ^ is translated to ** for Python exponentiation and vector*vector multiplication produces _me.outer()",
      "Describes the output_code list starting with 4 import statements (sympy.physics.mechanics, sympy, math, numpy)",
      "Explains Kane's method assembly: FR()/FRSTAR() triggers generation of kd_eqs, force lists, and KanesMethod code, with automatic u_ variables if kd_eqs not supplied",
      "Describes numeric codegen: CODE DYNAMICS() with include_numeric=True generates PyDy System integration code using _np.linspace",
      "Mentions grammar features: case-insensitive keywords, % for comments, & as whitespace, vectors identified by trailing >",
      "States final output is produced by joining my_listener.output_code"
    ],
    "key_files": [
      "sympy/parsing/autolev/__init__.py",
      "sympy/parsing/autolev/_parse_autolev_antlr.py",
      "sympy/parsing/autolev/_listener_autolev_antlr.py",
      "sympy/parsing/autolev/Autolev.g4"
    ],
    "source_doc": "[docstring: sympy/parsing/autolev/__init__.py] sympy.parsing.autolev.parse_autolev\nParses Autolev code (version 4.1) to SymPy code.\n\n    Parameters\n    =========\n    autolev_code : Can be an str or any object with a readlines() method (such as a file handle or StringIO).\n    include_numeric : boolean, optional\n        If True NumPy, PyDy, or other numeric code is included for numeric evaluation lines in the Autolev code.\n\n    Returns\n    =======\n    sympy_code : str\n        Equivalent SymPy and/or numpy/pydy code as the input code.\n\n\n    Example (Double Pendulum)\n    =========================\n    >>> my_al_text = (\"MOTIONVARIABLES' Q{2}', U{2}'\",\n    ... \"CONSTANTS L,M,G\",\n    ... \"NEWTONIAN N\",\n    ... \"FRAMES A,B\",\n    ... \"SIMPROT(N, A, 3, Q1)\",\n    ... \"SIMPROT(N, B, 3, Q2)\",\n    ... \"W_A_N>=U1*N3>\",\n    ... \"W_B_N>=U2*N3>\",\n    ... \"POINT O\",\n    ... \"PARTICLES P,R\",\n    ... \"P_O_P> = L*A1>\",\n    ... \"P_P_R> = L*B1>\",\n    ... \"V_O_N> = 0>\",\n    ... \"V2PTS(N, A, O, P)\",\n    ... \"V2PTS(N, B, P, R)\",\n    ... \"MASS P=M, R=M\",\n    ... \"Q1' = U1\",\n    ... \"Q2' = U2\",\n    ... \"GRAVITY(G*N1>)\",\n    ... \"ZERO = FR() + FRSTAR()\",\n    ... \"KANE()\",\n    ... \"INPUT M=1,G=9.81,L=1\",\n    ... \"INPUT Q1=.1,Q2=.2,U1=0,U2=0\",\n    ... \"INPUT TFINAL=10, INTEGSTP=.01\",\n    ... \"CODE DYNAMICS() some_filename.c\")\n    >>> my_al_text = '\\n'.join(my_al_text)\n    >>> from sympy.parsing.autolev import parse_autolev\n    >>> print(parse_autolev(my_al_text, include_numeric=True))\n    import sympy.physics.mechanics as _me\n    import sympy as _sm\n    import math as m\n    import numpy as _np\n    <BLANKLINE>\n    q1, q2, u1, u2 = _me.dynamicsymbols('q1 q2 u1 u2')\n    q1_d, q2_d, u1_d, u2_d = _me.dynamicsymbols('q1_ q2_ u1_ u2_', 1)\n    l, m, g = _sm.symbols('l m g', real=True)\n    frame_n = _me.ReferenceFrame('n')\n    frame_a = _me.ReferenceFrame('a')\n    frame_b = _me.ReferenceFrame('b')\n    frame_a.orient(frame_n, 'Axis', [q1, frame_n.z])\n    frame_b.orient(frame_n, 'Axis', [q2, frame_n.z])\n    frame_a.set_ang_vel(frame_n, u1*frame_n.z)\n    frame_b.set_ang_vel(frame_n, u2*frame_n.z)\n    point_o = _me.Point('o')\n    particle_p = _me.Particle('p', _me.Point('p_pt'), _sm.Symbol('m'))\n    particle_r = _me.Particle('r', _me.Point('r_pt'), _sm.Symbol('m'))\n    particle_p.point.set_pos(point_o, l*frame_a.x)\n    particle_r.point.set_pos(particle_p.point, l*frame_b.x)\n    point_o.set_vel(frame_n, 0)\n    particle_p.point.v2pt_theory(point_o,frame_n,frame_a)\n    particle_r.point.v2pt_theory(particle_p.point,frame_n,frame_b)\n    particle_p.mass = m\n    particle_r.mass = m\n    force_p = particle_p.mass*(g*frame_n.x)\n    force_r = particle_r.mass*(g*frame_n.x)\n    kd_eqs = [q1_d - u1, q2_d - u2]\n    forceList = [(particle_p.point,particle_p.mass*(g*frame_n.x)), (particle_r.point,particle_r.mass*(g*frame_n.x))]\n    kane = _me.KanesMethod(frame_n, q_ind=[q1,q2], u_ind=[u1, u2], kd_eqs = kd_eqs)\n    fr, frstar = kane.kanes_equations([particle_p, particle_r], forceList)\n    zero = fr+frstar\n    from pydy.system import System\n    sys = System(kane, constants = {l:1, m:1, g:9.81},\n    specifieds={},\n    initial_conditions={q1:.1, q2:.2, u1:0, u2:0},\n    times = _np.linspace(0.0, 10, 10/.01))\n    <BLANKLINE>\n    y=sys.integrate()\n    <BLANKLINE>",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims about internal implementation details (ANTLR4 pipeline, MyListener class, grammar details, etc.) go beyond what the documentation covers, but are plausible code-level details that cannot be contradicted by the provided documentation.",
      "The documentation shows `q1_d, q2_d, u1_d, u2_d = _me.dynamicsymbols('q1_ q2_ u1_ u2_', 1)` which is consistent with the answer's claim about `strfunc` generating '_d' suffixes for first derivatives.",
      "The documentation shows `real=True` in `_sm.symbols('l m g', real=True)` which is consistent with the answer's claim about default `real=True`.",
      "The documentation shows `_np.linspace(0.0, 10, 10/.01)` which is consistent with the answer's claim about `_np.linspace(0.0, tfinal, tfinal/integstp)`.",
      "The documentation shows the 4 import statements which matches the answer's claim about starting with 4 imports.",
      "The answer's claim about antlr4 version `4.11` is not verifiable from the documentation but is a plausible implementation detail."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for documentation-style explanations of purpose (line continuation, tree annotation description).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_36",
    "repo": "sympy",
    "question": "How does the Arch class model a three-hinged parabolic arch, from constructing the shape equation through solving for reactions, shear, and axial forces?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Arch` class in `sympy/physics/continuum_mechanics/arch.py` implements a three-hinged parabolic arch structural analysis system. Here is how it works in detail:\n\n**Shape Equation Construction (`get_shape_eqn`):**\nThe arch shape is modeled as a downward-opening parabola `a*(x-x0)^2 + y0` where `x0` is `crown_x` and `y0` is `crown_y`. The coefficient `a` is declared with `Symbol('a', positive=False)`. When both `crown_x` and `crown_y` are provided, `a` is solved from the left support position, then validated against the right support. When only `crown_x` is given, both `a` and a constant `c` are solved simultaneously using both support positions, and `crown_y` is set to the solved value of `c`.\n\n**Load Tracking Architecture:**\nInternally, loads are tracked through multiple parallel data structures:\n- `_conc_loads` and `_distributed_loads` dictionaries store load parameters\n- `_moment_x`, `_moment_y`, `_load_x`, `_load_y` dictionaries map x-position discontinuity points to symbolic expressions\n- `_points_disc_x` and `_points_disc_y` are sets tracking discontinuity points\n- For distributed loads, `sympy.Min(x, end)` is used to create piecewise moment expressions: the moment formula is `-mag*(Min(x,end)-start)*(x0-(start+Min(x,end))/2)`\n- For concentrated loads, the y-height is automatically computed from the shape equation (`self._shape_eqn.subs({'x':start})`), and force components are decomposed using `mag*cos(rad(angle))` and `mag*sin(rad(angle))` where `rad` converts degrees to radians\n\n**Solve Method - Equation Assembly:**\nThe `solve()` method builds `Piecewise` functions by iterating over sorted discontinuity points and accumulating contributions. It computes three key moments:\n1. `moment_A` - moment about the left support evaluated at the right support\n2. `moment_hinge_left` - moment of forces to the left of crown, about the crown\n3. `moment_hinge_right` - moment of forces to the right of crown, about the crown\n\nThe method handles 4 major support configurations with sub-cases based on member height relative to support heights: both-hinge (4 equations, 4 unknowns: R_A_x, R_A_y, R_B_x, R_B_y), both-roller, left-roller-only, and right-roller-only (each with 5 equations including tension T). If a roller support exists but no member has been added, the method prints a warning and returns without solving.\n\n**Force Computation:**\nAfter solving reactions, the bending moment uses `x0` as the variable (not `x`), incorporating both the moment from loads and reaction times height difference terms. Shear and axial forces are computed by rotating the net force components (fx = load_x + R_A_x, fy = load_y + R_A_y) by the local arch tangent angle: `angle = atan(diff(shape_eqn, x))`, then `axial = fx*cos(angle) + fy*sin(angle)` and `shear = -fx*sin(angle) + fy*cos(angle)`.\n\n**Member (Tie Rod) Calculation (`add_member`):**\nThe member x-coordinates are computed by finding where the parabola intersects the given height y. It uses the derivative at `crown_x + 1` divided by 2 to get the parabola coefficient `a`, then computes `x_diff = sqrt((y - crown_y)/a)` and positions at `crown_x \u00b1 x_diff`.\n\n**Drawing (`draw`):**\nThe `draw` method requires numpy and uses `sympy.plotting.plot`. The `_draw_loads` method uses `numpy.arange` to create evenly spaced arrow points for distributed loads.\n\n**Key Design Differences from Beam:**\nThe `Arch` class uses `Piecewise` functions and `Min` to handle distributed loads across their range. The `Arch` class does not inherit from `Beam` \u2014 it is a standalone class. Both are exported from the `sympy.physics.continuum_mechanics` package's `__init__.py`.",
    "rubric": [
      "Explains that get_shape_eqn builds a downward-opening parabola a*(x-x0)^2 + y0 using Symbol('a', positive=False), solving 'a' from the left support when both crown_x and crown_y are given, and solving both 'a' and 'c' from both supports when only crown_x is given",
      "Describes the parallel internal data structures: _conc_loads/_distributed_loads dicts, _moment_x/_moment_y/_load_x/_load_y dicts keyed by x-position, and _points_disc_x/_points_disc_y sets of discontinuity points",
      "Explains that distributed loads use sympy.Min(x, end) to create piecewise moment expressions like -mag*(Min(x,end)-start)*(x0-(start+Min(x,end))/2)",
      "Explains that concentrated loads compute y-height from the shape equation and decompose forces using mag*cos(rad(angle)) and mag*sin(rad(angle))",
      "Describes that solve() builds Piecewise functions by iterating over sorted discontinuity points with accumulated contributions, then computes moment_A (about left support at right support), moment_hinge_left, and moment_hinge_right",
      "Identifies the four support configuration cases: both-hinge (4 equations, 4 unknowns), both-roller, left-roller-only, and right-roller-only (each with 5 equations including tension T), with sub-cases based on member height relative to support heights",
      "Explains that shear and axial forces are computed by rotating net force components by the local arch tangent angle: angle = atan(diff(shape_eqn, x)), axial = fx*cos(angle) + fy*sin(angle), shear = -fx*sin(angle) + fy*cos(angle)",
      "Notes that the bending moment formula uses x0 (not x) as the variable, incorporating both load moments and reaction terms with the shape equation height difference",
      "Mentions that add_member computes tie rod x-coordinates by finding where the parabola intersects a given height y, using the derivative at crown_x+1 divided by 2 as the coefficient"
    ],
    "key_files": [
      "sympy/physics/continuum_mechanics/arch.py",
      "sympy/physics/continuum_mechanics/tests/test_arch.py",
      "sympy/physics/continuum_mechanics/__init__.py"
    ],
    "source_doc": "[docstring: sympy/physics/continuum_mechanics/arch.py] sympy.physics.continuum_mechanics.arch.Arch\nsympy.physics.continuum_mechanics.arch.Arch:\n  This class is used to solve problems related to a three hinged arch(determinate) structure.\n\n    An arch is a curved vertical structure spanning an open space underneath it.\n\n    Arches can be used to reduce the bending moments in long-span structures.\n\n\n    Arches are used in structural engineering(over windows, door and even bridges)\n\n    because they can support a very large mass placed on top of them.\n\n    Example\n    ========\n    >>> from sympy.physics.continuum_mechanics.arch import Arch\n    >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n    >>> a.get_shape_eqn\n    5 - (x - 5)**2/5\n\n    >>> from sympy.physics.continuum_mechanics.arch import Arch\n    >>> a = Arch((0,0),(10,1),crown_x=6)\n    >>> a.get_shape_eqn\n    9/5 - (x - 6)**2/20\n\nsympy.physics.continuum_mechanics.arch.Arch.get_shape_eqn:\n  returns the equation of the shape of arch developed\n\nsympy.physics.continuum_mechanics.arch.Arch.get_loads:\n  return the position of the applied load and angle (for concentrated loads)\n\nsympy.physics.continuum_mechanics.arch.Arch.supports:\n  Returns the type of support\n\nsympy.physics.continuum_mechanics.arch.Arch.left_support:\n  Returns the position of the left support.\n\nsympy.physics.continuum_mechanics.arch.Arch.right_support:\n  Returns the position of the right support.\n\nsympy.physics.continuum_mechanics.arch.Arch.reaction_force:\n  return the reaction forces generated\n\nsympy.physics.continuum_mechanics.arch.Arch.apply_load:\n  This method adds load to the Arch.\n\n        Parameters\n        ==========\n\n            order : Integer\n                Order of the applied load.\n\n                    - For point/concentrated loads, order = -1\n                    - For distributed load, order = 0\n\n            label : String or Symbol\n                The label of the load\n                - should not use 'A' or 'B' as it is used for supports.\n\n            start : Float\n\n                    - For concentrated/point loads, start is the x coordinate\n                    - For distributed loads, start is the starting position of distributed load\n\n            mag : Sympifyable\n                Magnitude of the applied load. Must be positive\n\n            end : Float\n                Required for distributed loads\n\n                    - For concentrated/point load , end is None(may not be given)\n                    - For distributed loads, end is the end position of distributed load\n\n            angle: Sympifyable\n                The angle in degrees, the load vector makes with the horizontal\n                in the counter-clockwise direction.\n\n        Examples\n        ========\n        For applying distributed load\n\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n        >>> a.apply_load(0,'C',start=3,end=5,mag=-10)\n\n        For applying point/concentrated_loads\n\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n        >>> a.apply_load(-1,'C',start=2,mag=15,angle=45)\n\nsympy.physics.continuum_mechanics.arch.Arch.remove_load:\n  This methods removes the load applied to the arch\n\n        Parameters\n        ==========\n\n        label : String or Symbol\n            The label of the applied load\n\n        Examples\n        ========\n\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n        >>> a.apply_load(0,'C',start=3,end=5,mag=-10)\n        >>> a.remove_load('C')\n        removed load C: {'start': 3, 'end': 5, 'f_y': -10}\n\nsympy.physics.continuum_mechanics.arch.Arch.change_support_position:\n  Change position of supports.\n        If not provided , defaults to the old value.\n        Parameters\n        ==========\n\n            left_support: tuple (x, y)\n                x: float\n                    x-coordinate value of the left_support\n\n                y: float\n                    y-coordinate value of the left_support\n\n            right_support: tuple (x, y)\n                x: float\n                    x-coordinate value of the right_support\n\n                y: float\n                    y-coordinate value of the right_support\n\nsympy.physics.continuum_mechanics.arch.Arch.change_crown_position:\n  Change the position of the crown/hinge of the arch\n\n        Parameters\n        ==========\n\n            crown_x: Float\n                The x coordinate of the position of the hinge\n                - if not provided, defaults to old value\n\n            crown_y: Float\n                The y coordinate of the position of the hinge\n                - if not provided defaults to None\n\nsympy.physics.continuum_mechanics.arch.Arch.change_support_type:\n  Add the type for support at each end.\n        Can use roller or hinge support at each end.\n\n        Parameters\n        ==========\n\n            left_support, right_support : string\n                Type of support at respective end\n\n                    - For roller support , left_support/right_support = \"roller\"\n                    - For hinged support, left_support/right_support = \"hinge\"\n                    - defaults to hinge if value not provided\n\n        Examples\n        ========\n\n        For applying roller support at right end\n\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n        >>> a.change_support_type(right_support=\"roller\")\n\nsympy.physics.continuum_mechanics.arch.Arch.add_member:\n  This method adds a member/rod at a particular height y.\n        A rod is used for stability of the structure in case of a roller support.\n\nsympy.physics.continuum_mechanics.arch.Arch.shear_force_at:\n  return the shear at some x-coordinates\n        if no x value provided, returns the formula\n\nsympy.physics.continuum_mechanics.arch.Arch.bending_moment_at:\n  return the bending moment at some x-coordinates\n        if no x value provided, returns the formula\n\nsympy.physics.continuum_mechanics.arch.Arch.axial_force_at:\n  return the axial/normal force generated at some x-coordinate\n        if no x value provided, returns the formula\n\nsympy.physics.continuum_mechanics.arch.Arch.solve:\n  This method solves for the reaction forces generated at the supports,\n\n        and bending moment and generated in the arch and tension produced in the member if used.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(10,0),crown_x=5,crown_y=5)\n        >>> a.apply_load(0,'C',start=3,end=5,mag=-10)\n        >>> a.solve()\n        >>> a.reaction_force\n        {R_A_x: 8, R_A_y: 12, R_B_x: -8, R_B_y: 8}\n\n        >>> from sympy import Symbol\n        >>> t = Symbol('t')\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(16,0),crown_x=8,crown_y=5)\n        >>> a.apply_load(0,'C',start=3,end=5,mag=t)\n        >>> a.solve()\n        >>> a.reaction_force\n        {R_A_x: -4*t/5, R_A_y: -3*t/2, R_B_x: 4*t/5, R_B_y: -t/2}\n\n        >>> a.bending_moment_at(4)\n        -5*t/2\n\nsympy.physics.continuum_mechanics.arch.Arch.draw:\n  This method returns a plot object containing the diagram of the specified arch along with the supports\n        and forces applied to the structure.\n\n        Examples\n        ========\n\n        >>> from sympy import Symbol\n        >>> t = Symbol('t')\n        >>> from sympy.physics.continuum_mechanics.arch import Arch\n        >>> a = Arch((0,0),(40,0),crown_x=20,crown_y=12)\n        >>> a.apply_load(-1,'C',8,150,angle=270)\n        >>> a.apply_load(0,'D',start=20,end=40,mag=-4)\n        >>> a.apply_load(-1,'E',10,t,angle=300)\n        >>> p = a.draw()\n        >>> p # doctest: +ELLIPSIS\n        Plot object containing:\n        [0]: cartesian line: 11.325 - 3*(x - 20)**2/100 for x over (0.0, 40.0)\n        [1]: cartesian line: 12 - 3*(x - 20)**2/100 for x over (0.0, 40.0)\n        ...\n        >>> p.show()",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many implementation details (e.g., Symbol('a', positive=False), _conc_loads, _distributed_loads, _moment_x, _moment_y, _load_x, _load_y, _points_disc_x, _points_disc_y, sympy.Min usage, Piecewise functions, moment equations, rotation formulas for shear/axial, member x-coordinate computation, drawing details like marker types and colors, numpy dependency) are not present in the documentation and cannot be verified from it alone - these are plausible code-level extrapolations but could be hallucinations",
      "The claim about the shape equation formula 'a*(x-x0)^2 + y0' is consistent with the documented examples (e.g., 5 - (x-5)**2/5 matches this pattern)",
      "The solve() example outputs match the documentation exactly: {R_A_x: 8, R_A_y: 12, R_B_x: -8, R_B_y: 8} and {R_A_x: -4*t/5, R_A_y: -3*t/2, R_B_x: 4*t/5, R_B_y: -t/2}",
      "The bending_moment_at(4) = -5*t/2 matches the documentation",
      "The claim about apply_load parameters (order, label, start, mag, end, angle) matches the documentation",
      "The claim about remove_load output matches the documented example",
      "The draw() plot output description is consistent with the documented example showing cartesian lines"
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor leakage in domain terminology ('determinate', 'arch effect'), drawing details from truncated code, and Beam comparison.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_34",
    "repo": "sympy",
    "question": "Why does `TendonForceLengthDeGroote2016` wrap subexpressions in `UnevaluatedExpr`, and how does this interact with SymPy's `Function` evaluation machinery?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `TendonForceLengthDeGroote2016` class uses `UnevaluatedExpr` to prevent constant folding. Here's how the mechanism works:\n\n1. **The `eval` method returns `None` (via `pass`) intentionally**: In SymPy's `Function` architecture (in `sympy/core/function.py`), when `eval()` returns `None`, the function remains unevaluated as a symbolic object. This is critical because `Function._new_()` then checks `_should_evalf` for each argument \u2014 if all arguments are `Float`, it calls `result.evalf()` which triggers the custom `_eval_evalf`.\n\n2. **`_eval_evalf` delegates to `doit(deep=False, evaluate=False)`**: When `evaluate=False`, the `doit` method wraps the subtraction `(l_T_tilde - c1)` in `UnevaluatedExpr` (`c0*exp(c3*UnevaluatedExpr(l_T_tilde - c1)) - c2`). This prevents SymPy from simplifying the inner expression before evaluation.\n\n3. **The `_print_code` method in `CharacteristicCurveFunction` also calls `doit(deep=False, evaluate=False)`** and uses `PRECEDENCE['Atom']` in the `parenthesize` call. This method is aliased to 11 different code printers: `_ccode`, `_cupycode`, `_cxxcode`, `_fcode`, `_jaxcode`, `_lambdacode`, `_mpmathcode`, `_octave`, `_pythoncode`, `_numpycode`, `_scipycode`.\n\n4. **The `fdiff` method also uses `UnevaluatedExpr`**: For all 5 argument derivatives, the `exp(c3*UnevaluatedExpr(l_T_tilde - c1))` pattern is used (except argindex=4 which returns `Integer(-1)`).\n\n5. **`Function.__new__` is decorated with `@cacheit`**: This caches function construction, meaning repeated instantiations with the same arguments won't re-run the construction logic.",
    "rubric": [
      "Explains that the `eval` method returns `None` (via `pass`), which keeps the function unevaluated as a symbolic object in SymPy's `Function.__new__`",
      "Explains that `Function.__new__` checks `_should_evalf` for each argument and if all are Float, calls `evalf()` which triggers the custom `_eval_evalf` method",
      "Explains that `_eval_evalf` delegates to `doit(deep=False, evaluate=False)`, which wraps `(l_T_tilde - c1)` in `UnevaluatedExpr` to prevent constant folding / premature simplification for numerical stability",
      "Mentions that `doit` with `evaluate=True` returns the normal expression without `UnevaluatedExpr`, while `evaluate=False` uses `UnevaluatedExpr` wrapping",
      "Mentions that the `_print_code` method in `CharacteristicCurveFunction` also uses `doit(deep=False, evaluate=False)` and is aliased to multiple code printers (e.g., _ccode, _numpycode, _lambdacode, etc.)",
      "Mentions that the `fdiff` method also uses `UnevaluatedExpr` in its derivative expressions to maintain the same numerical stability"
    ],
    "key_files": [
      "sympy/physics/biomechanics/curve.py",
      "sympy/core/function.py",
      "sympy/core/expr.py"
    ],
    "source_doc": "[docstring: sympy/physics/biomechanics/curve.py] sympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016:\n  Tendon force-length curve based on De Groote et al., 2016 [1]_.\n\n    Explanation\n    ===========\n\n    Gives the normalized tendon force produced as a function of normalized\n    tendon length.\n\n    The function is defined by the equation:\n\n    $fl^T = c_0 \\exp{c_3 \\left( \\tilde{l}^T - c_1 \\right)} - c_2$\n\n    with constant values of $c_0 = 0.2$, $c_1 = 0.995$, $c_2 = 0.25$, and\n    $c_3 = 33.93669377311689$.\n\n    While it is possible to change the constant values, these were carefully\n    selected in the original publication to give the characteristic curve\n    specific and required properties. For example, the function produces no\n    force when the tendon is in an unstrained state. It also produces a force\n    of 1 normalized unit when the tendon is under a 5% strain.\n\n    Examples\n    ========\n\n    The preferred way to instantiate :class:`TendonForceLengthDeGroote2016` is using\n    the :meth:`~.with_defaults` constructor because this will automatically\n    populate the constants within the characteristic curve equation with the\n    floating point values from the original publication. This constructor takes\n    a single argument corresponding to normalized tendon length. We'll create a\n    :class:`~.Symbol` called ``l_T_tilde`` to represent this.\n\n    >>> from sympy import Symbol\n    >>> from sympy.physics.biomechanics import TendonForceLengthDeGroote2016\n    >>> l_T_tilde = Symbol('l_T_tilde')\n    >>> fl_T = TendonForceLengthDeGroote2016.with_defaults(l_T_tilde)\n    >>> fl_T\n    TendonForceLengthDeGroote2016(l_T_tilde, 0.2, 0.995, 0.25,\n    33.93669377311689)\n\n    It's also possible to populate the four constants with your own values too.\n\n    >>> from sympy import symbols\n    >>> c0, c1, c2, c3 = symbols('c0 c1 c2 c3')\n    >>> fl_T = TendonForceLengthDeGroote2016(l_T_tilde, c0, c1, c2, c3)\n    >>> fl_T\n    TendonForceLengthDeGroote2016(l_T_tilde, c0, c1, c2, c3)\n\n    You don't just have to use symbols as the arguments, it's also possible to\n    use expressions. Let's create a new pair of symbols, ``l_T`` and\n    ``l_T_slack``, representing tendon length and tendon slack length\n    respectively. We can then represent ``l_T_tilde`` as an expression, the\n    ratio of these.\n\n    >>> l_T, l_T_slack = symbols('l_T l_T_slack')\n    >>> l_T_tilde = l_T/l_T_slack\n    >>> fl_T = TendonForceLengthDeGroote2016.with_defaults(l_T_tilde)\n    >>> fl_T\n    TendonForceLengthDeGroote2016(l_T/l_T_slack, 0.2, 0.995, 0.25,\n    33.93669377311689)\n\n    To inspect the actual symbolic expression that this function represents,\n    we can call the :meth:`~.doit` method on an instance. We'll use the keyword\n    argument ``evaluate=False`` as this will keep the expression in its\n    canonical form and won't simplify any constants.\n\n    >>> fl_T.doit(evaluate=False)\n    -0.25 + 0.2*exp(33.93669377311689*(l_T/l_T_slack - 0.995))\n\n    The function can also be differentiated. We'll differentiate with respect\n    to l_T using the ``diff`` method on an instance with the single positional\n    argument ``l_T``.\n\n    >>> fl_T.diff(l_T)\n    6.787338754623378*exp(33.93669377311689*(l_T/l_T_slack - 0.995))/l_T_slack\n\n    References\n    ==========\n\n    .. [1] De Groote, F., Kinney, A. L., Rao, A. V., & Fregly, B. J., Evaluation\n           of direct collocation optimal control problem formulations for\n           solving the muscle redundancy problem, Annals of biomedical\n           engineering, 44(10), (2016) pp. 2922-2936\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016.with_defaults:\n  Recommended constructor that will use the published constants.\n\n        Explanation\n        ===========\n\n        Returns a new instance of the tendon force-length function using the\n        four constant values specified in the original publication.\n\n        These have the values:\n\n        $c_0 = 0.2$\n        $c_1 = 0.995$\n        $c_2 = 0.25$\n        $c_3 = 33.93669377311689$\n\n        Parameters\n        ==========\n\n        l_T_tilde : Any (sympifiable)\n            Normalized tendon length.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016.eval:\n  Evaluation of basic inputs.\n\n        Parameters\n        ==========\n\n        l_T_tilde : Any (sympifiable)\n            Normalized tendon length.\n        c0 : Any (sympifiable)\n            The first constant in the characteristic equation. The published\n            value is ``0.2``.\n        c1 : Any (sympifiable)\n            The second constant in the characteristic equation. The published\n            value is ``0.995``.\n        c2 : Any (sympifiable)\n            The third constant in the characteristic equation. The published\n            value is ``0.25``.\n        c3 : Any (sympifiable)\n            The fourth constant in the characteristic equation. The published\n            value is ``33.93669377311689``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016._eval_evalf:\n  Evaluate the expression numerically using ``evalf``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016.doit:\n  Evaluate the expression defining the function.\n\n        Parameters\n        ==========\n\n        deep : bool\n            Whether ``doit`` should be recursively called. Default is ``True``.\n        evaluate : bool.\n            Whether the SymPy expression should be evaluated as it is\n            constructed. If ``False``, then no constant folding will be\n            conducted which will leave the expression in a more numerically-\n            stable for values of ``l_T_tilde`` that correspond to a sensible\n            operating range for a musculotendon. Default is ``True``.\n        **kwargs : dict[str, Any]\n            Additional keyword argument pairs to be recursively passed to\n            ``doit``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016.fdiff:\n  Derivative of the function with respect to a single argument.\n\n        Parameters\n        ==========\n\n        argindex : int\n            The index of the function's arguments with respect to which the\n            derivative should be taken. Argument indexes start at ``1``.\n            Default is ``1``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016.inverse:\n  Inverse function.\n\n        Parameters\n        ==========\n\n        argindex : int\n            Value to start indexing the arguments at. Default is ``1``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthDeGroote2016._latex:\n  Print a LaTeX representation of the function defining the curve.\n\n        Parameters\n        ==========\n\n        printer : Printer\n            The printer to be used to print the LaTeX string representation.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `eval` returns `None` via `pass` is not documented \u2014 this is an implementation detail not mentioned in the documentation. The doc only says eval does 'Evaluation of basic inputs' with parameter descriptions.",
      "The claim about `_eval_evalf` delegating to `doit(deep=False, evaluate=False)` is not in the documentation \u2014 this is an implementation detail that may or may not be accurate.",
      "The claim about `UnevaluatedExpr` wrapping `(l_T_tilde - c1)` in the `doit` method is not documented. The documentation only shows that `doit(evaluate=False)` keeps the expression in canonical form without simplifying constants.",
      "The claim about `_print_code` method, `PRECEDENCE['Atom']`, and the 11 aliased code printers is entirely absent from the documentation \u2014 this is implementation-level detail that cannot be verified from the provided docs.",
      "The claim that `fdiff` uses `UnevaluatedExpr` for all 5 argument derivatives except argindex=4 which returns `Integer(-1)` is not in the documentation. The doc only says fdiff takes an argindex parameter. The claim that argindex=4 returns Integer(-1) seems suspicious \u2014 c2 is subtracted in the formula, so the derivative with respect to c2 would be -1, which would be argindex=4 (c2 is the 4th argument), so this is plausible.",
      "The claim about `Function.__new__` being decorated with `@cacheit` using `lru_cache` with `SYMPY_CACHE_SIZE` defaulting to 1000 is not in the documentation \u2014 this is a general SymPy implementation detail.",
      "The documentation shows the `doit(evaluate=False)` output as `-0.25 + 0.2*exp(33.93669377311689*(l_T/l_T_slack - 0.995))`, which is consistent with the formula but doesn't explicitly mention `UnevaluatedExpr`."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about design rationale (numerical stability, catastrophic cancellation) rely on documentation rather than code, while structural/behavioral claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_35",
    "repo": "sympy",
    "question": "Where is `TendonForceLengthInverseDeGroote2016` defined, what does it inherit from, and how is it used in musculotendon dynamics?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `TendonForceLengthInverseDeGroote2016` class is located in `sympy/physics/biomechanics/curve.py` and inherits from `CharacteristicCurveFunction`, which itself inherits from `sympy.core.function.Function`. It is used in the `MusculotendonBase._tendon_force_explicit_musculotendon_dynamics()` method in `sympy/physics/biomechanics/musculotendon.py`, where it computes the normalized tendon length (`_l_T_tilde`) from the tendon force state variable (`_F_T_tilde`) during the TENDON_FORCE_EXPLICIT formulation of elastic tendon musculotendon dynamics.\n\nKey structural details:\n\n1. The `CharacteristicCurveFunction` base class assigns the `_print_code` method to 11 different printer aliases (`_ccode`, `_cupycode`, `_cxxcode`, `_fcode`, `_jaxcode`, `_lambdacode`, `_mpmathcode`, `_octave`, `_pythoncode`, `_numpycode`, `_scipycode`), which all delegate to `self.doit(deep=False, evaluate=False)` via `printer.parenthesize(..., PRECEDENCE['Atom'])`.\n\n2. The `eval()` classmethod of `TendonForceLengthInverseDeGroote2016` has a body of just `pass`, meaning it returns `None` implicitly, which causes SymPy's `Function.__new__` to leave the expression unevaluated (symbolic). The base class `CharacteristicCurveFunction.eval()` raises a `TypeError` to prevent direct instantiation of the base class.\n\n3. In `fdiff`, when `argindex=5` (derivative w.r.t. c3), the result is `-log(UnevaluatedExpr((fl_T + c2)/c0))/c3**2`, using `UnevaluatedExpr`. When `argindex=3` (derivative w.r.t. c1), it returns `Integer(1)`. When `argindex=2` (derivative w.r.t. c0), it returns `-1/(c0*c3)`.\n\n4. The `inverse()` method returns the class `TendonForceLengthDeGroote2016` (not an instance), and reciprocally, `TendonForceLengthDeGroote2016.inverse()` returns `TendonForceLengthInverseDeGroote2016`.\n\n5. In `musculotendon.py`, in the `_tendon_force_explicit_musculotendon_dynamics` method, when `self._with_defaults` is True, `self.curves.tendon_force_length_inverse.with_defaults(self._fl_T)` is called (where `self._fl_T = self._F_T_tilde`), and the result is assigned to `self._fl_T_inv`, which is then assigned to `self._l_T_tilde`. This normalized tendon length is then used to compute the actual tendon length as `self._l_T = self._l_T_tilde * self._l_T_slack`.\n\n6. `CharacteristicCurveCollection` is defined as a `@dataclass(frozen=True)` (based on the `dataclass` import and its usage in musculotendon.py with field accesses like `self.curves.tendon_force_length`, `self.curves.tendon_force_length_inverse`, `self.curves.fiber_force_length_passive`, `self.curves.fiber_force_length_passive_inverse`, `self.curves.fiber_force_length_active`, `self.curves.fiber_force_velocity`, `self.curves.fiber_force_velocity_inverse`).\n\n7. In `_eval_evalf`, the implementation calls `self.doit(deep=False, evaluate=False)._eval_evalf(prec)` \u2014 it uses `deep=False` and `evaluate=False` before evaluating to a float.",
    "rubric": [
      "Located in sympy/physics/biomechanics/curve.py and inherits from CharacteristicCurveFunction (which inherits from sympy.core.function.Function)",
      "Used in MusculotendonBase._tendon_force_explicit_musculotendon_dynamics() in musculotendon.py to compute normalized tendon length from the tendon force state variable",
      "CharacteristicCurveFunction assigns _print_code to multiple printer aliases (_ccode, _numpycode, _lambdacode, etc.) that delegate to self.doit(deep=False, evaluate=False)",
      "Its eval() classmethod has a body of just pass (returns None), leaving the expression unevaluated/symbolic",
      "Its inverse() method returns the class TendonForceLengthDeGroote2016 (and vice versa)",
      "In musculotendon.py, when _with_defaults is True, self.curves.tendon_force_length_inverse.with_defaults(self._fl_T) is called and the result becomes self._l_T_tilde",
      "The normalized tendon length is used to compute actual tendon length as self._l_T = self._l_T_tilde * self._l_T_slack"
    ],
    "key_files": [
      "sympy/physics/biomechanics/curve.py",
      "sympy/physics/biomechanics/musculotendon.py",
      "sympy/physics/biomechanics/tests/test_curve.py"
    ],
    "source_doc": "[docstring: sympy/physics/biomechanics/curve.py] sympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016:\n  Inverse tendon force-length curve based on De Groote et al., 2016 [1]_.\n\n    Explanation\n    ===========\n\n    Gives the normalized tendon length that produces a specific normalized\n    tendon force.\n\n    The function is defined by the equation:\n\n    ${fl^T}^{-1} = frac{\\log{\\frac{fl^T + c_2}{c_0}}}{c_3} + c_1$\n\n    with constant values of $c_0 = 0.2$, $c_1 = 0.995$, $c_2 = 0.25$, and\n    $c_3 = 33.93669377311689$. This function is the exact analytical inverse\n    of the related tendon force-length curve ``TendonForceLengthDeGroote2016``.\n\n    While it is possible to change the constant values, these were carefully\n    selected in the original publication to give the characteristic curve\n    specific and required properties. For example, the function produces no\n    force when the tendon is in an unstrained state. It also produces a force\n    of 1 normalized unit when the tendon is under a 5% strain.\n\n    Examples\n    ========\n\n    The preferred way to instantiate :class:`TendonForceLengthInverseDeGroote2016` is\n    using the :meth:`~.with_defaults` constructor because this will automatically\n    populate the constants within the characteristic curve equation with the\n    floating point values from the original publication. This constructor takes\n    a single argument corresponding to normalized tendon force-length, which is\n    equal to the tendon force. We'll create a :class:`~.Symbol` called ``fl_T`` to\n    represent this.\n\n    >>> from sympy import Symbol\n    >>> from sympy.physics.biomechanics import TendonForceLengthInverseDeGroote2016\n    >>> fl_T = Symbol('fl_T')\n    >>> l_T_tilde = TendonForceLengthInverseDeGroote2016.with_defaults(fl_T)\n    >>> l_T_tilde\n    TendonForceLengthInverseDeGroote2016(fl_T, 0.2, 0.995, 0.25,\n    33.93669377311689)\n\n    It's also possible to populate the four constants with your own values too.\n\n    >>> from sympy import symbols\n    >>> c0, c1, c2, c3 = symbols('c0 c1 c2 c3')\n    >>> l_T_tilde = TendonForceLengthInverseDeGroote2016(fl_T, c0, c1, c2, c3)\n    >>> l_T_tilde\n    TendonForceLengthInverseDeGroote2016(fl_T, c0, c1, c2, c3)\n\n    To inspect the actual symbolic expression that this function represents,\n    we can call the :meth:`~.doit` method on an instance. We'll use the keyword\n    argument ``evaluate=False`` as this will keep the expression in its\n    canonical form and won't simplify any constants.\n\n    >>> l_T_tilde.doit(evaluate=False)\n    c1 + log((c2 + fl_T)/c0)/c3\n\n    The function can also be differentiated. We'll differentiate with respect\n    to l_T using the ``diff`` method on an instance with the single positional\n    argument ``l_T``.\n\n    >>> l_T_tilde.diff(fl_T)\n    1/(c3*(c2 + fl_T))\n\n    References\n    ==========\n\n    .. [1] De Groote, F., Kinney, A. L., Rao, A. V., & Fregly, B. J., Evaluation\n           of direct collocation optimal control problem formulations for\n           solving the muscle redundancy problem, Annals of biomedical\n           engineering, 44(10), (2016) pp. 2922-2936\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016.with_defaults:\n  Recommended constructor that will use the published constants.\n\n        Explanation\n        ===========\n\n        Returns a new instance of the inverse tendon force-length function\n        using the four constant values specified in the original publication.\n\n        These have the values:\n\n        $c_0 = 0.2$\n        $c_1 = 0.995$\n        $c_2 = 0.25$\n        $c_3 = 33.93669377311689$\n\n        Parameters\n        ==========\n\n        fl_T : Any (sympifiable)\n            Normalized tendon force as a function of tendon length.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016.eval:\n  Evaluation of basic inputs.\n\n        Parameters\n        ==========\n\n        fl_T : Any (sympifiable)\n            Normalized tendon force as a function of tendon length.\n        c0 : Any (sympifiable)\n            The first constant in the characteristic equation. The published\n            value is ``0.2``.\n        c1 : Any (sympifiable)\n            The second constant in the characteristic equation. The published\n            value is ``0.995``.\n        c2 : Any (sympifiable)\n            The third constant in the characteristic equation. The published\n            value is ``0.25``.\n        c3 : Any (sympifiable)\n            The fourth constant in the characteristic equation. The published\n            value is ``33.93669377311689``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016._eval_evalf:\n  Evaluate the expression numerically using ``evalf``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016.doit:\n  Evaluate the expression defining the function.\n\n        Parameters\n        ==========\n\n        deep : bool\n            Whether ``doit`` should be recursively called. Default is ``True``.\n        evaluate : bool.\n            Whether the SymPy expression should be evaluated as it is\n            constructed. If ``False``, then no constant folding will be\n            conducted which will leave the expression in a more numerically-\n            stable for values of ``l_T_tilde`` that correspond to a sensible\n            operating range for a musculotendon. Default is ``True``.\n        **kwargs : dict[str, Any]\n            Additional keyword argument pairs to be recursively passed to\n            ``doit``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016.fdiff:\n  Derivative of the function with respect to a single argument.\n\n        Parameters\n        ==========\n\n        argindex : int\n            The index of the function's arguments with respect to which the\n            derivative should be taken. Argument indexes start at ``1``.\n            Default is ``1``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016.inverse:\n  Inverse function.\n\n        Parameters\n        ==========\n\n        argindex : int\n            Value to start indexing the arguments at. Default is ``1``.\n\nsympy.physics.biomechanics.curve.TendonForceLengthInverseDeGroote2016._latex:\n  Print a LaTeX representation of the function defining the curve.\n\n        Parameters\n        ==========\n\n        printer : Printer\n            The printer to be used to print the LaTeX string representation.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Line numbers (e.g., 'starting at line 306', 'line 434', 'line 784-787', 'line 28-73') are not present in the documentation and cannot be verified \u2014 these could be accurate from source code but are unverifiable from the provided doc.",
      "Claim about `CharacteristicCurveFunction` base class assigning `_print_code` to 11 printer aliases is not mentioned in the documentation \u2014 plausible but unverifiable.",
      "Claim about `eval()` having a body of just `pass` is not in the documentation \u2014 the doc only describes parameters, not implementation details.",
      "Claim about `fdiff` returning specific expressions for argindex=5, argindex=3, argindex=2 with `UnevaluatedExpr` \u2014 the doc only shows the derivative with respect to fl_T (argindex=1) producing `1/(c3*(c2 + fl_T))`. The other derivatives are plausible but unverifiable from the doc.",
      "Claim about `inverse()` returning the class `TendonForceLengthDeGroote2016` is consistent with the doc's description that this is 'the exact analytical inverse of the related tendon force-length curve TendonForceLengthDeGroote2016'.",
      "Claim about `CharacteristicCurveCollection` being a `@dataclass(frozen=True)` with 7 fields and specific `__iter__` order is not in the documentation \u2014 unverifiable.",
      "Claim about `_eval_evalf` using `deep=False` and `evaluate=False` is not explicitly in the documentation but is consistent with the doc's description of evaluate=False keeping expressions numerically stable.",
      "Claims about musculotendon.py usage are beyond the scope of the provided documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with exact line numbers, design rationale phrases, and one slightly inaccurate variable assignment description.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_38",
    "repo": "sympy",
    "question": "Why does the Density class inherit from HermitianOperator, and how does this choice affect its behavior with Dagger and representation?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `Density` class in `sympy/physics/quantum/density.py` inherits from `HermitianOperator` (not just `Operator`). This inheritance chain is `Density \u2192 HermitianOperator \u2192 Operator \u2192 QExpr \u2192 Expr`.\n\n**Why Density inherits from HermitianOperator:** By inheriting from `HermitianOperator`, the `Density` class automatically gets `is_hermitian = True` (set in the `HermitianOperator` class). `Dagger.__new__` calls `arg.adjoint()` when `evaluate=True`, and `QExpr` defines `_eval_adjoint`.\n\n**Design decisions in `_eval_args` validation:** The `Density._eval_args` classmethod first calls `super()._eval_args(args)` which routes through `QExpr._eval_args`, which calls `_qsympify_sequence(args)`. This helper function recursively converts lists/tuples to `Tuple` objects and sympifies all elements. After this conversion, `Density._eval_args` validates that each argument is a `Tuple` of exactly length 2 \u2014 this is the check that ensures each state-probability pair is properly formed. If a user passes `[Ket(0)]` without a probability, the length check raises `ValueError`.\n\n**Design of `doit()` for superposition states:** The `doit()` method uses `state.expand()` before checking `isinstance(state, Add)`. When the state is an `Add` (superposition), it uses `itertools.product(state.args, repeat=2)` to generate all cross-terms (|\u03c8_i\u27e9\u27e8\u03c8_j| for all i,j).\n\n**`_generate_outer_prod` separates commutative and non-commutative parts:** It calls `args_cnc()` on each argument to split into commutative (c_part) and non-commutative (nc_part) components. It requires at least one non-commutative part per argument (raises ValueError otherwise). The outer product is formed as `Mul(*nc_part1)*Dagger(Mul(*nc_part2))` for the non-commutative parts, multiplied by `Mul(*c_part1)*Mul(*c_part2)` for the commutative coefficients. Note that the Dagger is only applied to the second argument's nc_part, not the first.\n\n**`_represent` delegates through `doit()`:** The `_represent` method first calls `self.doit()` to expand into outer products, then passes the result to the module-level `represent()` function. This means representation always goes through the expanded outer product form.\n\n**`_eval_trace` calls `Tr(self.doit(), indices).doit()`:** The trace implementation first expands the density operator via `doit()`, then creates a `Tr` object with the expanded form and any partial trace indices, and immediately evaluates it. The `Tr` class (in trace.py) dispatches its `doit()` by checking if the argument has `_eval_trace` defined \u2014 for `OuterProduct` this delegates to `self.ket._eval_trace(self.bra, **kwargs)` in operator.py.\n\n**Entropy computation uses eigenvalue decomposition:** The `entropy()` function converts a `Density` object to a matrix via `represent(density)`, then computes `-\u03a3 \u03bb*log(\u03bb)` using `density.eigenvals().keys()` for SymPy matrices. For numpy arrays, it uses `np.linalg.eigvals()`. For scipy sparse matrices, it first converts to numpy via `to_numpy()` before computing.\n\n**Fidelity uses matrix square root:** The `fidelity()` function computes `Tr(\u221a(\u221a\u03c1\u2081 \u00b7 \u03c1\u2082 \u00b7 \u221a\u03c1\u2081))` using `state1**S.Half` for the matrix square root, where `S.Half` is SymPy's rational half. The dimension validation has a potential bug: the condition `state1.shape != state2.shape and state1.is_square` uses `and` instead of `or`, meaning non-square matrices of equal shape would pass validation.\n\n**LaTeX and pretty printing:** Density prints as `\\rho` in LaTeX and uses the Unicode character `\u03c1` (GREEK SMALL LETTER RHO, via `\\N{GREEK SMALL LETTER RHO}`) for pretty printing, overriding the default `Operator` printing which uses the class name.\n\n**`numpy_ndarray` and `scipy_sparse_matrix` are conditionally defined:** In `matrixutils.py`, if numpy/scipy are not installed, dummy empty classes are defined so `isinstance` checks always return `False`.",
    "rubric": [
      "Explains that Density inherits from HermitianOperator (not plain Operator), which sets is_hermitian = True",
      "Explains the inheritance chain: Density \u2192 HermitianOperator \u2192 Operator \u2192 QExpr \u2192 Expr",
      "Explains that this means Dagger(Density(...)) returns the Density unchanged since HermitianOperator satisfies H == Dagger(H)",
      "Mentions that _represent delegates through doit() which expands the density into outer product form before calling represent()",
      "Mentions that _eval_args validates each argument is a Tuple of length 2 (state-probability pair), raising ValueError otherwise",
      "Mentions that doit() handles superposition states by expanding and using itertools.product to generate cross-terms"
    ],
    "key_files": [
      "sympy/physics/quantum/density.py",
      "sympy/physics/quantum/operator.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/trace.py",
      "sympy/physics/quantum/dagger.py",
      "sympy/physics/quantum/matrixutils.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/density.py] sympy.physics.quantum.density.Density\nsympy.physics.quantum.density.Density:\n  Density operator for representing mixed states.\n\n    TODO: Density operator support for Qubits\n\n    Parameters\n    ==========\n\n    values : tuples/lists\n    Each tuple/list should be of form (state, prob) or [state,prob]\n\n    Examples\n    ========\n\n    Create a density operator with 2 states represented by Kets.\n\n    >>> from sympy.physics.quantum.state import Ket\n    >>> from sympy.physics.quantum.density import Density\n    >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n    >>> d\n    Density((|0>, 0.5),(|1>, 0.5))\n\nsympy.physics.quantum.density.Density.states:\n  Return list of all states.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.states()\n        (|0>, |1>)\n\nsympy.physics.quantum.density.Density.probs:\n  Return list of all probabilities.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.probs()\n        (0.5, 0.5)\n\nsympy.physics.quantum.density.Density.get_state:\n  Return specific state by index.\n\n        Parameters\n        ==========\n\n        index : index of state to be returned\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.states()[1]\n        |1>\n\nsympy.physics.quantum.density.Density.get_prob:\n  Return probability of specific state by index.\n\n        Parameters\n        ===========\n\n        index : index of states whose probability is returned.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.probs()[1]\n        0.500000000000000\n\nsympy.physics.quantum.density.Density.apply_op:\n  op will operate on each individual state.\n\n        Parameters\n        ==========\n\n        op : Operator\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> from sympy.physics.quantum.operator import Operator\n        >>> A = Operator('A')\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.apply_op(A)\n        Density((A*|0>, 0.5),(A*|1>, 0.5))\n\nsympy.physics.quantum.density.Density.doit:\n  Expand the density operator into an outer product format.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.quantum.state import Ket\n        >>> from sympy.physics.quantum.density import Density\n        >>> from sympy.physics.quantum.operator import Operator\n        >>> A = Operator('A')\n        >>> d = Density([Ket(0), 0.5], [Ket(1),0.5])\n        >>> d.doit()\n        0.5*|0><0| + 0.5*|1><1|\n\nsympy.physics.quantum.density.Density.entropy:\n  Compute the entropy of a density matrix.\n\n        Refer to density.entropy() method  for examples.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not mention HermitianOperator inheritance, MRO chain, or implementation details like _eval_args, _generate_outer_prod, _represent, _eval_trace, etc. These are plausible code-level details but cannot be verified from the provided documentation.",
      "The documentation does not describe LaTeX printing, pretty printing, fidelity function, or matrixutils.py details. These are extrapolations beyond the documentation scope.",
      "The claim about entropy using eigenvalue decomposition is not contradicted but also not supported by the documentation, which only says 'Refer to density.entropy() method for examples'.",
      "The documentation shows d.states()[1] and d.probs()[1] as examples for get_state and get_prob respectively, but doesn't describe the internal implementation details claimed in the answer.",
      "The doit() example in the documentation shows '0.5*|0><0| + 0.5*|1><1|' which is consistent with the answer's description of outer product expansion."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only physics rationale explanations and a couple of line number references needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_37",
    "repo": "sympy",
    "question": "What is the architecture connecting `pole_zero_numerical_data` to the LTI class hierarchy, including how roots are computed and how the plot is rendered?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `pole_zero_numerical_data` function in `sympy/physics/control/control_plots.py` is part of a control systems plotting architecture that bridges symbolic LTI system representations with numerical matplotlib visualization. Here is how it works architecturally:\n\n**Function Implementation:**\n1. It first calls `_check_system(system)` to validate the input is a `SISOLinearTimeInvariant` instance, has at most 1 free symbol, and doesn't contain `exp` (time delay terms).\n2. It calls `system.doit()` to reduce composite system objects (like `Series`, `Parallel`, `Feedback`, `PIDController`) into a basic `TransferFunction` (or `DiscreteTransferFunction`).\n3. It constructs `Poly` objects from `system.num` and `system.den` using `system.var`.\n4. It calls the helper `_poly_roots()` on each polynomial to compute zeros and poles.\n\n**Root computation via `_poly_roots`:**\nThis helper function has a branching strategy based on the polynomial's coefficient domain:\n- If the polynomial domain is `QQ` (rationals) or `ZZ` (integers), it calls `poly.all_roots()` for exact algebraic root computation.\n- Otherwise (for other coefficient domains), it falls back to `poly.nroots()` for numerical approximation.\n- The inner `_eval` function converts results: real roots become Python `float`, complex roots become Python `complex`.\n\n**Class Hierarchy:**\n- `LinearTimeInvariant(Basic, EvalfMixin, ABC)` \u2192 `SISOLinearTimeInvariant` \u2192 `TransferFunctionBase(ABC)` \u2192 `TransferFunction` / `DiscreteTransferFunction`\n- `SISOLinearTimeInvariant` also parents `Series`, `Parallel`, and `Feedback`\n- `SISOLinearTimeInvariant._is_SISO = True` and defines `num_inputs` and `num_outputs` both returning 1.\n- `PIDController` is referenced in the module's `__all__` and imports.\n\n**`doit()` behavior:**\n- `TransferFunctionBase` does not define its own `doit()` method.\n\n**Consumer:** `pole_zero_plot` calls `pole_zero_numerical_data` and separates real/imaginary parts. Poles are plotted as 'x' markers (blue, size 10) and zeros as 'o' markers (orange, size 7). The plot title uses LaTeX rendering via `sympy.printing.latex.latex(system)`.\n\n**Module-level exports:** The `__all__` list in `control_plots.py` exports 17 functions total, including all numerical data functions, plot functions, and expression-generating functions (`nyquist_plot_expr`, `nichols_plot_expr`).",
    "rubric": [
      "Explains that `pole_zero_numerical_data` calls `_check_system` to validate input is a `SISOLinearTimeInvariant` instance, checks for at most 1 free symbol, and rejects `exp` (time delay) terms",
      "Explains that `system.doit()` is called to reduce composite system types (Series, Parallel, Feedback, PIDController) into a base TransferFunction",
      "Describes that `Poly` objects are constructed from `system.num` and `system.den` using `system.var`, then `_poly_roots()` is called on each",
      "Explains the branching logic in `_poly_roots`: uses `poly.all_roots()` for QQ/ZZ domains, falls back to `poly.nroots()` otherwise, and converts results to Python float/complex",
      "Describes the class hierarchy: `LinearTimeInvariant(Basic, EvalfMixin, ABC)` \u2192 `SISOLinearTimeInvariant` \u2192 `TransferFunctionBase(ABC)` \u2192 `TransferFunction`/`DiscreteTransferFunction`",
      "Notes that `Series`, `Parallel`, `Feedback` inherit from `SISOLinearTimeInvariant` directly (not through TransferFunctionBase)",
      "Explains that `pole_zero_plot` consumes the numerical data and plots poles as 'x' markers (blue, size 10) and zeros as 'o' markers (orange, size 7)"
    ],
    "key_files": [
      "sympy/physics/control/control_plots.py",
      "sympy/physics/control/lti.py",
      "sympy/polys/polytools.py",
      "sympy/physics/control/__init__.py"
    ],
    "source_doc": "[docstring: sympy/physics/control/control_plots.py] sympy.physics.control.control_plots.pole_zero_numerical_data\nReturns the numerical data of poles and zeros of the system.\n    It is internally used by ``pole_zero_plot`` to get the data\n    for plotting poles and zeros. Users can use this data to further\n    analyse the dynamics of the system or plot using a different\n    backend/plotting-module.\n\n    Parameters\n    ==========\n\n    system : SISOLinearTimeInvariant\n        The system for which the pole-zero data is to be computed.\n\n    Returns\n    =======\n\n    tuple : (zeros, poles)\n        zeros = Zeros of the system as a list of Python float/complex.\n        poles = Poles of the system as a list of Python float/complex.\n\n    Raises\n    ======\n\n    NotImplementedError\n        When a SISO LTI system is not passed.\n\n        When time delay terms are present in the system.\n\n    ValueError\n        When more than one free symbol is present in the system.\n        The only variable in the transfer function should be\n        the variable of the Laplace transform.\n\n    Examples\n    ========\n\n    >>> from sympy.abc import s\n    >>> from sympy.physics.control.lti import TransferFunction\n    >>> from sympy.physics.control.control_plots import pole_zero_numerical_data\n    >>> tf1 = TransferFunction(s**2 + 1, s**4 + 4*s**3 + 6*s**2 + 5*s + 2, s)\n    >>> pole_zero_numerical_data(tf1)\n    ([-1j, 1j], [-2.0, -1.0, (-0.5-0.8660254037844386j), (-0.5+0.8660254037844386j)])\n\n    See Also\n    ========\n\n    pole_zero_plot",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.23,
    "strip_verify_summary": "Several claims about doit() implementations and internal method details rely on code not shown in the truncated stripped files, and one claim about a comment referencing a GitHub issue is pure doc-leakage.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_40",
    "repo": "sympy",
    "question": "How does the `bilinear` transform function work, and how do the discretization methods relate to the generalized bilinear transform?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `bilinear` function in `sympy/physics/control/lti.py` is implemented as a thin wrapper that delegates entirely to the more general `gbt` (Generalized Bilinear Transform) function with `alpha=S.Half` (i.e., 1/2). Here's how the system is designed:\n\n**Delegation pattern:** `bilinear`, `forward_diff`, and `backward_diff` are all special cases of `gbt`:\n- `bilinear(tf, sample_per)` calls `gbt(tf, sample_per, S.Half)` (alpha=1/2)\n- `forward_diff(tf, sample_per)` calls `gbt(tf, sample_per, S.Zero)` (alpha=0)\n- `backward_diff(tf, sample_per)` calls `gbt(tf, sample_per, S.One)` (alpha=1)\n\n**Algorithm in `gbt`:**\n1. First checks `tf.is_SISO` and raises `NotImplementedError` for MIMO systems.\n2. Reuses `tf.var` (the Laplace variable `s`) as a dummy discrete variable `z` \u2014 the same symbol is repurposed rather than introducing a new symbol.\n3. Extracts numerator and denominator polynomial coefficients via `tf.num.as_poly(s).all_coeffs()` and `tf.den.as_poly(s).all_coeffs()`.\n4. Converts the alpha parameter to a SymPy `Rational` with `Rational(alpha).limit_denominator(1000)`, which limits the denominator to at most 1000.\n5. Computes `N = max(len(np), len(dp)) - 1` (the maximum polynomial order).\n6. Constructs the discrete numerator and denominator using the formula: `sum of T^(N-i) * c * (z-1)^i * (alpha*z + 1 - alpha)^(N-i)` for each coefficient `c` (reversed order).\n7. Converts the resulting expressions to polynomials in `z` and extracts coefficients.\n8. Normalizes by dividing all coefficients by the leading denominator coefficient (`den_coefs[0]`), so the first denominator coefficient is always 1.\n\n**Integration with `DiscreteTransferFunction`:** The `DiscreteTransferFunction` class has class methods `from_gbt`, `from_bilinear`, `from_forward_diff`, and `from_backward_diff` that each call the corresponding standalone function and then pass the resulting coefficient lists to `cls.from_coeff_lists(num, den, var, sampling_time)`. The `from_coeff_lists` method (defined in `TransferFunctionBase`) reconstructs polynomial expressions from the coefficient lists by reversing them, creating power terms, and summing them.\n\n**Class hierarchy:** `TransferFunction` inherits from `TransferFunctionBase` \u2192 `SISOLinearTimeInvariant` \u2192 `LinearTimeInvariant` \u2192 `Basic, EvalfMixin, ABC`. The `_is_SISO = True` flag is set on `SISOLinearTimeInvariant`, which is what `gbt` checks before proceeding.\n\n**Export:** All four functions (`gbt`, `bilinear`, `forward_diff`, `backward_diff`) are listed in `__all__` and re-exported from `sympy/physics/control/__init__.py`.",
    "rubric": [
      "Explains that `bilinear`, `forward_diff`, and `backward_diff` are thin wrappers that delegate to the `gbt` function with alpha values of S.Half (1/2), S.Zero (0), and S.One (1) respectively",
      "Describes the `gbt` algorithm: checks `tf.is_SISO`, extracts polynomial coefficients from numerator and denominator, and reuses the Laplace variable as a discrete variable z",
      "Describes the core GBT formula: constructs discrete numerator/denominator via sum of T^(N-i) * c * (z-1)^i * (alpha*z + 1-alpha)^(N-i) terms",
      "Mentions that alpha is converted to a Rational with limit_denominator(1000)",
      "Explains the normalization step: all coefficients are divided by the leading denominator coefficient so the first denominator coefficient becomes 1",
      "Describes how DiscreteTransferFunction class methods (from_gbt, from_bilinear, etc.) call the standalone functions and pass results to from_coeff_lists to construct the discrete transfer function object"
    ],
    "key_files": [
      "sympy/physics/control/lti.py",
      "sympy/physics/control/tests/test_lti.py",
      "sympy/physics/control/__init__.py"
    ],
    "source_doc": "[docstring: sympy/physics/control/lti.py] sympy.physics.control.lti.bilinear\nReturns falling coefficients of H(z) from numerator and denominator.\n\n    Parameters\n    ==========\n\n    tf : TransferFunction\n        The continuous transfer function H(s) to be discretized.\n    sampling_time : Symbol, Number\n        Time interval between two consecutive sampling instants.\n\n    Explanation\n    ===========\n\n    Where H(z) is the corresponding discretized transfer function,\n    discretized with the bilinear transform method.\n    H(z) is obtained from the continuous transfer function H(s)\n    by substituting $s(z) = \\frac{2}{T}\\frac{z-1}{z+1}$ into H(s), where T is\n    the sample period.\n    Coefficients are falling, i.e. $H(z) = \\frac{az+b}{cz+d}$ is returned\n    as [a, b], [c, d].\n\n    Examples\n    ========\n\n    >>> from sympy.physics.control.lti import TransferFunction, bilinear\n    >>> from sympy.abc import s, L, R, T\n\n    >>> tf = TransferFunction(1, s*L + R, s)\n    >>> numZ, denZ = bilinear(tf, T)\n    >>> numZ\n    [T/(2*(L + R*T/2)), T/(2*(L + R*T/2))]\n    >>> denZ\n    [1, (-L + R*T/2)/(L + R*T/2)]",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation describes the substitution as s(z) = (2/T)*(z-1)/(z+1), but the answer describes the gbt formula as s = (z-1) / (T * (alpha*z + 1-alpha)). For alpha=1/2, this gives s = (z-1)/(T*(z/2 + 1/2)) = (z-1)/(T*(z+1)/2) = 2(z-1)/(T*(z+1)), which is consistent with the documentation.",
      "The claim about using Rational(alpha).limit_denominator(1000) is not verifiable from the documentation and could be a hallucination about implementation details.",
      "The claim about reusing tf.var as dummy discrete variable z is not verifiable from the documentation.",
      "The detailed algorithm steps (lines 110-133, specific implementation details) go far beyond what the documentation states and cannot be verified from it.",
      "The class hierarchy details (TransferFunctionBase \u2192 SISOLinearTimeInvariant \u2192 LinearTimeInvariant \u2192 Basic, EvalfMixin, ABC) cannot be verified from the documentation.",
      "The DiscreteTransferFunction class methods and from_coeff_lists details cannot be verified from the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the mathematical rationale for the substitution formula came from documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_41",
    "repo": "sympy",
    "question": "What is the RouthHurwitz class's inheritance structure, and what are the key steps in how it constructs and handles special cases in the table?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `RouthHurwitz` class is a specialized matrix class that constructs a table from polynomials. It inherits from `MutableDenseMatrix` (which itself inherits from `DenseMatrix` and `MutableRepMatrix`), making it a mutable matrix that supports element assignment via `__setitem__`. Here are the key implementation details:\n\n**Class Architecture:**\n- `RouthHurwitz` extends `MutableDenseMatrix` (from `sympy.matrices.dense`), which is the same class aliased as `Matrix` in SymPy. This means the Routh-Hurwitz table IS a SymPy Matrix, supporting all standard matrix operations like slicing (`[:, 0]`), `.equals()`, etc.\n- The inheritance chain is: `RouthHurwitz \u2192 MutableDenseMatrix \u2192 DenseMatrix + MutableRepMatrix \u2192 RepMatrix \u2192 MatrixBase`.\n\n**Construction (`__new__` and `__init__`):**\n- In `__new__`, the table dimensions are computed as `(n+1)` rows and `(n//2 + 1)` columns, where `n` is the polynomial degree. The table is initialized with all zeros: `[0]*(n + 1)*(n//2 + 1)`.\n- In `__init__`, the polynomial is converted to a `Poly` object, and `all_coeffs()` is called to get the coefficients. The `_build_table()` method is then called (unless degree < 1, in which case only `self[0, 0]` is set).\n\n**Initialization (`_initialize`):**\n- Coefficients are distributed into the first two rows using an alternating pattern: even-indexed coefficients go to row 0, odd-indexed to row 1. The alternation is controlled by `row = (row+1) % 2` and `col = col + 1 - row`.\n- After filling the first two rows, if `self[1, 0]` is zero, `_handle_special_cases(1)` is called immediately.\n\n**Row Calculation (`_calculate_row`):**\n- Each element is computed as `(self[i-1, 0] * self[i-2, j+1] - self[i-2, 0] * self[i-1, j+1]) / self[i-1, 0]`.\n- The active row length decreases with row index: `active_row_length = self.cols - i//2`.\n\n**Special Case Handling (`_handle_special_cases`):**\n1. **Full Row Zero Case:** When all elements in the active portion of a row are zero, the auxiliary polynomial degree is computed as `self._poly_degree - i + 1`. The zero row is replaced by multiplying each element from the row above by its corresponding degree in the auxiliary polynomial: `self[i, j] = self[i - 1, j] * aux_coeff_deg` where `aux_coeff_deg = aux_poly_degree - 2*j`.\n\n2. **First Column Zero Case:** When only the first column is zero (but the row is not entirely zero), the number of consecutive leading zeros is counted. Then elements are shifted: `self[i, k] = self[i, k] + (-1)**n_zeros * expr` for each `expr` in `self[i, n_zeros:active_row_length]`.\n\n**`auxiliary_polynomials` property:**\n- Reconstructs polynomials from stored degree info. The row used is `self._poly_degree - aux_poly_degree`, and the polynomial is built as a sum of `self[row, j] * var**exp` where `exp` ranges from `aux_poly_degree` down by 2.\n\n**Note on parameters:** The `__new__` and `__init__` signatures only accept `polynomial` and `var` as parameters.\n\n**Testing:** The test file (`test_routh_table.py`) contains exactly 3 test cases: a generic symbolic polynomial (degree 4, `b4*s^4 + b3*s^3 + b2*s^2 + b1*s + b0`), a case with polynomial `s^4 + s^3 + 3s^2 + 3s + 3` (where `zero_row_case` is False), and a case with polynomial `s^6 + 2s^5 + 8s^4 + 12s^3 + 20s^2 + 16s + 16` (where `zero_row_case` is True and auxiliary polynomials are verified).",
    "rubric": [
      "States that RouthHurwitz inherits from MutableDenseMatrix, making it a mutable SymPy matrix supporting standard matrix operations",
      "Explains that __new__ computes table dimensions as (n+1) rows by (n//2+1) columns initialized with zeros, where n is the polynomial degree",
      "Explains that __init__ converts the polynomial to a Poly object, extracts coefficients via all_coeffs(), and calls _build_table()",
      "Describes _initialize: coefficients are distributed into the first two rows using alternating row assignment (row = (row+1) % 2, col = col + 1 - row)",
      "Describes _calculate_row formula: (self[i-1,0]*self[i-2,j+1] - self[i-2,0]*self[i-1,j+1]) / self[i-1,0] with active_row_length = self.cols - i//2",
      "Explains the full row zero special case: replaces zeros using derivative of auxiliary polynomial by multiplying row above elements by corresponding degree values (aux_poly_degree - 2*j)",
      "Explains the first column zero special case: counts consecutive leading zeros, then shifts non-zero elements into the row using (-1)**n_zeros multiplier",
      "Describes auxiliary_polynomials property: reconstructs polynomials from stored degree info using elements from the row above the zero row"
    ],
    "key_files": [
      "sympy/physics/control/routh_table.py",
      "sympy/physics/control/__init__.py",
      "sympy/physics/control/tests/test_routh_table.py",
      "sympy/matrices/dense.py"
    ],
    "source_doc": "[docstring: sympy/physics/control/routh_table.py] sympy.physics.control.routh_table.RouthHurwitz\nsympy.physics.control.routh_table.RouthHurwitz:\n  A class for creating a Routh-Hurwitz table from a given polynomial.\n    It handles special cases with methods discussed in [1]_.\n\n    Note: When at least a row of the table is zero,\n    the property ``zero_row_case`` is set to True.\n\n    Explanation\n    ============\n\n    In mathematics, the Routh-Hurwitz table is used to determine the number of\n    roots of a polynomial that have positive or negative real parts.\n\n    It's crucial in the control system theory because it can be used to\n    retrieve necessary and sufficient conditions for the stability of a linear\n    time-invariant control system.\n\n    Once the table is constructed, the stability of the system can be assessed\n    by counting the number of sign changes in the first column.\n    Each sign change corresponds to a root with a positive real part, whereas\n    each preservation of sign corresponds to a root with a negative real part.\n\n    There are two special cases to consider during the construction of the\n    table:\n\n    1.  First Column Zero Case:\n        If a zero appears in the first column of a row (while the row is not\n        entirely zero), the Extended Routh's Table is constructed [2]_ and every\n        information of these rows is stored in  ``zero_col_infos``.\n\n    2.  Full Row Zero Case:\n        If an entire row becomes zero, we can substitute the row with the\n        coefficients of the derivative of an auxiliary polynomial.\n        The auxiliary polynomial is constructed using the row immediately\n        above the zero row [3]_.\n        For instance, consider the following example:\n\n        .. math::\n            \\begin{matrix}3\\\\2\\\\1\\end{matrix}\\begin{bmatrix}b_3&b_1\\\\\n            b_2&b_0\\\\ 0&0\\end{bmatrix}\n\n        The auxiliary polynomial will be: :math:`a(s) = b_2 s^2 + b_0`\n        The characteristic is that, if :math:`p(s)` is the polynomial we are\n        analyzing, :math:`p(s)=a(s)\\cdot other(s)` and\n        the roots of :math:`a(s)` are symmetric about the origin in the\n        s-plane, so when we\n        fall in this case, we should note that there could be poles with only\n        imaginary part or poles with negative and positive real parts.\n\n\n    The table is constructed for a polynomial of the form\n    :math:`p(s) = b_n s^n + b_{n-1} s^{n-1} + \\ldots + b_1 s + b_0`\n    and the table has :math:`n+1` rows and the following structure:\n\n    .. math::\n        \\begin{bmatrix}b_n&b_{n-2}&b_{n-4}&\\cdots\\\\\n        b_{n-1}&b_{n-3}&b_{n-5}&\\cdots\\\\ c_{1}&c_2&c_3&\\cdots\\\\\n            d_{1}&d_2&d_3&\\cdots\\\\ \\vdots&\\vdots&\\vdots&\\ddots\\end{bmatrix}\n\n    In this table, the elements in the subsequent rows are computed using the\n    formulas:\n    :math:`c_i = \\frac{b_{n-1}\\cdot b_{n-2i}-b_n\\cdot b_{n-(2i+1)}}{b_{n-1}}`\n\n    :math:`d_i = \\frac{c_1 \\cdot b_{n-(2i+1)}-b_{n-1}\\cdot c_{i+1}}{c_1}`\n\n    Parameters\n    ==========\n\n    polynomial : :py:class:`~.Expr`, :py:class:`~.Number`\n        The polynomial whose Routh-Hurwitz table is to be created.\n    var : :py:class:`~.Symbol`\n        The symbol representing the variable in the polynomial.\n    infinitesimal_element : None, :py:class:`~.Symbol`, optional\n        The symbol representing the infinitesimal element for the first column\n        zero case.\n        If not provided, a default symbol ``epsilon`` will be used.\n\n    Examples\n    ========\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.control import RouthHurwitz\n    >>> b1, b2, b3, b4 = symbols('b_1 b_2 b_3 b_4')\n    >>> s = symbols(\"s\")\n\n    Here is a generic example of how to use the ``RouthHurwitz`` class:\n\n    >>> p = b1*s**3 + b2*s**2 + b3*s + b4\n    >>> RouthHurwitz(p, s)\n    Matrix([\n    [                     b_1, b_3],\n    [                     b_2, b_4],\n    [(-b_1*b_4 + b_2*b_3)/b_2,   0],\n    [                     b_4,   0]])\n    >>> RouthHurwitz(p, s)[:, 0]\n    Matrix([\n    [                     b_1],\n    [                     b_2],\n    [(-b_1*b_4 + b_2*b_3)/b_2],\n    [                     b_4]])\n\n    Here you can see how the table appears in the first column zero case:\n\n    >>> p1 = s**4 + s**3 + 3*s**2 + 3*s + 3\n    >>> RouthHurwitz(p1, s)\n    Matrix([\n    [ 1, 3, 3],\n    [ 1, 3, 0],\n    [-3, 3, 0],\n    [ 4, 0, 0],\n    [ 3, 0, 0]])\n    >>> RouthHurwitz(p1, s).zero_col_infos\n    [(2, 1)]\n    >>> RouthHurwitz(p1, s).zero_row_case\n    False\n\n    Here you can see how the table appears in the full row zero case\n    (poles with only imaginary part):\n\n    >>> p2 = s**6 + 2*s**5 + 8*s**4 + 12*s**3 + 20*s**2 + 16*s + 16\n    >>> RouthHurwitz(p2, s)\n    Matrix([\n    [  1,  8, 20, 16],\n    [  2, 12, 16,  0],\n    [  2, 12, 16,  0],\n    [  8, 24,  0,  0],\n    [  6, 16,  0,  0],\n    [8/3,  0,  0,  0],\n    [ 16,  0,  0,  0]])\n    >>> RouthHurwitz(p2, s).zero_row_case\n    True\n    >>> RouthHurwitz(p2, s).auxiliary_polynomials\n    [Poly(2*s**4 + 12*s**2 + 16, s, domain='ZZ')]\n\n    References\n    ==========\n    .. [1] https://en.wikipedia.org/wiki/Routh-Hurwitz_stability_criterion\n    .. [2] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=b1ed2c8cbd00da0a4aac7b7e9684255a833af6b4\n    .. [3] https://www.circuitbread.com/tutorials/routh-hurwitz-criterion-part-2-3-3\n\nsympy.physics.control.routh_table.RouthHurwitz._build_table:\n  Build the Routh-Hurwitz table.\n\nsympy.physics.control.routh_table.RouthHurwitz._initialize:\n  \"Initialize the table with the coefficients of the polynomial.\n\nsympy.physics.control.routh_table.RouthHurwitz._calculate:\n  Calculate the table using the first 2 rows.\n\nsympy.physics.control.routh_table.RouthHurwitz._count_consecutive_zeros:\n  Count the number of consecutive zeros in the i-th row of the table.\n\nsympy.physics.control.routh_table.RouthHurwitz.zero_col_infos:\n  Return a list of tuple.\n\n        - The first element of the tuple represents the index of a row in which\n          the First Column Zero Case occurs.\n        - The second element of the tuple represents the index of the first\n          column different from 0 before the Extended Routh's Table construction.\n\nsympy.physics.control.routh_table.RouthHurwitz.zero_row_case:\n  Return True if during the building of the table the Full Row Zero Case\n        (see the explanation section) has been encountered, else False.\n\nsympy.physics.control.routh_table.RouthHurwitz.auxiliary_polynomials:\n  If ``zero_row_case`` is True, returns a list of auxiliary polynomials\n        associated with the Full Row Zero Case.\n        Otherwise, return None.\n\n        It is used to handle the Full Row Zero Case during the\n        construction of the Routh-Hurwitz table.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims the `infinitesimal_element` parameter is NOT actually accepted in `__new__` and `__init__` signatures, contradicting the documentation which explicitly lists it as an optional parameter with description. This is a significant claim that cannot be verified from the documentation alone and may be incorrect.",
      "The answer provides detailed implementation specifics (e.g., `_calculate_row` formula, `_handle_special_cases` logic, `_initialize` alternating pattern, `MutableDenseMatrix` inheritance, `DomainMatrix` internals, exact number of test cases) that go well beyond the documentation. These are plausible code-level details but cannot be verified from the provided documentation.",
      "The generic example in the documentation uses degree 3 (`b1*s**3 + b2*s**2 + b3*s + b4`), but the answer states the test file contains 'a generic symbolic polynomial (degree 4)'. This is a minor discrepancy \u2014 the doc shows degree 3, though it's possible the test file differs.",
      "The answer's description of the Extended Routh's Table shifting operation (`self[i, k] = self[i, k] + (-1)**n_zeros * expr`) is implementation detail not verifiable from the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in domain-specific naming (Extended Routh's Table, epsilon method), descending order description, and _rep attribute reference; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_39",
    "repo": "sympy",
    "question": "Where does data flow through the `gbt` function \u2014 what calls it, how does it transform its inputs, and what consumes its output?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `gbt` function in `sympy/physics/control/lti.py` is a module-level function. Here's how data flows through it:\n\n**Input data flow:**\n1. `gbt` receives a `TransferFunction` (or subclass of `TransferFunctionBase`) object, a sample period, and an alpha parameter.\n2. It first checks `tf.is_SISO` (which comes from `_is_SISO = True` on `SISOLinearTimeInvariant` class), raising `NotImplementedError` for MIMO systems.\n3. It extracts coefficients using `tf.num.as_poly(s).all_coeffs()` and `tf.den.as_poly(s).all_coeffs()` \u2014 where `num` and `den` are accessed via `self.args[0]` and `self.args[1]` respectively.\n4. The `alpha` parameter is converted to a `Rational` and then `limit_denominator(1000)` is called.\n\n**Internal computation:**\n- The function reuses `tf.var` (the continuous `s` variable) as both `s` and `z` \u2014 i.e., `z = s`.\n- `N = max(len(np), len(dp)) - 1` determines the polynomial order.\n- The numerator and denominator are constructed using `sympy.core.add.Add` with a list comprehension that iterates over reversed coefficient lists with `(z-1)**i * (alpha * z + 1 - alpha)**(N-i)` terms.\n- After converting back to polynomial form with `as_poly(z).all_coeffs()`, coefficients are normalized by dividing by `den_coefs[0]`.\n\n**Output data flow (callers):**\n1. `bilinear(tf, sample_per)` calls `gbt(tf, sample_per, S.Half)`\n2. `forward_diff(tf, sample_per)` calls `gbt(tf, sample_per, S.Zero)`\n3. `backward_diff(tf, sample_per)` calls `gbt(tf, sample_per, S.One)`\n4. `DiscreteTransferFunction.from_gbt(cls, cont_tf, sampling_time, alpha, var)` calls `gbt(cont_tf, sampling_time, alpha)` and passes the returned coefficient lists to `cls.from_coeff_lists(num, den, var, sampling_time)`.\n5. `DiscreteTransferFunction.from_bilinear`, `from_forward_diff`, and `from_backward_diff` call `bilinear`, `forward_diff`, and `backward_diff` respectively, which in turn delegate to `gbt`.\n\n**Downstream reconstruction:**\n- `from_coeff_lists` reverses the falling coefficient lists, multiplies each by `var**i`, and sums them to reconstruct numerator/denominator polynomials, then calls the class constructor.\n\nThe function is exported in `__all__` at both the module level (`sympy/physics/control/lti.py`) and the package level (`sympy/physics/control/__init__.py`).",
    "rubric": [
      "Explains that gbt receives a TransferFunction, sample period, and alpha parameter, and checks is_SISO before proceeding",
      "Describes that coefficients are extracted via tf.num.as_poly(s).all_coeffs() and tf.den.as_poly(s).all_coeffs()",
      "Mentions that alpha is converted to a Rational with limit_denominator(1000)",
      "Explains the internal computation: reuses tf.var as both s and z, computes N = max(len(np), len(dp)) - 1, builds polynomials using (z-1)**i * (alpha*z + 1-alpha)**(N-i) terms via Add",
      "Notes that output coefficients are normalized by dividing by den_coefs[0]",
      "Identifies that bilinear, forward_diff, and backward_diff are wrappers calling gbt with S.Half, S.Zero, and S.One respectively",
      "Explains that DiscreteTransferFunction.from_gbt (and from_bilinear, from_forward_diff, from_backward_diff) consume gbt's output and pass it to from_coeff_lists",
      "Describes that from_coeff_lists reverses the coefficient lists, multiplies by var**i, sums to reconstruct polynomials, and calls the class constructor"
    ],
    "key_files": [
      "sympy/physics/control/lti.py",
      "sympy/physics/control/__init__.py",
      "sympy/core/numbers.py",
      "sympy/physics/control/tests/test_lti.py"
    ],
    "source_doc": "[docstring: sympy/physics/control/lti.py] sympy.physics.control.lti.gbt\nReturns falling coefficients of H(z) from numerator and denominator.\n\n    Parameters\n    ==========\n\n    tf : TransferFunction\n        The continuous transfer function H(s) to be discretized.\n    sample_per : Symbol, Number\n        Time interval between two consecutive sampling instants.\n    alpha: Symbol, Number\n        The parameter for the generalised bilinear transformation method.\n\n    Explanation\n    ===========\n\n    Where H(z) is the corresponding discretized transfer function,\n    discretized with the generalised bilinear transformation method.\n    H(z) is obtained from the continuous transfer function H(s)\n    by substituting $s(z) = \\frac{z-1}{T(\\alpha z + (1-\\alpha))}$ into H(s),\n    where T is the sample period.\n    Coefficients are falling, i.e. $H(z) = \\frac{az+b}{cz+d}$ is returned\n    as [a, b], [c, d].\n\n    Examples\n    ========\n\n    >>> from sympy.physics.control.lti import TransferFunction, gbt\n    >>> from sympy.abc import s, L, R, T\n\n    >>> tf = TransferFunction(1, s*L + R, s)\n    >>> numZ, denZ = gbt(tf, T, 0.5)\n    >>> numZ\n    [T/(2*(L + R*T/2)), T/(2*(L + R*T/2))]\n    >>> denZ\n    [1, (-L + R*T/2)/(L + R*T/2)]\n\n    >>> numZ, denZ = gbt(tf, T, 0)\n    >>> numZ\n    [T/L]\n    >>> denZ\n    [1, (-L + R*T)/L]\n\n    >>> numZ, denZ = gbt(tf, T, 1)\n    >>> numZ\n    [T/(L + R*T), 0]\n    >>> denZ\n    [1, -L/(L + R*T)]\n\n    >>> numZ, denZ = gbt(tf, T, 0.3)\n    >>> numZ\n    [3*T/(10*(L + 3*R*T/10)), 7*T/(10*(L + 3*R*T/10))]\n    >>> denZ\n    [1, (-L + 7*R*T/10)/(L + 3*R*T/10)]\n\n    References\n    ==========\n\n    .. [1] https://www.polyu.edu.hk/ama/profile/gfzhang/Research/ZCC09_IJC.pdf",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about `limit_denominator(1000)` being called on the alpha parameter (line 119) is not mentioned in the documentation and cannot be verified from it \u2014 this is a plausible implementation detail but could be a hallucination.",
      "The claim that `z = s` (reusing tf.var as both s and z) is not mentioned in the documentation \u2014 plausible implementation detail but unverifiable.",
      "The claim about specific line numbers (49, 115, 119, 122, 123-124, 129-131, 171, 209, 247, 819, 2070-2071, etc.) cannot be verified from the documentation.",
      "The claims about `DiscreteTransferFunction.from_gbt`, `from_coeff_lists`, and the various wrapper functions (bilinear, forward_diff, backward_diff) are not mentioned in the provided documentation but are plausible code-level details.",
      "The description of the substitution formula matches the documentation: $s(z) = \\frac{z-1}{T(\\alpha z + (1-\\alpha))}$, though the internal formula described uses `(alpha * z + 1 - alpha)**(N-i)` which is consistent.",
      "The documentation says 'Returns falling coefficients of H(z) from numerator and denominator' which aligns with the answer's description of the output."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the 'Generalized Bilinear Transformation' description and a design rationale comment rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_43",
    "repo": "sympy",
    "question": "Where does `ForceActuator` produce its loads from, and how does that force-to-load pipeline flow through to the equations of motion system?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `ForceActuator` class is located in `sympy/physics/mechanics/actuator.py`. Here is where key functionality resides and how it connects to the broader system:\n\n**Class hierarchy and location:**\n- `ForceActuator` inherits from `ActuatorBase`, which is an ABC with an abstract `to_loads` method.\n- `ForceActuator` is the parent class for `LinearSpring`, `LinearDamper`, `DuffingSpring`, and `CoulombKineticFriction`, all in the same file.\n- In the biomechanics package, `MusculotendonBase` (in `sympy/physics/biomechanics/musculotendon.py`) also inherits from `ForceActuator`.\n\n**Delegation pattern for `to_loads`:**\n- `ForceActuator.to_loads()` simply delegates to `self.pathway.to_loads(self.force)`, passing the force expression to the pathway.\n- For `LinearPathway.to_loads(force)` (in `pathway.py`), it computes the relative position vector between the two attachment points using `_point_pair_relative_position` (which calls `point_2.pos_from(point_1)`), divides by the pathway length, and creates two `Force` namedtuples: one at `attachments[0]` with `-force*relative_position/length` and one at `attachments[-1]` with `force*relative_position/length`.\n\n**Immutability enforcement:**\n- Both `force` and `pathway` properties use `hasattr` checks on private attributes (`_force`, `_pathway`) in their setters to enforce immutability. Once set, attempting to set them again raises `AttributeError` with a message like `\"Can't set attribute `force` to ... as it is immutable.\"`\n- The `force` setter uses `sympify(force, strict=True)` to validate and convert the input.\n- The `pathway` setter checks `isinstance(pathway, PathwayBase)` and raises `TypeError` if the check fails.\n\n**Load types:**\n- `Force` (in `loads.py`) is a namedtuple subclass of `LoadBase` (which itself inherits from both `ABC` and `namedtuple('LoadBase', ['location', 'vector'])`). It has `point` and `force` property aliases for `location` and `vector`. Its `__new__` method validates that location is a `Point` and force is a `Vector`, and also auto-extracts masscenter from `BodyBase` instances.\n- `LoadBase` explicitly blocks `__add__` and `__mul__` operations by raising `TypeError`.\n\n**Integration with System:**\n- In `system.py`, `System.form_eoms()` automatically collects loads from all actuators: `loads = self.loads + tuple(load for act in self.actuators for load in act.to_loads())`. The `actuators` property validates that all items are `ActuatorBase` instances.\n\n**MusculotendonBase workaround for immutability:**\n- `MusculotendonBase.__init__` (in `musculotendon.py`) passes a placeholder `Symbol('F')` to `super().__init__()` then later directly overrides `self._force = -self._F_T` to bypass the immutability check in the setter.\n\n**Helper functions in pathway.py:**\n- `_point_pair_length` computes length as `_point_pair_relative_position(p1, p2).magnitude()`\n- `_point_pair_extension_velocity` computes the time derivative of length: `_point_pair_length(p1, p2).diff(dynamicsymbols._t)`",
    "rubric": [
      "Identifies ForceActuator in the actuator module, inheriting from the abstract ActuatorBase class which defines an abstract to_loads method",
      "Explains that ForceActuator.to_loads() delegates to self.pathway.to_loads(self.force), passing the force expression to the pathway",
      "Describes how LinearPathway.to_loads computes relative position via _point_pair_relative_position (which calls pos_from), divides by length, and creates two Force namedtuples with opposite signs at the two attachment points",
      "Mentions the immutability enforcement pattern: force and pathway setters use hasattr checks on private attributes (_force, _pathway) and raise AttributeError if already set",
      "Explains that Force is a namedtuple subclass of LoadBase (which combines ABC and namedtuple) with point/force property aliases, and its __new__ validates Point and Vector types",
      "Describes System.form_eoms integration: loads are collected as self.loads + tuple(load for act in self.actuators for load in act.to_loads())",
      "Mentions that MusculotendonBase inherits from ForceActuator, passes a placeholder Symbol('F') to super().__init__, then later directly sets self._force = -self._F_T to bypass the immutability check",
      "Identifies subclasses of ForceActuator: LinearSpring, LinearDamper, DuffingSpring, CoulombKineticFriction (and MusculotendonBase in the biomechanics package)"
    ],
    "key_files": [
      "sympy/physics/mechanics/actuator.py",
      "sympy/physics/mechanics/pathway.py",
      "sympy/physics/mechanics/loads.py",
      "sympy/physics/mechanics/system.py",
      "sympy/physics/biomechanics/musculotendon.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/actuator.py] sympy.physics.mechanics.actuator.ForceActuator\nsympy.physics.mechanics.actuator.ForceActuator:\n  Force-producing actuator.\n\n    Explanation\n    ===========\n\n    A ``ForceActuator`` is an actuator that produces a (expansile) force along\n    its length.\n\n    A force actuator uses a pathway instance to determine the direction and\n    number of forces that it applies to a system. Consider the simplest case\n    where a ``LinearPathway`` instance is used. This pathway is made up of two\n    points that can move relative to each other, and results in a pair of equal\n    and opposite forces acting on the endpoints. If the positive time-varying\n    Euclidean distance between the two points is defined, then the \"extension\n    velocity\" is the time derivative of this distance. The extension velocity\n    is positive when the two points are moving away from each other and\n    negative when moving closer to each other. The direction for the force\n    acting on either point is determined by constructing a unit vector directed\n    from the other point to this point. This establishes a sign convention such\n    that a positive force magnitude tends to push the points apart, this is the\n    meaning of \"expansile\" in this context. The following diagram shows the\n    positive force sense and the distance between the points::\n\n       P           Q\n       o<--- F --->o\n       |           |\n       |<--l(t)--->|\n\n    Examples\n    ========\n\n    To construct an actuator, an expression (or symbol) must be supplied to\n    represent the force it can produce, alongside a pathway specifying its line\n    of action. Let's also create a global reference frame and spatially fix one\n    of the points in it while setting the other to be positioned such that it\n    can freely move in the frame's x direction specified by the coordinate\n    ``q``.\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import (ForceActuator, LinearPathway,\n    ...     Point, ReferenceFrame)\n    >>> from sympy.physics.vector import dynamicsymbols\n    >>> N = ReferenceFrame('N')\n    >>> q = dynamicsymbols('q')\n    >>> force = symbols('F')\n    >>> pA, pB = Point('pA'), Point('pB')\n    >>> pA.set_vel(N, 0)\n    >>> pB.set_pos(pA, q*N.x)\n    >>> pB.pos_from(pA)\n    q(t)*N.x\n    >>> linear_pathway = LinearPathway(pA, pB)\n    >>> actuator = ForceActuator(force, linear_pathway)\n    >>> actuator\n    ForceActuator(F, LinearPathway(pA, pB))\n\n    Parameters\n    ==========\n\n    force : Expr\n        The scalar expression defining the (expansile) force that the actuator\n        produces.\n    pathway : PathwayBase\n        The pathway that the actuator follows. This must be an instance of a\n        concrete subclass of ``PathwayBase``, e.g. ``LinearPathway``.\n\nsympy.physics.mechanics.actuator.ForceActuator.__init__:\n  Initializer for ``ForceActuator``.\n\n        Parameters\n        ==========\n\n        force : Expr\n            The scalar expression defining the (expansile) force that the\n            actuator produces.\n        pathway : PathwayBase\n            The pathway that the actuator follows. This must be an instance of\n            a concrete subclass of ``PathwayBase``, e.g. ``LinearPathway``.\n\nsympy.physics.mechanics.actuator.ForceActuator.force:\n  The magnitude of the force produced by the actuator.\n\nsympy.physics.mechanics.actuator.ForceActuator.pathway:\n  The ``Pathway`` defining the actuator's line of action.\n\nsympy.physics.mechanics.actuator.ForceActuator.to_loads:\n  Loads required by the equations of motion method classes.\n\n        Explanation\n        ===========\n\n        ``KanesMethod`` requires a list of ``Point``-``Vector`` tuples to be\n        passed to the ``loads`` parameters of its ``kanes_equations`` method\n        when constructing the equations of motion. This method acts as a\n        utility to produce the correctly-structred pairs of points and vectors\n        required so that these can be easily concatenated with other items in\n        the list of loads and passed to ``KanesMethod.kanes_equations``. These\n        loads are also in the correct form to also be passed to the other\n        equations of motion method classes, e.g. ``LagrangesMethod``.\n\n        Examples\n        ========\n\n        The below example shows how to generate the loads produced by a force\n        actuator that follows a linear pathway. In this example we'll assume\n        that the force actuator is being used to model a simple linear spring.\n        First, create a linear pathway between two points separated by the\n        coordinate ``q`` in the ``x`` direction of the global frame ``N``.\n\n        >>> from sympy.physics.mechanics import (LinearPathway, Point,\n        ...     ReferenceFrame)\n        >>> from sympy.physics.vector import dynamicsymbols\n        >>> q = dynamicsymbols('q')\n        >>> N = ReferenceFrame('N')\n        >>> pA, pB = Point('pA'), Point('pB')\n        >>> pB.set_pos(pA, q*N.x)\n        >>> pathway = LinearPathway(pA, pB)\n\n        Now create a symbol ``k`` to describe the spring's stiffness and\n        instantiate a force actuator that produces a (contractile) force\n        proportional to both the spring's stiffness and the pathway's length.\n        Note that actuator classes use the sign convention that expansile\n        forces are positive, so for a spring to produce a contractile force the\n        spring force needs to be calculated as the negative for the stiffness\n        multiplied by the length.\n\n        >>> from sympy import symbols\n        >>> from sympy.physics.mechanics import ForceActuator\n        >>> stiffness = symbols('k')\n        >>> spring_force = -stiffness*pathway.length\n        >>> spring = ForceActuator(spring_force, pathway)\n\n        The forces produced by the spring can be generated in the list of loads\n        form that ``KanesMethod`` (and other equations of motion methods)\n        requires by calling the ``to_loads`` method.\n\n        >>> spring.to_loads()\n        [(pA, k*q(t)*N.x), (pB, - k*q(t)*N.x)]\n\n        A simple linear damper can be modeled in a similar way. Create another\n        symbol ``c`` to describe the dampers damping coefficient. This time\n        instantiate a force actuator that produces a force proportional to both\n        the damper's damping coefficient and the pathway's extension velocity.\n        Note that the damping force is negative as it acts in the opposite\n        direction to which the damper is changing in length.\n\n        >>> damping_coefficient = symbols('c')\n        >>> damping_force = -damping_coefficient*pathway.extension_velocity\n        >>> damper = ForceActuator(damping_force, pathway)\n\n        Again, the forces produces by the damper can be generated by calling\n        the ``to_loads`` method.\n\n        >>> damper.to_loads()\n        [(pA, c*Derivative(q(t), t)*N.x), (pB, - c*Derivative(q(t), t)*N.x)]\n\nsympy.physics.mechanics.actuator.ForceActuator.__repr__:\n  Representation of a ``ForceActuator``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, class hierarchy, internal attribute names, helper functions, MusculotendonBase workaround, System integration) that are not present in the provided documentation. These cannot be verified from the doc alone but are plausible code-level details.",
      "The answer states that LinearPathway.to_loads creates Force namedtuples with '-force*relative_position/length' at attachments[0] and 'force*relative_position/length' at attachments[-1]. The documentation's example shows spring.to_loads() returns [(pA, k*q(t)*N.x), (pB, -k*q(t)*N.x)] where the spring force is -k*q(t). If the force is -k*q(t) and the relative position is q(t)*N.x with length q(t), then at pA we get -(-k*q(t))*q(t)*N.x/q(t) = k*q(t)*N.x and at pB we get (-k*q(t))*q(t)*N.x/q(t) = -k*q(t)*N.x. This is consistent with the answer's description of '-force*relative_position/length' at attachments[0] and 'force*relative_position/length' at attachments[-1].",
      "The answer's claims about ForceActuator being parent of LinearSpring, LinearDamper, DuffingSpring, CoulombKineticFriction, and MusculotendonBase cannot be verified from the provided documentation but are not contradicted by it.",
      "The immutability enforcement details (hasattr checks, sympify, isinstance checks) are plausible implementation details but not verifiable from the provided documentation."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed to remove purpose/rationale descriptions that would only come from documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_42",
    "repo": "sympy",
    "question": "Why does `kahane_simplify` use virtual indices with negative keys, return `eye(4)` for fully-contracted expressions, and why is `GammaMatrix` created with `comm=None`?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `kahane_simplify` function in `sympy/physics/hep/gamma_matrices.py` implements a graph-based algorithm for reducing contracted products of gamma matrices. Several key design decisions explain why the code is structured as it is:\n\n1. **Why virtual indices with negative positions are used:** When two dummy indices appear consecutively in the expression (`block_free_count == 0`), the code inserts \"virtual indices\" represented by negative integer keys in the `links` dictionary (e.g., `-1-i` and `-1-i+1`). A connected component containing only virtual and dummy indices (i.e., `len(prepend_indices) == 0`) contributes a factor of 2 to `resulting_coeff`.\n\n2. **Why `eye(4)` is returned for fully-contracted expressions:** When all gamma matrices are contracted (no free Lorentz indices remain), the function checks whether the result `t` is a `TensAdd` or `TensMul`; if not, it multiplies by `eye(4)` \u2014 a 4\u00d74 identity matrix \u2014 to represent the identity in the 4-dimensional Dirac spinor space.\n\n3. **Why `kahane_simplify` requires input with only GammaMatrix tensors:** The function asserts `gamma.component == GammaMatrix` for every argument. The separation of gamma matrices from other tensors (like momenta) is handled upstream by `extract_type_tens()` in `_simplify_single_line()`, which splits a mixed expression into a gamma matrix part and a residual part, applies `kahane_simplify` only to the gamma part, and then recombines.\n\n4. **Why `cum_sign` starts at -1:** The cumulative sign alternates based on the parity of free indices between consecutive dummy indices. The sign determines the direction of links in the graph \u2014 whether a dummy index links to its left or right neighbor. The formula `cum_sign *= 1 if (block_free_count % 2) else -1` means the sign stays the same if the preceding block of free indices has odd count, and flips if it has even count (including zero).\n\n5. **Why the sign correction uses `(-1)^(number_of_contractions - connected_components + 1)`:** This is implemented as `resulting_coeff *= -1 if (number_of_contractions - connected_components + 1) % 2 else 1`. Combined with the `2^(number_of_contractions)` power-of-two factor (`resulting_coeff *= 2**(number_of_contractions)`), these determine the overall numerical coefficient of the simplified expression.\n\n6. **Why `GammaMatrix` is created with `comm=None`:** The `comm=None` parameter in `TensorHead(\"GammaMatrix\", [LorentzIndex], TensorSymmetry.no_symmetry(1), comm=None)` means no commutation property is defined for gamma matrices. In `components_canon_args` in `tensor.py`, when `h.comm` is not in `(0, 1)`, the comm value is looked up via `TensorManager.get_comm(h.comm, h.comm)`, which results in different handling than for commuting (`0`) or anticommuting (`1`) tensors. This prevents canonical ordering routines from reordering gamma matrices, preserving their order which is essential for non-commutative algebra.",
    "rubric": [
      "Explains that virtual indices (negative keys in `links`) are inserted when two dummy indices appear consecutively (block_free_count == 0) to satisfy the algorithm's requirement that dummies be separated by free indices",
      "Explains that connected components containing only virtual/dummy indices (no free indices, i.e., prepend_indices is empty) contribute a factor of 2 to the resulting coefficient",
      "Explains that `eye(4)` is returned when the result has no free Lorentz indices (fully contracted), representing the 4\u00d74 identity matrix in Dirac spinor space",
      "Explains that `comm=None` means no commutation property is assigned to GammaMatrix, which prevents canonical ordering routines from reordering gamma matrices since their order matters for non-commutative algebra",
      "Mentions that the sign correction formula `(-1)^(number_of_contractions - connected_components + 1)` combined with `2^(number_of_contractions)` determines the overall numerical coefficient"
    ],
    "key_files": [
      "sympy/physics/hep/gamma_matrices.py",
      "sympy/tensor/tensor.py",
      "sympy/physics/hep/tests/test_gamma_matrices.py"
    ],
    "source_doc": "[docstring: sympy/physics/hep/gamma_matrices.py] sympy.physics.hep.gamma_matrices.kahane_simplify\nThis function cancels contracted elements in a product of four\n    dimensional gamma matrices, resulting in an expression equal to the given\n    one, without the contracted gamma matrices.\n\n    Parameters\n    ==========\n\n    `expression`    the tensor expression containing the gamma matrices to simplify.\n\n    Notes\n    =====\n\n    If spinor indices are given, the matrices must be given in\n    the order given in the product.\n\n    Algorithm\n    =========\n\n    The idea behind the algorithm is to use some well-known identities,\n    i.e., for contractions enclosing an even number of `\\gamma` matrices\n\n    `\\gamma^\\mu \\gamma_{a_1} \\cdots \\gamma_{a_{2N}} \\gamma_\\mu = 2 (\\gamma_{a_{2N}} \\gamma_{a_1} \\cdots \\gamma_{a_{2N-1}} + \\gamma_{a_{2N-1}} \\cdots \\gamma_{a_1} \\gamma_{a_{2N}} )`\n\n    for an odd number of `\\gamma` matrices\n\n    `\\gamma^\\mu \\gamma_{a_1} \\cdots \\gamma_{a_{2N+1}} \\gamma_\\mu = -2 \\gamma_{a_{2N+1}} \\gamma_{a_{2N}} \\cdots \\gamma_{a_{1}}`\n\n    Instead of repeatedly applying these identities to cancel out all contracted indices,\n    it is possible to recognize the links that would result from such an operation,\n    the problem is thus reduced to a simple rearrangement of free gamma matrices.\n\n    Examples\n    ========\n\n    When using, always remember that the original expression coefficient\n    has to be handled separately\n\n    >>> from sympy.physics.hep.gamma_matrices import GammaMatrix as G, LorentzIndex\n    >>> from sympy.physics.hep.gamma_matrices import kahane_simplify\n    >>> from sympy.tensor.tensor import tensor_indices\n    >>> i0, i1, i2 = tensor_indices('i0:3', LorentzIndex)\n    >>> ta = G(i0)*G(-i0)\n    >>> kahane_simplify(ta)\n    Matrix([\n    [4, 0, 0, 0],\n    [0, 4, 0, 0],\n    [0, 0, 4, 0],\n    [0, 0, 0, 4]])\n    >>> tb = G(i0)*G(i1)*G(-i0)\n    >>> kahane_simplify(tb)\n    -2*GammaMatrix(i1)\n    >>> t = G(i0)*G(-i0)\n    >>> kahane_simplify(t)\n    Matrix([\n    [4, 0, 0, 0],\n    [0, 4, 0, 0],\n    [0, 0, 4, 0],\n    [0, 0, 0, 4]])\n    >>> t = G(i0)*G(-i0)\n    >>> kahane_simplify(t)\n    Matrix([\n    [4, 0, 0, 0],\n    [0, 4, 0, 0],\n    [0, 0, 4, 0],\n    [0, 0, 0, 4]])\n\n    If there are no contractions, the same expression is returned\n\n    >>> tc = G(i0)*G(i1)\n    >>> kahane_simplify(tc)\n    GammaMatrix(i0)*GammaMatrix(i1)\n\n    References\n    ==========\n\n    [1] Algorithm for Reducing Contracted Products of gamma Matrices,\n    Joseph Kahane, Journal of Mathematical Physics, Vol. 9, No. 10, October 1968.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., lines 530-541, 683, 708-717, 431-432, 492, 529, 692, 694, 43-44) that cannot be verified from the provided documentation alone \u2014 these are plausible code-level details but are not directly supported by the docstring.",
      "The answer discusses implementation details like virtual indices, `cum_sign`, `links` dictionary, `eye(4)` return behavior, `extract_type_tens()`, `_simplify_single_line()`, and `_TensorManager` \u2014 none of these are mentioned in the provided documentation. These are plausible extrapolations from reading the actual source code but go well beyond the docstring.",
      "The claim about `eye(4)` being returned for fully-contracted expressions is consistent with the docstring examples showing a 4\u00d74 diagonal matrix with 4s on the diagonal (which equals 4*eye(4)).",
      "The answer's description of the algorithm as 'converting the problem into a graph traversal rather than repeatedly applying algebraic identities' is consistent with the docstring's statement that 'it is possible to recognize the links that would result from such an operation, the problem is thus reduced to a simple rearrangement of free gamma matrices'.",
      "The answer correctly identifies the reference to Kahane's 1968 paper, which matches the documentation.",
      "The identities described in the answer (even and odd number of gamma matrices) are consistent with those in the documentation's Algorithm section.",
      "The claim that `comm=None` means gamma matrices 'neither commute nor anticommute' is a code-level detail not verifiable from the docstring alone."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims reference Kahane's 1968 paper and theoretical explanations only found in documentation; code-verifiable algorithmic structure and logic are preserved.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_44",
    "repo": "sympy",
    "question": "How does LinearSpring initialize itself, compute its force, and generate loads without calling its parent class's __init__?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `LinearSpring` class is implemented as a subclass of `ForceActuator` (which inherits from `ActuatorBase`) in `sympy/physics/mechanics/actuator.py`. Here's how the algorithm/implementation works:\n\n**Initialization (`__init__`):**\n`LinearSpring.__init__` does NOT call `super().__init__()` (i.e., `ForceActuator.__init__`). This is because `ForceActuator.__init__` would attempt to set `self.force = force`, which would trigger `LinearSpring`'s force setter that raises `AttributeError('Can\\'t set computed attribute `force`.')`. Instead, `LinearSpring.__init__` directly sets `self.stiffness`, `self.pathway`, and `self.equilibrium_length`. The `pathway` setter is inherited from `ForceActuator`, which validates that the value is an instance of `PathwayBase`.\n\n**Immutability pattern:**\nAll property setters use a `hasattr(self, '_attribute_name')` guard to prevent re-assignment after initial setting. On first set, the private attribute doesn't exist yet so it passes; on subsequent attempts, it raises `AttributeError`. Both `stiffness` and `equilibrium_length` are passed through `sympify(value, strict=True)`, which means passing `None` or a plain string like `'k'` raises `SympifyError`.\n\n**Force computation:**\nThe `force` property is computed dynamically as `-self.stiffness * (self.pathway.length - self.equilibrium_length)`. The pathway's `length` for a `LinearPathway` is computed by `_point_pair_length(point_1, point_2)`, which calls `point_2.pos_from(point_1).magnitude()` (the `.magnitude()` method computes `sqrt(dot(v,v))`, hence why `sqrt(q(t)**2)` appears rather than `abs(q(t))`).\n\n**Load generation (`to_loads`):**\n`LinearSpring` does NOT define its own `to_loads` method. It inherits `ForceActuator.to_loads()`, which simply calls `self.pathway.to_loads(self.force)`. For a `LinearPathway`, this method creates two `Force` namedtuples: one at `attachments[0]` with force `-force * relative_position / length`, and one at `attachments[-1]` with force `force * relative_position / length`. The `Force` class is a namedtuple subclass of `LoadBase` with fields `location` and `vector`, and also exposes `.point` and `.force` properties as aliases.\n\n**`__repr__` conditional logic:**\nThe `__repr__` method conditionally includes `equilibrium_length` in the output. It uses `self.equilibrium_length == S.Zero` (symbolic equality check) to decide whether to omit or include it. When the equilibrium length is zero, output is `LinearSpring(k, LinearPathway(pA, pB))`. Otherwise it's `LinearSpring(k, LinearPathway(pA, pB), equilibrium_length=l)`.\n\n**Default equilibrium length:**\nThe default value for `equilibrium_length` is `S.Zero` (SymPy's singleton zero, imported from `sympy`), not Python's `0`.",
    "rubric": [
      "Explains that LinearSpring.__init__ does NOT call super().__init__() / ForceActuator.__init__ because that would trigger the force setter which raises AttributeError for computed attributes",
      "Describes that LinearSpring.__init__ directly sets self.stiffness, self.pathway, and self.equilibrium_length (with default S.Zero)",
      "Explains the immutability pattern: property setters use hasattr(self, '_attribute_name') guard to prevent re-assignment after first set",
      "Explains that stiffness and equilibrium_length are processed through sympify(value, strict=True)",
      "Describes force as a computed property: -self.stiffness * (self.pathway.length - self.equilibrium_length)",
      "Explains that to_loads is inherited from ForceActuator, which calls self.pathway.to_loads(self.force)",
      "Describes that LinearPathway.to_loads creates two Force objects: one at attachments[0] with -force*relative_position/length, one at attachments[-1] with force*relative_position/length",
      "Mentions that __repr__ conditionally includes equilibrium_length only when it is not equal to S.Zero"
    ],
    "key_files": [
      "sympy/physics/mechanics/actuator.py",
      "sympy/physics/mechanics/pathway.py",
      "sympy/physics/mechanics/loads.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/actuator.py] sympy.physics.mechanics.actuator.LinearSpring\nsympy.physics.mechanics.actuator.LinearSpring:\n  A spring with its spring force as a linear function of its length.\n\n    Explanation\n    ===========\n\n    Note that the \"linear\" in the name ``LinearSpring`` refers to the fact that\n    the spring force is a linear function of the springs length. I.e. for a\n    linear spring with stiffness ``k``, distance between its ends of ``x``, and\n    an equilibrium length of ``0``, the spring force will be ``-k*x``, which is\n    a linear function in ``x``. To create a spring that follows a linear, or\n    straight, pathway between its two ends, a ``LinearPathway`` instance needs\n    to be passed to the ``pathway`` parameter.\n\n    A ``LinearSpring`` is a subclass of ``ForceActuator`` and so follows the\n    same sign conventions for length, extension velocity, and the direction of\n    the forces it applies to its points of attachment on bodies. The sign\n    convention for the direction of forces is such that, for the case where a\n    linear spring is instantiated with a ``LinearPathway`` instance as its\n    pathway, they act to push the two ends of the spring away from one another.\n    Because springs produces a contractile force and acts to pull the two ends\n    together towards the equilibrium length when stretched, the scalar portion\n    of the forces on the endpoint are negative in order to flip the sign of the\n    forces on the endpoints when converted into vector quantities. The\n    following diagram shows the positive force sense and the distance between\n    the points::\n\n       P           Q\n       o<--- F --->o\n       |           |\n       |<--l(t)--->|\n\n    Examples\n    ========\n\n    To construct a linear spring, an expression (or symbol) must be supplied to\n    represent the stiffness (spring constant) of the spring, alongside a\n    pathway specifying its line of action. Let's also create a global reference\n    frame and spatially fix one of the points in it while setting the other to\n    be positioned such that it can freely move in the frame's x direction\n    specified by the coordinate ``q``.\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import (LinearPathway, LinearSpring,\n    ...     Point, ReferenceFrame)\n    >>> from sympy.physics.vector import dynamicsymbols\n    >>> N = ReferenceFrame('N')\n    >>> q = dynamicsymbols('q')\n    >>> stiffness = symbols('k')\n    >>> pA, pB = Point('pA'), Point('pB')\n    >>> pA.set_vel(N, 0)\n    >>> pB.set_pos(pA, q*N.x)\n    >>> pB.pos_from(pA)\n    q(t)*N.x\n    >>> linear_pathway = LinearPathway(pA, pB)\n    >>> spring = LinearSpring(stiffness, linear_pathway)\n    >>> spring\n    LinearSpring(k, LinearPathway(pA, pB))\n\n    This spring will produce a force that is proportional to both its stiffness\n    and the pathway's length. Note that this force is negative as SymPy's sign\n    convention for actuators is that negative forces are contractile.\n\n    >>> spring.force\n    -k*sqrt(q(t)**2)\n\n    To create a linear spring with a non-zero equilibrium length, an expression\n    (or symbol) can be passed to the ``equilibrium_length`` parameter on\n    construction on a ``LinearSpring`` instance. Let's create a symbol ``l``\n    to denote a non-zero equilibrium length and create another linear spring.\n\n    >>> l = symbols('l')\n    >>> spring = LinearSpring(stiffness, linear_pathway, equilibrium_length=l)\n    >>> spring\n    LinearSpring(k, LinearPathway(pA, pB), equilibrium_length=l)\n\n    The spring force of this new spring is again proportional to both its\n    stiffness and the pathway's length. However, the spring will not produce\n    any force when ``q(t)`` equals ``l``. Note that the force will become\n    expansile when ``q(t)`` is less than ``l``, as expected.\n\n    >>> spring.force\n    -k*(-l + sqrt(q(t)**2))\n\n    Parameters\n    ==========\n\n    stiffness : Expr\n        The spring constant.\n    pathway : PathwayBase\n        The pathway that the actuator follows. This must be an instance of a\n        concrete subclass of ``PathwayBase``, e.g. ``LinearPathway``.\n    equilibrium_length : Expr, optional\n        The length at which the spring is in equilibrium, i.e. it produces no\n        force. The default value is 0, i.e. the spring force is a linear\n        function of the pathway's length with no constant offset.\n\n    See Also\n    ========\n\n    ForceActuator: force-producing actuator (superclass of ``LinearSpring``).\n    LinearPathway: straight-line pathway between a pair of points.\n\nsympy.physics.mechanics.actuator.LinearSpring.__init__:\n  Initializer for ``LinearSpring``.\n\n        Parameters\n        ==========\n\n        stiffness : Expr\n            The spring constant.\n        pathway : PathwayBase\n            The pathway that the actuator follows. This must be an instance of\n            a concrete subclass of ``PathwayBase``, e.g. ``LinearPathway``.\n        equilibrium_length : Expr, optional\n            The length at which the spring is in equilibrium, i.e. it produces\n            no force. The default value is 0, i.e. the spring force is a linear\n            function of the pathway's length with no constant offset.\n\nsympy.physics.mechanics.actuator.LinearSpring.force:\n  The spring force produced by the linear spring.\n\nsympy.physics.mechanics.actuator.LinearSpring.stiffness:\n  The spring constant for the linear spring.\n\nsympy.physics.mechanics.actuator.LinearSpring.equilibrium_length:\n  The length of the spring at which it produces no force.\n\nsympy.physics.mechanics.actuator.LinearSpring.__repr__:\n  Representation of a ``LinearSpring``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `LinearSpring.__init__` does NOT call `super().__init__()` is not verifiable from the documentation provided - it's an implementation detail that may be correct but is not documented.",
      "The claim about immutability pattern using `hasattr(self, '_attribute_name')` guards is not verifiable from the documentation - it's an implementation detail beyond the doc scope.",
      "The claim about `sympify(value, strict=True)` being used for stiffness and equilibrium_length is not verifiable from the documentation.",
      "The claim about `to_loads` behavior and `Force` namedtuples is not verifiable from the documentation provided, though it's plausible given the ForceActuator superclass relationship.",
      "The claim about `__repr__` using `self.equilibrium_length == S.Zero` for conditional logic is not directly verifiable but is consistent with the documented output examples.",
      "The claim about `S.Zero` as default is consistent with the documentation stating 'The default value is 0' but adds implementation specificity not in the docs."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only minor rationale wording ('deliberate design choice') was softened.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_45",
    "repo": "sympy",
    "question": "What is the full dependency chain when `linear_momentum` in the mechanics module computes the total linear momentum of a system of bodies?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `linear_momentum` function in `sympy/physics/mechanics/functions.py` is a standalone module-level function that computes the total linear momentum of a system by delegating to instance methods on each body. Here's how the dependency chain works:\n\n1. **Function implementation**: `linear_momentum(frame, *body)` initializes a zero vector via `Vector(0)` (from `sympy.physics.vector`), iterates over each body, checks if it's a `RigidBody` or `Particle` instance, and calls `e.linear_momentum(frame)` on each, accumulating the results via vector addition.\n\n2. **Abstract base**: Both `Particle` and `RigidBody` inherit from `BodyBase` (defined in `sympy/physics/mechanics/body_base.py`), which is an `ABC` (abstract base class). `BodyBase` declares `linear_momentum(self, frame)` as an `@abstractmethod`, along with `kinetic_energy`, `angular_momentum`, and `parallel_axis`.\n\n3. **Particle.linear_momentum**: Returns `self.mass * self.point.vel(frame)`. Notably, `Particle.point` is a class-level alias set as `point = BodyBase.masscenter`, meaning `Particle.point` and `Particle.masscenter` are the same property descriptor.\n\n4. **RigidBody.linear_momentum**: Returns `self.mass * self.masscenter.vel(frame)`. This is the same formula as Particle but uses `masscenter` directly.\n\n5. **BodyBase.__init__**: When `mass` is `None`, it auto-generates a Symbol named `'{name}_mass'`. When `masscenter` is `None`, it auto-creates a `Point(f'{name}_masscenter')`. It also initializes `self.potential_energy = 0` and `self.points = []`.\n\n6. **Point.vel()**: Called by both implementations, `Point.vel(frame)` (in `sympy/physics/vector/point.py`) first checks if the frame is in `self._vel_dict`. If not, it performs a BFS over connected points (via `_pos_dict`) to find the nearest neighbor with a defined velocity, then auto-calculates via `self.pos_from(neighbor).dt(frame) + neighbor_velocity`. It warns about kinematic loops (cyclic graphs) and ambiguous velocity sources.\n\n7. **Vector(0)**: The `Vector.__init__` constructor (in `sympy/physics/vector/vector.py`) treats `inlist == 0` as a special case, converting it to an empty list `[]`, resulting in a zero vector with `self.args = []`.\n\n8. **Error handling**: The function raises `TypeError('Please specify a valid ReferenceFrame')` if `frame` is not a `ReferenceFrame`, and `TypeError('*body must have only Particle or RigidBody')` for invalid body types.\n\n9. **Exports**: The function is listed in the `__all__` of `functions.py`.",
    "rubric": [
      "Explains that `linear_momentum(frame, *body)` initializes a zero vector via `Vector(0)`, iterates over bodies, type-checks for RigidBody/Particle, and calls each body's `linear_momentum(frame)` method",
      "Mentions that both Particle and RigidBody inherit from BodyBase, which is an ABC that declares `linear_momentum` as an @abstractmethod",
      "Describes Particle.linear_momentum returning `self.mass * self.point.vel(frame)` and that `point` is an alias for `BodyBase.masscenter`",
      "Describes RigidBody.linear_momentum returning `self.mass * self.masscenter.vel(frame)`",
      "Explains that BodyBase.__init__ auto-generates a Symbol for mass ('{name}_mass') and a Point for masscenter ('{name}_masscenter') when None is provided",
      "Describes how Point.vel(frame) works: checks _vel_dict first, then does BFS over connected points to auto-calculate velocity via pos_from().dt(frame) + neighbor_velocity",
      "Mentions Vector(0) special case in Vector.__init__ that converts inlist to an empty list, yielding a zero vector with self.args = []",
      "Mentions the TypeError raised for invalid frame or invalid body types"
    ],
    "key_files": [
      "sympy/physics/mechanics/functions.py",
      "sympy/physics/mechanics/particle.py",
      "sympy/physics/mechanics/rigidbody.py",
      "sympy/physics/mechanics/body_base.py",
      "sympy/physics/vector/point.py",
      "sympy/physics/vector/vector.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/functions.py] sympy.physics.mechanics.functions.linear_momentum\nLinear momentum of the system.\n\n    Explanation\n    ===========\n\n    This function returns the linear momentum of a system of Particle's and/or\n    RigidBody's. The linear momentum of a system is equal to the vector sum of\n    the linear momentum of its constituents. Consider a system, S, comprised of\n    a rigid body, A, and a particle, P. The linear momentum of the system, L,\n    is equal to the vector sum of the linear momentum of the particle, L1, and\n    the linear momentum of the rigid body, L2, i.e.\n\n    L = L1 + L2\n\n    Parameters\n    ==========\n\n    frame : ReferenceFrame\n        The frame in which linear momentum is desired.\n    body1, body2, body3... : Particle and/or RigidBody\n        The body (or bodies) whose linear momentum is required.\n\n    Examples\n    ========\n\n    >>> from sympy.physics.mechanics import Point, Particle, ReferenceFrame\n    >>> from sympy.physics.mechanics import RigidBody, outer, linear_momentum\n    >>> N = ReferenceFrame('N')\n    >>> P = Point('P')\n    >>> P.set_vel(N, 10 * N.x)\n    >>> Pa = Particle('Pa', P, 1)\n    >>> Ac = Point('Ac')\n    >>> Ac.set_vel(N, 25 * N.y)\n    >>> I = outer(N.x, N.x)\n    >>> A = RigidBody('A', Ac, N, 20, (I, Ac))\n    >>> linear_momentum(N, A, Pa)\n    10*N.x + 500*N.y",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor line number references and an unverifiable __init__.py export claim needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_47",
    "repo": "sympy",
    "question": "Where is the `inertia` function defined, how does it construct its result, and how does the `Inertia` class relate to it?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `inertia` function is defined in `sympy/physics/mechanics/inertia.py`. It is the canonical location since version 1.13 \u2014 a deprecated wrapper still exists in `sympy/physics/mechanics/functions.py`, which imports the real function as `_inertia` and emits a `sympy_deprecation_warning` with `deprecated_since_version=\"1.13\"` and target `\"moved-mechanics-functions\"` before delegating to it.\n\nThe function builds a symmetric inertia Dyadic by calling `outer()` (from `sympy.physics.vector`) 9 times \u2014 once for each combination of frame unit vectors (xx, xy, xz, yx, yy, yz, zx, zy, zz). The off-diagonal terms ixy and izx are used symmetrically: ixy multiplies both `outer(frame.x, frame.y)` and `outer(frame.y, frame.x)`, and izx multiplies both `outer(frame.x, frame.z)` and `outer(frame.z, frame.x)`. The `outer` function in `sympy/physics/vector/functions.py` is a convenience wrapper that type-checks the first argument is a `Vector` and delegates to `Vector.outer()` (in `sympy/physics/vector/vector.py`), which constructs the result by creating `Dyadic` objects from component pairs.\n\nThe `Dyadic` class (in `sympy/physics/vector/dyadic.py`) stores its data in `self.args` as a list of 3-tuples `(measure_number, unit_vector_1, unit_vector_2)`. Its `__init__` merges entries with matching unit vector pairs and removes zero entries.\n\nThe `Inertia` class in the same file (`inertia.py`) is a `namedtuple` subclass with fields `('dyadic', 'point')` and `__slots__ = ()`. Its `__new__` method auto-swaps arguments if a `Point` is passed first and `Dyadic` second. It blocks `__add__` and `__mul__` with explicit `TypeError`. The class method `from_inertia_scalars` takes `(point, frame, ixx, iyy, izz, ixy=0, iyz=0, izx=0)` and calls the `inertia()` function to create the Dyadic.\n\nThe `inertia` function is used by `RigidBody` (in `rigidbody.py`), whose `inertia` setter uses `inertia_of_point_mass` to compute the offset (`I[0] - I_Ss_O`) to derive `_central_inertia`.",
    "rubric": [
      "Identifies that the canonical `inertia` function is defined in `sympy/physics/mechanics/inertia.py`",
      "Mentions that a deprecated wrapper exists in `sympy/physics/mechanics/functions.py` that emits a deprecation warning (since version 1.13) and delegates to the real function",
      "Explains that `inertia` constructs a symmetric Dyadic by calling `outer()` 9 times for each combination of frame unit vectors (xx, xy, xz, yx, yy, yz, zx, zy, zz)",
      "Notes that off-diagonal terms (ixy, izx, iyz) are used symmetrically \u2014 e.g., ixy multiplies both outer(frame.x, frame.y) and outer(frame.y, frame.x)",
      "Explains that `outer()` in `sympy/physics/vector/functions.py` type-checks the first argument is a Vector and delegates to `Vector.outer()`",
      "Describes how `Dyadic.__init__` stores data as a list of 3-tuples (measure_number, unit_vector_1, unit_vector_2), merging matching pairs and removing zeros",
      "Explains that the `Inertia` class is a namedtuple with fields ('dyadic', 'point') and auto-swaps arguments if Point is passed first",
      "Mentions that `Inertia.from_inertia_scalars` class method calls the `inertia()` function to create the Dyadic"
    ],
    "key_files": [
      "sympy/physics/mechanics/inertia.py",
      "sympy/physics/vector/functions.py",
      "sympy/physics/vector/vector.py",
      "sympy/physics/vector/dyadic.py",
      "sympy/physics/mechanics/functions.py",
      "sympy/physics/mechanics/rigidbody.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/inertia.py] sympy.physics.mechanics.inertia.inertia\nSimple way to create inertia Dyadic object.\n\n    Explanation\n    ===========\n\n    Creates an inertia Dyadic based on the given tensor values and a body-fixed\n    reference frame.\n\n    Parameters\n    ==========\n\n    frame : ReferenceFrame\n        The frame the inertia is defined in.\n    ixx : Sympifyable\n        The xx element in the inertia dyadic.\n    iyy : Sympifyable\n        The yy element in the inertia dyadic.\n    izz : Sympifyable\n        The zz element in the inertia dyadic.\n    ixy : Sympifyable\n        The xy element in the inertia dyadic.\n    iyz : Sympifyable\n        The yz element in the inertia dyadic.\n    izx : Sympifyable\n        The zx element in the inertia dyadic.\n\n    Examples\n    ========\n\n    >>> from sympy.physics.mechanics import ReferenceFrame, inertia\n    >>> N = ReferenceFrame('N')\n    >>> inertia(N, 1, 2, 3)\n    (N.x|N.x) + 2*(N.y|N.y) + 3*(N.z|N.z)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the Particle import claim and __init__.py export claim rely on files not shown.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_46",
    "repo": "sympy",
    "question": "Why does RigidBody eagerly compute central inertia in the inertia setter rather than computing it on demand in angular_momentum?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `angular_momentum` function in `sympy/physics/mechanics/functions.py` delegates computation to the `angular_momentum` method of each body (Particle or RigidBody) and accumulates results via vector addition. The key performance-related design decisions are:\n\n1. **Precomputed central inertia (eager computation on inertia setter):** In `RigidBody`, the `inertia` setter (line 111-124 of rigidbody.py) eagerly computes `_central_inertia` using the parallel axis theorem (`I_S/O - I_S*/O`) at the time inertia is set, rather than recomputing it each time `angular_momentum` is called. This means `RigidBody.angular_momentum()` simply reads `self.central_inertia` (a cached property), avoiding redundant parallel-axis-theorem computation on every call. The formula is `H = I.dot(w) + r.cross(m * v)` where `I` is the precomputed central inertia.\n\n2. **Vector accumulation without `__iadd__`:** The function accumulates `angular_momentum_sys` via `+=` on a `Vector(0)` (line 179-182). Since `Vector` does not define `__iadd__`, each `+=` calls `Vector.__add__`, which creates a new Vector by concatenating the `.args` lists of both operands (`self.args + other.args`). This means for N bodies, N new Vector objects are created. The Vector constructor then merges components by reference frame using a dict, consolidating entries for the same frame. This is simple but creates intermediate objects for each body in the loop.\n\n3. **Polymorphic dispatch via isinstance checks instead of abstract method dispatch:** The `angular_momentum` top-level function checks `isinstance(e, (RigidBody, Particle))` (line 181) for each body, rather than simply calling `e.angular_momentum(point, frame)` directly. While `BodyBase` defines `angular_momentum` as an `@abstractmethod`, the top-level function still performs explicit isinstance checks, adding a defensive TypeError for non-body objects but also adding overhead per body compared to a duck-typing approach.\n\n4. **Different computation paths for Particle vs RigidBody:** `Particle.angular_momentum` computes `cross(r, m*v)` \u2014 a pure cross product of position vector and linear momentum. `RigidBody.angular_momentum` computes `I.dot(w) + r.cross(m*v)` which involves a `Dyadic.dot(Vector)` operation (iterating over dyadic args, computing `v[0] * v[1] * v[2].dot(other)` for each term) plus the cross product. The Dyadic dot product has O(n) complexity in the number of dyadic terms.\n\n5. **No simplification during accumulation:** Unlike `Vector.dot` which optionally applies `trigsimp` when `Vector.simp` is True, the accumulation in `angular_momentum` performs no simplification between additions. Symbolic expressions can grow large across many bodies, with simplification deferred to the caller.",
    "rubric": [
      "Explains that the inertia setter uses the parallel axis theorem (I_S/O - I_S*/O) to precompute _central_inertia at set time",
      "Notes that angular_momentum simply reads self.central_inertia without recomputation, avoiding redundant parallel-axis-theorem calculations on every call",
      "Mentions that this caching strategy trades memory/setter cost for faster repeated angular_momentum evaluations",
      "Describes the angular_momentum formula used: I.dot(w) + r.cross(m * v) where I is the precomputed central inertia",
      "Identifies that Vector accumulation in the top-level angular_momentum function creates intermediate objects since Vector lacks __iadd__"
    ],
    "key_files": [
      "sympy/physics/mechanics/functions.py",
      "sympy/physics/mechanics/rigidbody.py",
      "sympy/physics/mechanics/particle.py",
      "sympy/physics/mechanics/body_base.py",
      "sympy/physics/vector/vector.py",
      "sympy/physics/vector/dyadic.py",
      "sympy/physics/mechanics/inertia.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/functions.py] sympy.physics.mechanics.functions.angular_momentum\nAngular momentum of a system.\n\n    Explanation\n    ===========\n\n    This function returns the angular momentum of a system of Particle's and/or\n    RigidBody's. The angular momentum of such a system is equal to the vector\n    sum of the angular momentum of its constituents. Consider a system, S,\n    comprised of a rigid body, A, and a particle, P. The angular momentum of\n    the system, H, is equal to the vector sum of the angular momentum of the\n    particle, H1, and the angular momentum of the rigid body, H2, i.e.\n\n    H = H1 + H2\n\n    Parameters\n    ==========\n\n    point : Point\n        The point about which angular momentum of the system is desired.\n    frame : ReferenceFrame\n        The frame in which angular momentum is desired.\n    body1, body2, body3... : Particle and/or RigidBody\n        The body (or bodies) whose angular momentum is required.\n\n    Examples\n    ========\n\n    >>> from sympy.physics.mechanics import Point, Particle, ReferenceFrame\n    >>> from sympy.physics.mechanics import RigidBody, outer, angular_momentum\n    >>> N = ReferenceFrame('N')\n    >>> O = Point('O')\n    >>> O.set_vel(N, 0 * N.x)\n    >>> P = O.locatenew('P', 1 * N.x)\n    >>> P.set_vel(N, 10 * N.x)\n    >>> Pa = Particle('Pa', P, 1)\n    >>> Ac = O.locatenew('Ac', 2 * N.y)\n    >>> Ac.set_vel(N, 5 * N.y)\n    >>> a = ReferenceFrame('a')\n    >>> a.set_ang_vel(N, 10 * N.z)\n    >>> I = outer(N.z, N.z)\n    >>> A = RigidBody('A', Ac, a, 20, (I, Ac))\n    >>> angular_momentum(O, N, Pa, A)\n    10*N.z",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims in the answer go far beyond what the documentation covers, referencing specific line numbers, internal implementation details of Vector, Dyadic, RigidBody, and Particle classes that are not present in the provided documentation chunk.",
      "The documentation only describes the function signature, parameters, and a usage example. The answer's claims about eager computation, Vector.__iadd__, isinstance checks, Dyadic.dot complexity, and simplification behavior are all extrapolations from source code not shown in the documentation.",
      "However, none of the claims directly contradict the documentation. The answer correctly reflects that angular_momentum takes a point, frame, and bodies (Particle/RigidBody), and returns the vector sum of their angular momenta, which aligns with the documentation."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_48",
    "repo": "sympy",
    "question": "How does `inertia_of_point_mass` compute the inertia dyadic, and how is it used by `Particle` and `RigidBody`?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `inertia_of_point_mass` function computes the inertia dyadic of a point mass. It is located in `sympy/physics/mechanics/inertia.py`.\n\n**Algorithm:**\nThe function computes: `mass * (I_identity * (pos_vec \u00b7 pos_vec) - outer(pos_vec, pos_vec))`\n\nSpecifically, it constructs the identity dyadic manually as `outer(frame.x, frame.x) + outer(frame.y, frame.y) + outer(frame.z, frame.z)`, multiplies it by the scalar dot product `pos_vec.dot(pos_vec)`, and subtracts the outer product of the position vector with itself. The result is then scaled by `mass`.\n\nNotably, the function does NOT use `frame.u` but instead explicitly constructs the identity dyadic from three `outer()` calls.\n\n**Key implementation details:**\n1. The function does NO type checking on its arguments (unlike the `inertia()` function which raises `TypeError` if frame is not a `ReferenceFrame`).\n2. The `outer` function is imported from `sympy.physics.vector`, and `Vector.outer()` is the core implementation.\n3. `Vector.outer()` builds a `Dyadic` by iterating over all combinations of components from both vectors, creating 9 dyadic terms per frame pair.\n4. `Vector.dot()` computes the dot product using the direction cosine matrix (`dcm`) between frames for cross-frame components, yielding a scalar (via `out += ((v2[0].T) * (v2[1].dcm(v1[1])) * (v1[0]))[0]`).\n5. `Dyadic.__mul__` handles the scalar multiplication by multiplying each component's measure number by the scalar, while the subtraction `__sub__` delegates to `__add__(other * -1)`.\n\n**Usage in the broader system:**\n- `Particle.parallel_axis()` calls `inertia_of_point_mass(self.mass, self.point.pos_from(point), frame)` to compute a particle's inertia about an arbitrary point.\n- `RigidBody.parallel_axis()` uses it as: `self.central_inertia + inertia_of_point_mass(self.mass, self.masscenter.pos_from(point), frame)`.\n- `RigidBody.inertia.setter` uses it to extract the central inertia from a given inertia about any point: `I[0] - I_Ss_O` where `I_Ss_O = inertia_of_point_mass(self.mass, self.masscenter.pos_from(I[1]), self.frame)`.",
    "rubric": [
      "Explains the formula: mass * (identity_dyadic * pos_vec\u00b7pos_vec - outer(pos_vec, pos_vec))",
      "Notes that the identity dyadic is manually constructed from three outer products of frame unit vectors (frame.x, frame.y, frame.z), not using a built-in identity method",
      "Mentions that the function does no type checking on its arguments, unlike the inertia() function which checks that frame is a ReferenceFrame",
      "Explains that Particle.parallel_axis() calls inertia_of_point_mass with the particle's mass and position from the given point",
      "Explains that RigidBody.parallel_axis() adds the central inertia to the result of inertia_of_point_mass",
      "Explains that RigidBody's inertia setter uses inertia_of_point_mass to back-calculate the central inertia by subtracting the point mass contribution"
    ],
    "key_files": [
      "sympy/physics/mechanics/inertia.py",
      "sympy/physics/vector/vector.py",
      "sympy/physics/vector/dyadic.py",
      "sympy/physics/mechanics/particle.py",
      "sympy/physics/mechanics/rigidbody.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/inertia.py] sympy.physics.mechanics.inertia.inertia_of_point_mass\nInertia dyadic of a point mass relative to point O.\n\n    Parameters\n    ==========\n\n    mass : Sympifyable\n        Mass of the point mass\n    pos_vec : Vector\n        Position from point O to point mass\n    frame : ReferenceFrame\n        Reference frame to express the dyadic in\n\n    Examples\n    ========\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import ReferenceFrame, inertia_of_point_mass\n    >>> N = ReferenceFrame('N')\n    >>> r, m = symbols('r m')\n    >>> px = r * N.x\n    >>> inertia_of_point_mass(m, px, N)\n    m*r**2*(N.y|N.y) + m*r**2*(N.z|N.z)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed a claim about a deprecated wrapper in an unshown file and softened a few documentation-dependent phrasings.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_49",
    "repo": "sympy",
    "question": "What is the architecture of the SphericalJoint class, including its inheritance, initialization order, and how it implements the abstract joint interface?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `SphericalJoint` class is part of a six-joint hierarchy in `sympy/physics/mechanics/joint.py`, all inheriting from the abstract base class `Joint`. The full set of joint types exported in `__all__` is: `['Joint', 'PinJoint', 'PrismaticJoint', 'CylindricalJoint', 'PlanarJoint', 'SphericalJoint', 'WeldJoint']`.\n\n**Architecture Details:**\n\n1. **Inheritance**: `SphericalJoint` inherits from `Joint` (an ABC), which defines five abstract methods: `_generate_coordinates`, `_generate_speeds`, `_orient_frames`, `_set_angular_velocity`, and `_set_linear_velocity`.\n\n2. **Constructor ordering**: In `SphericalJoint.__init__`, the `_rot_type`, `_amounts`, and `_rot_order` attributes are set *before* calling `super().__init__()`. This is critical because the parent `Joint.__init__` calls `self._orient_frames()` and `self._set_angular_velocity()` which depend on those attributes. The parent's `__init__` executes operations in this fixed order: validate name/parent/child \u2192 locate interframes \u2192 locate points \u2192 `_generate_coordinates` \u2192 `_generate_speeds` \u2192 `_validate_coordinates` \u2192 `_generate_kdes` \u2192 `_orient_frames` \u2192 `_set_angular_velocity` \u2192 `_set_linear_velocity`.\n\n3. **Coordinate generation**: `_generate_coordinates` always generates exactly 3 coordinates (hardcoded `n_coords=3`), while `_generate_speeds` uses `len(self.coordinates)` to determine how many speeds to generate (also 3).\n\n4. **Angular velocity computation**: `_set_angular_velocity` obtains the angular velocity from `child_interframe.ang_vel_in(parent_interframe)` (which was set by the `orient` call in `_orient_frames`), then replaces time derivatives of generalized coordinates (`q.diff(t)`) with generalized speeds (`u`) using `xreplace`. This substitution is what creates the KDE-consistent velocity expressions.\n\n5. **Linear velocity**: `_set_linear_velocity` sets `child_point.pos_from(parent_point)` to zero (enforcing no translation), sets both parent_point and child_point velocities to zero in their respective body frames, then uses `v2pt_theory` to compute the child masscenter velocity. The `v2pt_theory` method (in `sympy/physics/vector/point.py`) computes `v = otherpoint.vel(outframe) + omega.cross(dist)` where `omega` is `fixedframe.ang_vel_in(outframe)`.\n\n6. **Rotation type validation**: `_orient_frames` only supports `'BODY'` and `'SPACE'` rotation types (checked via `.upper()`). It raises `NotImplementedError` for other types. The actual orientation is delegated to `ReferenceFrame.orient_body_fixed` and `ReferenceFrame.orient_space_fixed` in `sympy/physics/vector/frame.py`.\n\n7. **KDE generation**: The base class `_generate_kdes` creates kinematic differential equations as `-q_i.diff(t) + u_i` for each coordinate-speed pair, meaning if speeds are passed as derivatives of coordinates, the KDEs simplify to zero.\n\n8. **Integration with System**: The `System.add_joints` method extracts joint properties and adds them to the system.\n\n9. **Default parameter**: The `rot_type` parameter default is the string `'BODY'` (all caps), and the default `rot_order` is the integer `123`.\n\n10. **Frame handling for Particles**: In `Joint.__init__`, if a parent body lacks a `frame` attribute (i.e., it's a Particle), the code creates a new `ReferenceFrame` named `f'{self.name}_{self._parent.name}_frame'` unless a `ReferenceFrame` was provided as `parent_interframe`.",
    "rubric": [
      "Identifies that SphericalJoint inherits from the abstract base class Joint, which is an ABC",
      "Lists the five abstract methods that Joint defines: _generate_coordinates, _generate_speeds, _orient_frames, _set_angular_velocity, _set_linear_velocity",
      "Explains that SphericalJoint.__init__ sets _rot_type, _amounts, and _rot_order BEFORE calling super().__init__(), because the parent's __init__ calls _orient_frames and _set_angular_velocity which depend on those attributes",
      "Describes the parent Joint.__init__ execution order: validate name/parent/child \u2192 set interframes \u2192 set points \u2192 _generate_coordinates \u2192 _generate_speeds \u2192 _validate_coordinates \u2192 _generate_kdes \u2192 _orient_frames \u2192 _set_angular_velocity \u2192 _set_linear_velocity",
      "Notes that _generate_coordinates hardcodes 3 coordinates and _generate_speeds uses len(self.coordinates) to generate 3 speeds",
      "Explains that _set_angular_velocity computes angular velocity from child_interframe.ang_vel_in(parent_interframe) and uses xreplace to substitute q.diff(t) with u (generalized speeds)",
      "Explains that _set_linear_velocity sets child_point position to zero relative to parent_point (no translation) and uses v2pt_theory for the child masscenter velocity",
      "Notes that _orient_frames only supports 'BODY' and 'SPACE' rotation types (raises NotImplementedError for others)",
      "Mentions that the default rot_type is 'BODY' and default rot_order is 123",
      "Lists the full set of joint types in __all__: Joint, PinJoint, PrismaticJoint, CylindricalJoint, PlanarJoint, SphericalJoint, WeldJoint"
    ],
    "key_files": [
      "sympy/physics/mechanics/joint.py",
      "sympy/physics/vector/frame.py",
      "sympy/physics/vector/point.py",
      "sympy/physics/mechanics/functions.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/joint.py] sympy.physics.mechanics.joint.SphericalJoint\nsympy.physics.mechanics.joint.SphericalJoint:\n  Spherical (Ball-and-Socket) Joint.\n\n    .. image:: SphericalJoint.svg\n        :align: center\n        :width: 600\n\n    Explanation\n    ===========\n\n    A spherical joint is defined such that the child body is free to rotate in\n    any direction, without allowing a translation of the ``child_point``. As can\n    also be seen in the image, the ``parent_point`` and ``child_point`` are\n    fixed on top of each other, i.e. the ``joint_point``. This rotation is\n    defined using the :func:`parent_interframe.orient(child_interframe,\n    rot_type, amounts, rot_order)\n    <sympy.physics.vector.frame.ReferenceFrame.orient>` method. The default\n    rotation consists of three relative rotations, i.e. body-fixed rotations.\n    Based on the direction cosine matrix following from these rotations, the\n    angular velocity is computed based on the generalized coordinates and\n    generalized speeds.\n\n    Parameters\n    ==========\n\n    name : string\n        A unique name for the joint.\n    parent : Particle or RigidBody\n        The parent body of joint.\n    child : Particle or RigidBody\n        The child body of joint.\n    coordinates: iterable of dynamicsymbols, optional\n        Generalized coordinates of the joint.\n    speeds : iterable of dynamicsymbols, optional\n        Generalized speeds of joint.\n    parent_point : Point or Vector, optional\n        Attachment point where the joint is fixed to the parent body. If a\n        vector is provided, then the attachment point is computed by adding the\n        vector to the body's mass center. The default value is the parent's mass\n        center.\n    child_point : Point or Vector, optional\n        Attachment point where the joint is fixed to the child body. If a\n        vector is provided, then the attachment point is computed by adding the\n        vector to the body's mass center. The default value is the child's mass\n        center.\n    parent_interframe : ReferenceFrame, optional\n        Intermediate frame of the parent body with respect to which the joint\n        transformation is formulated. If a Vector is provided then an interframe\n        is created which aligns its X axis with the given vector. The default\n        value is the parent's own frame.\n    child_interframe : ReferenceFrame, optional\n        Intermediate frame of the child body with respect to which the joint\n        transformation is formulated. If a Vector is provided then an interframe\n        is created which aligns its X axis with the given vector. The default\n        value is the child's own frame.\n    rot_type : str, optional\n        The method used to generate the direction cosine matrix. Supported\n        methods are:\n\n        - ``'Body'``: three successive rotations about new intermediate axes,\n          also called \"Euler and Tait-Bryan angles\"\n        - ``'Space'``: three successive rotations about the parent frames' unit\n          vectors\n\n        The default method is ``'Body'``.\n    amounts :\n        Expressions defining the rotation angles or direction cosine matrix.\n        These must match the ``rot_type``. See examples below for details. The\n        input types are:\n\n        - ``'Body'``: 3-tuple of expressions, symbols, or functions\n        - ``'Space'``: 3-tuple of expressions, symbols, or functions\n\n        The default amounts are the given ``coordinates``.\n    rot_order : str or int, optional\n        If applicable, the order of the successive of rotations. The string\n        ``'123'`` and integer ``123`` are equivalent, for example. Required for\n        ``'Body'`` and ``'Space'``. The default value is ``123``.\n\n    Attributes\n    ==========\n\n    name : string\n        The joint's name.\n    parent : Particle or RigidBody\n        The joint's parent body.\n    child : Particle or RigidBody\n        The joint's child body.\n    coordinates : Matrix\n        Matrix of the joint's generalized coordinates.\n    speeds : Matrix\n        Matrix of the joint's generalized speeds.\n    parent_point : Point\n        Attachment point where the joint is fixed to the parent body.\n    child_point : Point\n        Attachment point where the joint is fixed to the child body.\n    parent_interframe : ReferenceFrame\n        Intermediate frame of the parent body with respect to which the joint\n        transformation is formulated.\n    child_interframe : ReferenceFrame\n        Intermediate frame of the child body with respect to which the joint\n        transformation is formulated.\n    kdes : Matrix\n        Kinematical differential equations of the joint.\n\n    Examples\n    =========\n\n    A single spherical joint is created from two bodies and has the following\n    basic attributes:\n\n    >>> from sympy.physics.mechanics import RigidBody, SphericalJoint\n    >>> parent = RigidBody('P')\n    >>> parent\n    P\n    >>> child = RigidBody('C')\n    >>> child\n    C\n    >>> joint = SphericalJoint('PC', parent, child)\n    >>> joint\n    SphericalJoint: PC  parent: P  child: C\n    >>> joint.name\n    'PC'\n    >>> joint.parent\n    P\n    >>> joint.child\n    C\n    >>> joint.parent_point\n    P_masscenter\n    >>> joint.child_point\n    C_masscenter\n    >>> joint.parent_interframe\n    P_frame\n    >>> joint.child_interframe\n    C_frame\n    >>> joint.coordinates\n    Matrix([\n    [q0_PC(t)],\n    [q1_PC(t)],\n    [q2_PC(t)]])\n    >>> joint.speeds\n    Matrix([\n    [u0_PC(t)],\n    [u1_PC(t)],\n    [u2_PC(t)]])\n    >>> child.frame.ang_vel_in(parent.frame).to_matrix(child.frame)\n    Matrix([\n    [ u0_PC(t)*cos(q1_PC(t))*cos(q2_PC(t)) + u1_PC(t)*sin(q2_PC(t))],\n    [-u0_PC(t)*sin(q2_PC(t))*cos(q1_PC(t)) + u1_PC(t)*cos(q2_PC(t))],\n    [                             u0_PC(t)*sin(q1_PC(t)) + u2_PC(t)]])\n    >>> child.frame.x.to_matrix(parent.frame)\n    Matrix([\n    [                                            cos(q1_PC(t))*cos(q2_PC(t))],\n    [sin(q0_PC(t))*sin(q1_PC(t))*cos(q2_PC(t)) + sin(q2_PC(t))*cos(q0_PC(t))],\n    [sin(q0_PC(t))*sin(q2_PC(t)) - sin(q1_PC(t))*cos(q0_PC(t))*cos(q2_PC(t))]])\n    >>> joint.child_point.pos_from(joint.parent_point)\n    0\n\n    To further demonstrate the use of the spherical joint, the kinematics of a\n    spherical joint with a ZXZ rotation can be created as follows.\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import RigidBody, SphericalJoint\n    >>> l1 = symbols('l1')\n\n    First create bodies to represent the fixed floor and a pendulum bob.\n\n    >>> floor = RigidBody('F')\n    >>> bob = RigidBody('B')\n\n    The joint will connect the bob to the floor, with the joint located at a\n    distance of ``l1`` from the child's center of mass and the rotation set to a\n    body-fixed ZXZ rotation.\n\n    >>> joint = SphericalJoint('S', floor, bob, child_point=l1 * bob.y,\n    ...                        rot_type='body', rot_order='ZXZ')\n\n    Now that the joint is established, the kinematics of the connected body can\n    be accessed.\n\n    The position of the bob's masscenter is found with:\n\n    >>> bob.masscenter.pos_from(floor.masscenter)\n    - l1*B_frame.y\n\n    The angular velocities of the pendulum link can be computed with respect to\n    the floor.\n\n    >>> bob.frame.ang_vel_in(floor.frame).to_matrix(\n    ...     floor.frame).simplify()\n    Matrix([\n    [u1_S(t)*cos(q0_S(t)) + u2_S(t)*sin(q0_S(t))*sin(q1_S(t))],\n    [u1_S(t)*sin(q0_S(t)) - u2_S(t)*sin(q1_S(t))*cos(q0_S(t))],\n    [                          u0_S(t) + u2_S(t)*cos(q1_S(t))]])\n\n    Finally, the linear velocity of the bob's center of mass can be computed.\n\n    >>> bob.masscenter.vel(floor.frame).to_matrix(bob.frame)\n    Matrix([\n    [                           l1*(u0_S(t)*cos(q1_S(t)) + u2_S(t))],\n    [                                                             0],\n    [-l1*(u0_S(t)*sin(q1_S(t))*sin(q2_S(t)) + u1_S(t)*cos(q2_S(t)))]])",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 9 states the default `rot_type` is `'BODY'` (all caps), but the documentation says the default method is `'Body'` (title case). While the code may normalize via `.upper()`, the documented default is `'Body'`.",
      "Claim 9 states the default `rot_order` is the integer `123`, but the documentation says the default value is `123` without specifying it's an integer vs string. The doc does say 'The string `123` and integer `123` are equivalent', and the default is stated as `123`, so calling it 'the integer `123`' is a minor extrapolation.",
      "Claims about the full joint hierarchy (`__all__` list), abstract methods of `Joint`, constructor ordering details, `xreplace` substitution mechanism, `v2pt_theory` internals, `System.add_joints` behavior, and Particle frame handling are all implementation details not present in the provided documentation. These are plausible code-level extrapolations but cannot be verified from the given doc chunk alone."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped source code; only the System.add_joints claim needed minor rewriting since that code wasn't provided.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_52",
    "repo": "sympy",
    "question": "How is the Particle class designed in relation to its base class and RigidBody, and how do their parallel_axis and physics computations differ?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `Particle` class in sympy/physics/mechanics/particle.py is implemented as a subclass of `BodyBase` (an abstract base class defined in `sympy/physics/mechanics/body_base.py` that extends `ABC` from the `abc` module). Here's how the system is designed:\n\n## Inheritance and Property Aliasing\n\n`Particle` inherits from `BodyBase`, which defines abstract methods (`kinetic_energy`, `linear_momentum`, `angular_momentum`, `parallel_axis`) that Particle must implement. The critical design detail is the property alias: `point = BodyBase.masscenter`. This makes `Particle.point` a descriptor that delegates to the `masscenter` property defined on `BodyBase`, meaning `particle.point` and `particle.masscenter` access the same underlying `_masscenter` attribute. This alias allows Particle to use `point` while reusing the parent class's `masscenter` property.\n\n## Default Initialization\n\nWhen no arguments beyond `name` are supplied, `BodyBase.__init__` auto-generates a `Symbol(f'{name}_mass')` for the mass and a `Point(f'{name}_masscenter')` for the position. The `potential_energy` is initialized to `0` (and sympified), and an empty `points` list is initialized.\n\n## Method Implementations\n\n1. **`linear_momentum(frame)`**: Returns `self.mass * self.point.vel(frame)` \u2014 calls the `vel()` method on the Point object.\n\n2. **`angular_momentum(point, frame)`**: Uses the `cross()` function from `sympy.physics.vector` to compute `cross(self.point.pos_from(point), self.mass * self.point.vel(frame))`.\n\n3. **`kinetic_energy(frame)`**: Uses `S.Half` (sympy's rational 1/2) and `dot()` from `sympy.physics.vector`: `S.Half * self.mass * dot(self.point.vel(frame), self.point.vel(frame))`.\n\n4. **`parallel_axis(point, frame)`**: Delegates entirely to `inertia_of_point_mass(self.mass, self.point.pos_from(point), frame)`. This function uses the formula `mass * (I * r\u00b7r - r\u2297r)` where `I` is the identity dyadic constructed from `outer(frame.x, frame.x) + outer(frame.y, frame.y) + outer(frame.z, frame.z)`, `r\u00b7r` is the dot product of the position vector with itself, and `r\u2297r` is the outer product.\n\n## Contrast with RigidBody\n\nBoth `Particle` and `RigidBody` inherit from `BodyBase`, but `RigidBody` additionally has a `frame`, `inertia`, and `central_inertia` property. `RigidBody.parallel_axis` adds the central inertia to the point mass inertia: `self.central_inertia + inertia_of_point_mass(...)`, while `Particle.parallel_axis` has no central inertia term. Also, `RigidBody.parallel_axis` defaults `frame` to `self.frame` if not specified, while `Particle.parallel_axis` requires the `frame` argument.\n\n## Integration with Kane's Method\n\nIn `KanesMethod` (kane.py), Particles are treated specially: only translational partial velocities are computed (`body.point.vel(N)`) as a list of length 1, while RigidBodies produce a list of length 2 (translational + rotational).\n\n## Deprecation\n\nThe `set_potential_energy` method is deprecated since version 1.5, emitting a `sympy_deprecation_warning` and directing users to set the property directly via `P.potential_energy = scalar`.\n\n## Validation\n\n`BodyBase.__init__` raises `TypeError` if `name` is not a string. The `masscenter` setter raises `TypeError` if the point is not a `Point` object. The `mass` setter uses `sympify()` to convert the mass to a SymPy expression.",
    "rubric": [
      "Explains that Particle inherits from BodyBase, an abstract base class using ABC from the abc module",
      "Describes the `point = BodyBase.masscenter` descriptor alias so that particle.point and particle.masscenter refer to the same _masscenter attribute",
      "Explains BodyBase.__init__ auto-generates Symbol(f'{name}_mass') and Point(f'{name}_masscenter') when no mass/point is supplied, and initializes potential_energy to 0",
      "Describes Particle.linear_momentum as mass * self.point.vel(frame)",
      "Describes Particle.angular_momentum as cross(position_from_point, mass * velocity)",
      "Describes Particle.kinetic_energy using S.Half * mass * dot(vel, vel)",
      "Explains Particle.parallel_axis delegates to inertia_of_point_mass, which computes mass*(I*r\u00b7r - r\u2297r) where I is the identity dyadic",
      "Contrasts with RigidBody.parallel_axis which adds central_inertia to the point mass inertia term, and defaults frame to self.frame if not provided",
      "Mentions that KanesMethod treats Particles differently by computing only translational partial velocities (list of length 1) vs RigidBodies (list of length 2)"
    ],
    "key_files": [
      "sympy/physics/mechanics/particle.py",
      "sympy/physics/mechanics/body_base.py",
      "sympy/physics/mechanics/inertia.py",
      "sympy/physics/mechanics/rigidbody.py",
      "sympy/physics/mechanics/kane.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/particle.py] sympy.physics.mechanics.particle.Particle\nsympy.physics.mechanics.particle.Particle:\n  A particle.\n\n    Explanation\n    ===========\n\n    Particles have a non-zero mass and lack spatial extension; they take up no\n    space.\n\n    Values need to be supplied on initialization, but can be changed later.\n\n    Parameters\n    ==========\n\n    name : str\n        Name of particle\n    point : Point\n        A physics/mechanics Point which represents the position, velocity, and\n        acceleration of this Particle\n    mass : Sympifyable\n        A SymPy expression representing the Particle's mass\n    potential_energy : Sympifyable\n        The potential energy of the Particle.\n\n    Examples\n    ========\n\n    >>> from sympy.physics.mechanics import Particle, Point\n    >>> from sympy import Symbol\n    >>> po = Point('po')\n    >>> m = Symbol('m')\n    >>> pa = Particle('pa', po, m)\n    >>> # Or you could change these later\n    >>> pa.mass = m\n    >>> pa.point = po\n\nsympy.physics.mechanics.particle.Particle.linear_momentum:\n  Linear momentum of the particle.\n\n        Explanation\n        ===========\n\n        The linear momentum L, of a particle P, with respect to frame N is\n        given by:\n\n        L = m * v\n\n        where m is the mass of the particle, and v is the velocity of the\n        particle in the frame N.\n\n        Parameters\n        ==========\n\n        frame : ReferenceFrame\n            The frame in which linear momentum is desired.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Particle, Point, ReferenceFrame\n        >>> from sympy.physics.mechanics import dynamicsymbols\n        >>> from sympy.physics.vector import init_vprinting\n        >>> init_vprinting(pretty_print=False)\n        >>> m, v = dynamicsymbols('m v')\n        >>> N = ReferenceFrame('N')\n        >>> P = Point('P')\n        >>> A = Particle('A', P, m)\n        >>> P.set_vel(N, v * N.x)\n        >>> A.linear_momentum(N)\n        m*v*N.x\n\nsympy.physics.mechanics.particle.Particle.angular_momentum:\n  Angular momentum of the particle about the point.\n\n        Explanation\n        ===========\n\n        The angular momentum H, about some point O of a particle, P, is given\n        by:\n\n        ``H = cross(r, m * v)``\n\n        where r is the position vector from point O to the particle P, m is\n        the mass of the particle, and v is the velocity of the particle in\n        the inertial frame, N.\n\n        Parameters\n        ==========\n\n        point : Point\n            The point about which angular momentum of the particle is desired.\n\n        frame : ReferenceFrame\n            The frame in which angular momentum is desired.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Particle, Point, ReferenceFrame\n        >>> from sympy.physics.mechanics import dynamicsymbols\n        >>> from sympy.physics.vector import init_vprinting\n        >>> init_vprinting(pretty_print=False)\n        >>> m, v, r = dynamicsymbols('m v r')\n        >>> N = ReferenceFrame('N')\n        >>> O = Point('O')\n        >>> A = O.locatenew('A', r * N.x)\n        >>> P = Particle('P', A, m)\n        >>> P.point.set_vel(N, v * N.y)\n        >>> P.angular_momentum(O, N)\n        m*r*v*N.z\n\nsympy.physics.mechanics.particle.Particle.kinetic_energy:\n  Kinetic energy of the particle.\n\n        Explanation\n        ===========\n\n        The kinetic energy, T, of a particle, P, is given by:\n\n        ``T = 1/2 (dot(m * v, v))``\n\n        where m is the mass of particle P, and v is the velocity of the\n        particle in the supplied ReferenceFrame.\n\n        Parameters\n        ==========\n\n        frame : ReferenceFrame\n            The Particle's velocity is typically defined with respect to\n            an inertial frame but any relevant frame in which the velocity is\n            known can be supplied.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Particle, Point, ReferenceFrame\n        >>> from sympy import symbols\n        >>> m, v, r = symbols('m v r')\n        >>> N = ReferenceFrame('N')\n        >>> O = Point('O')\n        >>> P = Particle('P', O, m)\n        >>> P.point.set_vel(N, v * N.y)\n        >>> P.kinetic_energy(N)\n        m*v**2/2\n\nsympy.physics.mechanics.particle.Particle.parallel_axis:\n  Returns an inertia dyadic of the particle with respect to another\n        point and frame.\n\n        Parameters\n        ==========\n\n        point : sympy.physics.vector.Point\n            The point to express the inertia dyadic about.\n        frame : sympy.physics.vector.ReferenceFrame\n            The reference frame used to construct the dyadic.\n\n        Returns\n        =======\n\n        inertia : sympy.physics.vector.Dyadic\n            The inertia dyadic of the particle expressed about the provided\n            point and frame.",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many implementation details (BodyBase inheritance, property aliasing, inertia_of_point_mass delegation, KanesMethod integration, deprecation of set_potential_energy, etc.) go beyond what the documentation explicitly states, but these are plausible code-level details rather than contradictions.",
      "The formulas for linear_momentum (L = m * v), angular_momentum (H = cross(r, m * v)), and kinetic_energy (T = 1/2 * dot(m*v, v)) are all consistent with the documentation.",
      "The parallel_axis description is consistent with the documented interface (takes point and frame, returns inertia dyadic).",
      "The claim about Particle parameters (name, point, mass, potential_energy) matches the documentation.",
      "The claim that particles 'have a non-zero mass and lack spatial extension' is consistent with the documentation's explanation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor removals of implementation details about cross/dot delegation and physics rationale; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_50",
    "repo": "sympy",
    "question": "Why does FermionOp enforce constraints like nilpotency and Fock state restrictions that BosonOp doesn't, and how are those constraints implemented?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `FermionOp` class has specific constraints built directly into its implementation. Here's how they work:\n\n1. **`_eval_power` enforces nilpotency (c\u00b2 = 0)**: Unlike `BosonOp` (which has no `_eval_power` method), `FermionOp._eval_power` explicitly returns `S.Zero` for any integer power greater than 1. It also raises a `ValueError` for negative or non-integer exponents (with the message \"Fermionic operators can only be raised to a positive integer power\"). For exponent 0 it returns `S.One`, and for exponent 1 it returns `self`. Otherwise it falls back to `Operator._eval_power`.\n\n2. **Asymmetric anticommutator dispatch pattern**: The `_eval_anticommutator_FermionOp` method only handles the case where `self` is a creation operator (`not self.is_annihilation`) and `other` is an annihilation operator (`other.is_annihilation`), returning `S.One`. For the reverse order (annihilation, creation), it returns `None`, which causes a `NotImplementedError` to be raised. The `AntiCommutator.doit()` method then catches this and swaps the arguments (tries `B._eval_anticommutator(A)` as a fallback).\n\n3. **`independent` hints mechanism for multi-mode operators**: Both `_eval_commutator_FermionOp` and `_eval_anticommutator_FermionOp` check for an `'independent'` keyword hint. When two fermionic operators have different names and aren't explicitly marked independent, the methods return `None` (unknown), leaving the result unevaluated.\n\n4. **Cross-species commutativity**: `_eval_anticommutator_BosonOp` returns `2 * self * other` and `_eval_commutator_BosonOp` returns `S.Zero`. The corresponding `BosonOp._eval_commutator_FermionOp` also returns `S.Zero`, providing the relation from both sides.\n\n5. **Fock state restriction to {0, 1}**: `FermionFockKet.__new__` explicitly validates that `n not in (0, 1)` raises ValueError, while `BosonFockKet` has no such constraint. Additionally, `FermionFockKet._eval_hilbert_space` returns a generic `HilbertSpace()` rather than `FockSpace()` (which is used by `BosonFockKet`).\n\n6. **`annihilation` parameter stored as Integer**: In `__new__`, the boolean annihilation flag is converted to `Integer(args[1])` (via `S.One` if omitted). The `is_annihilation` property uses `bool(self.args[1])` to recover the boolean meaning.",
    "rubric": [
      "Explains that _eval_power returns S.Zero for integer exponents > 1 (nilpotency c\u00b2=0), while BosonOp has no _eval_power method",
      "Mentions that _eval_power raises ValueError for negative or non-integer exponents",
      "Describes the asymmetric anticommutator dispatch: _eval_anticommutator_FermionOp only handles creation\u2020\u00b7annihilation case (returns S.One), returning None otherwise, relying on AntiCommutator.doit() to swap and retry",
      "Explains the 'independent' hints mechanism for multi-mode operators in both commutator and anticommutator methods",
      "Notes cross-species commutativity: FermionOp._eval_commutator_BosonOp returns S.Zero and BosonOp._eval_commutator_FermionOp also returns S.Zero",
      "Explains that FermionFockKet restricts n to {0, 1} with a ValueError, while BosonFockKet has no such restriction",
      "Mentions that annihilation flag is stored as Integer (S.One) and recovered via bool(self.args[1])"
    ],
    "key_files": [
      "sympy/physics/quantum/fermion.py",
      "sympy/physics/quantum/anticommutator.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/boson.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/fermion.py] sympy.physics.quantum.fermion.FermionOp\nsympy.physics.quantum.fermion.FermionOp:\n  A fermionic operator that satisfies {c, Dagger(c)} == 1.\n\n    Parameters\n    ==========\n\n    name : str\n        A string that labels the fermionic mode.\n\n    annihilation : bool\n        A bool that indicates if the fermionic operator is an annihilation\n        (True, default value) or creation operator (False)\n\n    Examples\n    ========\n\n    >>> from sympy.physics.quantum import Dagger, AntiCommutator\n    >>> from sympy.physics.quantum.fermion import FermionOp\n    >>> c = FermionOp(\"c\")\n    >>> AntiCommutator(c, Dagger(c)).doit()\n    1",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed physical rationale/design motivation claims not present in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_51",
    "repo": "sympy",
    "question": "Where does data flow through JointsMethod, from joint inputs to equation-of-motion formation and result delegation?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data flow in `JointsMethod` follows this path:\n\n1. **Input**: Joints are passed to `__init__`, and an inertial frame is extracted from either a `BodyBase` instance (via `newtonion.frame`) or used directly as a `ReferenceFrame`.\n\n2. **Body extraction** (`_generate_bodylist`): Iterates through joints, adding `joint.child` first, then `joint.parent` to the bodies list (child comes before parent in ordering). Bodies are deduplicated by identity (`not in`).\n\n3. **Load collection** (`_generate_loadlist`): Only collects loads from objects that are instances of the `Body` class (accessing `body.loads` which returns `self._loads`, initialized as an empty list `[]` in `Body.__init__`).\n\n4. **Coordinate/speed extraction** (`_generate_q`, `_generate_u`): Raises `ValueError` with messages 'Coordinates of joints should be unique.' or 'Speeds of joints should be unique.' if duplicates are found.\n\n5. **KDE generation** (`_generate_kdes`): Starts with an empty column vector `Matrix(1, 0, []).T` and col_joins each joint's `kdes` property.\n\n6. **`form_eoms` dispatch**: Defaults to `KanesMethod` if no method argument is provided. Uses `issubclass(method, LagrangesMethod)` to determine the branch:\n   - **LagrangesMethod path**: Calls `Lagrangian(self.frame, *bodylist)`, then passes the result positionally as `LagrangesMethod(L, self.q, self.loads, bodylist, self.frame)`.\n   - **KanesMethod path**: Calls `KanesMethod(self.frame, q_ind=self.q, u_ind=self.u, kd_eqs=self.kdes, forcelist=self.loads, bodies=bodylist)`.\n\n7. **Body conversion** (`_convert_bodies`): Before passing to either method, converts `Body` objects to `RigidBody` or `Particle` based on `body.is_rigidbody` (which checks `hasattr(self, '_inertia')`). Transfers `potential_energy` to the new object.\n\n8. **EOM formation**: Calls `self.method._form_eoms()` which:\n   - For `LagrangesMethod`: calls `form_lagranges_equations()` \u2192 returns `self.eom`\n\n9. **Results delegation**: Properties `mass_matrix`, `forcing`, `mass_matrix_full`, `forcing_full` all delegate directly to the underlying method object.\n\n10. **`rhs` delegation**: `JointsMethod.rhs(inv_method=None)` directly calls `self.method.rhs(inv_method=inv_method)`. In `LagrangesMethod.rhs`, when `inv_method` is None, it uses `self.mass_matrix_full.LUsolve(self.forcing_full)`.\n\n`JointsMethod` inherits from `_Methods` (in `sympy/physics/mechanics/method.py`), which is an ABC with abstract methods for `q`, `u`, `bodies`, `loads`, `mass_matrix`, `forcing`, `mass_matrix_full`, `forcing_full`, and a non-abstract `_form_eoms` that raises `NotImplementedError`. Both `KanesMethod` and `LagrangesMethod` also inherit from `_Methods`.",
    "rubric": [
      "Explains that __init__ extracts the inertial frame from a BodyBase (via newtonion.frame) or uses a ReferenceFrame directly",
      "Describes _generate_bodylist adding joint.child before joint.parent, deduplicating by identity (not in)",
      "Notes that _generate_loadlist only collects loads from Body instances (not all body types)",
      "Mentions that _generate_q and _generate_u raise ValueError if duplicate coordinates or speeds are found",
      "Describes _generate_kdes building a column vector starting from Matrix(1,0,[]).T and col_joining each joint's kdes",
      "Explains form_eoms defaults to KanesMethod, and uses issubclass to branch between LagrangesMethod and KanesMethod paths",
      "For LagrangesMethod path: computes Lagrangian then creates LagrangesMethod(L, q, loads, bodylist, frame)",
      "For KanesMethod path: creates KanesMethod with frame, q_ind, u_ind, kd_eqs, forcelist, bodies as keyword arguments",
      "Describes _convert_bodies converting Body objects to RigidBody or Particle based on is_rigidbody (which checks hasattr(self, '_inertia')), transferring potential_energy",
      "Notes that mass_matrix, forcing, mass_matrix_full, forcing_full, and rhs all delegate to the underlying method object",
      "Mentions JointsMethod inherits from _Methods ABC, which KanesMethod and LagrangesMethod also inherit from"
    ],
    "key_files": [
      "sympy/physics/mechanics/jointsmethod.py",
      "sympy/physics/mechanics/method.py",
      "sympy/physics/mechanics/kane.py",
      "sympy/physics/mechanics/lagrange.py",
      "sympy/physics/mechanics/body.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/jointsmethod.py] sympy.physics.mechanics.jointsmethod.JointsMethod\nsympy.physics.mechanics.jointsmethod.JointsMethod:\n  Method for formulating the equations of motion using a set of interconnected bodies with joints.\n\n    .. deprecated:: 1.13\n        The JointsMethod class is deprecated. Its functionality has been\n        replaced by the new :class:`~.System` class.\n\n    Parameters\n    ==========\n\n    newtonion : Body or ReferenceFrame\n        The newtonion(inertial) frame.\n    *joints : Joint\n        The joints in the system\n\n    Attributes\n    ==========\n\n    q, u : iterable\n        Iterable of the generalized coordinates and speeds\n    bodies : iterable\n        Iterable of Body objects in the system.\n    loads : iterable\n        Iterable of (Point, vector) or (ReferenceFrame, vector) tuples\n        describing the forces on the system.\n    mass_matrix : Matrix, shape(n, n)\n        The system's mass matrix\n    forcing : Matrix, shape(n, 1)\n        The system's forcing vector\n    mass_matrix_full : Matrix, shape(2*n, 2*n)\n        The \"mass matrix\" for the u's and q's\n    forcing_full : Matrix, shape(2*n, 1)\n        The \"forcing vector\" for the u's and q's\n    method : KanesMethod or Lagrange's method\n        Method's object.\n    kdes : iterable\n        Iterable of kde in they system.\n\n    Examples\n    ========\n\n    As Body and JointsMethod have been deprecated, the following examples are\n    for illustrative purposes only. The functionality of Body is fully captured\n    by :class:`~.RigidBody` and :class:`~.Particle` and the functionality of\n    JointsMethod is fully captured by :class:`~.System`. To ignore the\n    deprecation warning we can use the ignore_warnings context manager.\n\n    >>> from sympy.utilities.exceptions import ignore_warnings\n\n    This is a simple example for a one degree of freedom translational\n    spring-mass-damper.\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import Body, JointsMethod, PrismaticJoint\n    >>> from sympy.physics.vector import dynamicsymbols\n    >>> c, k = symbols('c k')\n    >>> x, v = dynamicsymbols('x v')\n    >>> with ignore_warnings(DeprecationWarning):\n    ...     wall = Body('W')\n    ...     body = Body('B')\n    >>> J = PrismaticJoint('J', wall, body, coordinates=x, speeds=v)\n    >>> wall.apply_force(c*v*wall.x, reaction_body=body)\n    >>> wall.apply_force(k*x*wall.x, reaction_body=body)\n    >>> with ignore_warnings(DeprecationWarning):\n    ...     method = JointsMethod(wall, J)\n    >>> method.form_eoms()\n    Matrix([[-B_mass*Derivative(v(t), t) - c*v(t) - k*x(t)]])\n    >>> M = method.mass_matrix_full\n    >>> F = method.forcing_full\n    >>> rhs = M.LUsolve(F)\n    >>> rhs\n    Matrix([\n    [                     v(t)],\n    [(-c*v(t) - k*x(t))/B_mass]])\n\n    Notes\n    =====\n\n    ``JointsMethod`` currently only works with systems that do not have any\n    configuration or motion constraints.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.bodies:\n  List of bodies in they system.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.loads:\n  List of loads on the system.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.q:\n  List of the generalized coordinates.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.u:\n  List of the generalized speeds.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.kdes:\n  List of the generalized coordinates.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.forcing_full:\n  The \"forcing vector\" for the u's and q's.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.mass_matrix_full:\n  The \"mass matrix\" for the u's and q's.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.mass_matrix:\n  The system's mass matrix.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.forcing:\n  The system's forcing vector.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.method:\n  Object of method used to form equations of systems.\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.form_eoms:\n  Method to form system's equation of motions.\n\n        Parameters\n        ==========\n\n        method : Class\n            Class name of method.\n\n        Returns\n        ========\n\n        Matrix\n            Vector of equations of motions.\n\n        Examples\n        ========\n\n        As Body and JointsMethod have been deprecated, the following examples\n        are for illustrative purposes only. The functionality of Body is fully\n        captured by :class:`~.RigidBody` and :class:`~.Particle` and the\n        functionality of JointsMethod is fully captured by :class:`~.System`. To\n        ignore the deprecation warning we can use the ignore_warnings context\n        manager.\n\n        >>> from sympy.utilities.exceptions import ignore_warnings\n\n        This is a simple example for a one degree of freedom translational\n        spring-mass-damper.\n\n        >>> from sympy import S, symbols\n        >>> from sympy.physics.mechanics import LagrangesMethod, dynamicsymbols, Body\n        >>> from sympy.physics.mechanics import PrismaticJoint, JointsMethod\n        >>> q = dynamicsymbols('q')\n        >>> qd = dynamicsymbols('q', 1)\n        >>> m, k, b = symbols('m k b')\n        >>> with ignore_warnings(DeprecationWarning):\n        ...     wall = Body('W')\n        ...     part = Body('P', mass=m)\n        >>> part.potential_energy = k * q**2 / S(2)\n        >>> J = PrismaticJoint('J', wall, part, coordinates=q, speeds=qd)\n        >>> wall.apply_force(b * qd * wall.x, reaction_body=part)\n        >>> with ignore_warnings(DeprecationWarning):\n        ...     method = JointsMethod(wall, J)\n        >>> method.form_eoms(LagrangesMethod)\n        Matrix([[b*Derivative(q(t), t) + k*q(t) + m*Derivative(q(t), (t, 2))]])\n\n        We can also solve for the states using the 'rhs' method.\n\n        >>> method.rhs()\n        Matrix([\n        [                Derivative(q(t), t)],\n        [(-b*Derivative(q(t), t) - k*q(t))/m]])\n\nsympy.physics.mechanics.jointsmethod.JointsMethod.rhs:\n  Returns equations that can be solved numerically.\n\n        Parameters\n        ==========\n\n        inv_method : str\n            The specific sympy inverse matrix calculation method to use. For a\n            list of valid methods, see\n            :meth:`~sympy.matrices.matrixbase.MatrixBase.inv`\n\n        Returns\n        ========\n\n        Matrix\n            Numerically solvable equations.\n\n        See Also\n        ========\n\n        sympy.physics.mechanics.kane.KanesMethod.rhs:\n            KanesMethod's rhs function.\n        sympy.physics.mechanics.lagrange.LagrangesMethod.rhs:\n            LagrangesMethod's rhs function.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many implementation-level details (e.g., `_generate_bodylist` ordering, `_generate_loadlist` filtering by Body class, `_generate_kdes` using col_join, `_convert_bodies` logic, `_Methods` ABC inheritance) that are not present in the documentation. These are plausible code-level details but cannot be verified from the provided documentation alone.",
      "The claim about `form_eoms` calling `LagrangesMethod(L, self.q, self.loads, bodylist, self.frame)` positionally cannot be verified from the documentation, though the documentation does show LagrangesMethod being used.",
      "The claim about KanesMethod path using `kd_eqs=self.kdes` is plausible given the documentation mentions kdes attribute but cannot be directly verified.",
      "The documentation says the parameter is 'newtonion' (sic) and describes it as 'Body or ReferenceFrame'. The answer says 'an inertial frame is extracted from either a BodyBase instance (via newtonion.frame) or used directly as a ReferenceFrame' which is a plausible implementation detail.",
      "The answer's claim about `rhs` delegation is consistent with the documentation which shows `method.rhs()` working and references KanesMethod.rhs and LagrangesMethod.rhs in the See Also section.",
      "The documentation for `kdes` property says 'List of the generalized coordinates' which appears to be a documentation bug (should say KDEs), but the answer describes KDE generation which is consistent with the attribute name and class description."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor adjustments needed for truncated KanesMethod details and documentation-only deprecation labels.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_53",
    "repo": "sympy",
    "question": "What is the design and class hierarchy of LinearPathway, including how it computes length, extension velocity, and loads?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `LinearPathway` class in `sympy/physics/mechanics/pathway.py` is a concrete subclass of the abstract base class `PathwayBase`. It has two attachment points and provides methods to compute length, extension velocity, and loads.\n\n**Architecture and Key Design Details:**\n\n1. **Class Hierarchy:** `LinearPathway` inherits from `PathwayBase`, which is an abstract base class (ABC) defined in the same module. `PathwayBase` defines abstract properties `length` and `extension_velocity`, and an abstract method `to_loads(force)`. There are three concrete pathway subclasses: `LinearPathway`, `ObstacleSetPathway`, and `WrappingPathway`.\n\n2. **Immutability Pattern:** The `attachments` property in `PathwayBase` is made immutable by checking `hasattr(self, '_attachments')` in its setter \u2014 if already set, it raises `AttributeError`. The stored value is a tuple (not list), so individual elements also cannot be reassigned.\n\n3. **Internal Helper Functions:** `LinearPathway` delegates its computations to three module-level private helper functions:\n   - `_point_pair_relative_position(point_1, point_2)` \u2014 simply calls `point_2.pos_from(point_1)` returning a Vector\n   - `_point_pair_length(point_1, point_2)` \u2014 calls `_point_pair_relative_position(point_1, point_2).magnitude()`, where `Vector.magnitude()` returns `sqrt(self.dot(self))`\n   - `_point_pair_extension_velocity(point_1, point_2)` \u2014 calls `_point_pair_length(point_1, point_2).diff(dynamicsymbols._t)`\n\n4. **Load Generation:** The `to_loads` method returns a list of `Force` objects (not plain tuples). `Force` is a subclass of `LoadBase`, which itself inherits from both `ABC` and `namedtuple('LoadBase', ['location', 'vector'])`. The `Force` class has `point` and `force` properties as aliases for `location` and `vector` respectively. In `to_loads`, the force on `attachments[0]` is `-force * relative_position / self.length` and on `attachments[-1]` is `force * relative_position / self.length`.\n\n5. **Usage by Actuators:** `ForceActuator.to_loads()` in `sympy/physics/mechanics/actuator.py` directly delegates to `self.pathway.to_loads(self.force)`. Subclasses like `LinearSpring` compute force as `-self.stiffness*(self.pathway.length - self.equilibrium_length)` and `LinearDamper` computes force as `-self.damping*self.pathway.extension_velocity`.\n\n6. **Validation:** The `PathwayBase.attachments` setter validates that exactly 2 points are passed (raising `ValueError` for wrong count), and each must be a `Point` instance (raising `TypeError` otherwise). `ObstacleSetPathway` overrides this to require 3+ points.\n\n7. **Difference from `WrappingPathway`:** `LinearPathway` uses the helper functions for length/extension_velocity computation, while `WrappingPathway` gets its length from `self.geometry.geodesic_length(*self.attachments)` and its extension_velocity by differentiating the length expression: `self.length.diff(dynamicsymbols._t)`. Additionally, `WrappingPathway.to_loads` produces 3 force loads (on both attachment points and the geometry's center point), not just 2.",
    "rubric": [
      "LinearPathway inherits from PathwayBase, which is an abstract base class (ABC) with abstract properties length, extension_velocity, and abstract method to_loads",
      "Three concrete pathway subclasses exist: LinearPathway, ObstacleSetPathway, and WrappingPathway",
      "Attachments are made immutable via a hasattr check on _attachments in the setter, raising AttributeError if already set, and stored as a tuple",
      "PathwayBase validates exactly 2 Point instances are passed (ObstacleSetPathway overrides to require 3+)",
      "LinearPathway delegates to module-level private helpers: _point_pair_relative_position (calls point_2.pos_from(point_1)), _point_pair_length (calls magnitude on that), and _point_pair_extension_velocity (diffs length by dynamicsymbols._t)",
      "to_loads returns a list of Force objects (Force is a namedtuple subclass via LoadBase with point/force aliases for location/vector)",
      "In to_loads, attachments[0] gets force -force*relative_position/length and attachments[-1] gets force*relative_position/length",
      "ForceActuator.to_loads delegates directly to self.pathway.to_loads(self.force); LinearSpring computes force as -stiffness*(length - equilibrium_length); LinearDamper computes force as -damping*extension_velocity",
      "WrappingPathway differs by using geometry.geodesic_length for length, differentiating it for extension_velocity, and producing 3 Force loads (on both attachment points and the geometry's center point)"
    ],
    "key_files": [
      "sympy/physics/mechanics/pathway.py",
      "sympy/physics/mechanics/loads.py",
      "sympy/physics/mechanics/actuator.py",
      "sympy/physics/vector/vector.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/pathway.py] sympy.physics.mechanics.pathway.LinearPathway\nsympy.physics.mechanics.pathway.LinearPathway:\n  Linear pathway between a pair of attachment points.\n\n    Explanation\n    ===========\n\n    A linear pathway forms a straight-line segment between two points and is\n    the simplest pathway that can be formed. It will not interact with any\n    other objects in the system, i.e. a ``LinearPathway`` will intersect other\n    objects to ensure that the path between its two ends (its attachments) is\n    the shortest possible.\n\n    A linear pathway is made up of two points that can move relative to each\n    other, and a pair of equal and opposite forces acting on the points. If the\n    positive time-varying Euclidean distance between the two points is defined,\n    then the \"extension velocity\" is the time derivative of this distance. The\n    extension velocity is positive when the two points are moving away from\n    each other and negative when moving closer to each other. The direction for\n    the force acting on either point is determined by constructing a unit\n    vector directed from the other point to this point. This establishes a sign\n    convention such that a positive force magnitude tends to push the points\n    apart. The following diagram shows the positive force sense and the\n    distance between the points::\n\n       P           Q\n       o<--- F --->o\n       |           |\n       |<--l(t)--->|\n\n    Examples\n    ========\n\n    >>> from sympy.physics.mechanics import LinearPathway\n\n    To construct a pathway, two points are required to be passed to the\n    ``attachments`` parameter as a ``tuple``.\n\n    >>> from sympy.physics.mechanics import Point\n    >>> pA, pB = Point('pA'), Point('pB')\n    >>> linear_pathway = LinearPathway(pA, pB)\n    >>> linear_pathway\n    LinearPathway(pA, pB)\n\n    The pathway created above isn't very interesting without the positions and\n    velocities of its attachment points being described. Without this its not\n    possible to describe how the pathway moves, i.e. its length or its\n    extension velocity.\n\n    >>> from sympy.physics.mechanics import ReferenceFrame\n    >>> from sympy.physics.vector import dynamicsymbols\n    >>> N = ReferenceFrame('N')\n    >>> q = dynamicsymbols('q')\n    >>> pB.set_pos(pA, q*N.x)\n    >>> pB.pos_from(pA)\n    q(t)*N.x\n\n    A pathway's length can be accessed via its ``length`` attribute.\n\n    >>> linear_pathway.length\n    sqrt(q(t)**2)\n\n    Note how what appears to be an overly-complex expression is returned. This\n    is actually required as it ensures that a pathway's length is always\n    positive.\n\n    A pathway's extension velocity can be accessed similarly via its\n    ``extension_velocity`` attribute.\n\n    >>> linear_pathway.extension_velocity\n    sqrt(q(t)**2)*Derivative(q(t), t)/q(t)\n\n    Parameters\n    ==========\n\n    attachments : tuple[Point, Point]\n        Pair of ``Point`` objects between which the linear pathway spans.\n        Constructor expects two points to be passed, e.g.\n        ``LinearPathway(Point('pA'), Point('pB'))``. More or fewer points will\n        cause an error to be thrown.\n\nsympy.physics.mechanics.pathway.LinearPathway.__init__:\n  Initializer for ``LinearPathway``.\n\n        Parameters\n        ==========\n\n        attachments : Point\n            Pair of ``Point`` objects between which the linear pathway spans.\n            Constructor expects two points to be passed, e.g.\n            ``LinearPathway(Point('pA'), Point('pB'))``. More or fewer points\n            will cause an error to be thrown.\n\nsympy.physics.mechanics.pathway.LinearPathway.length:\n  Exact analytical expression for the pathway's length.\n\nsympy.physics.mechanics.pathway.LinearPathway.extension_velocity:\n  Exact analytical expression for the pathway's extension velocity.\n\nsympy.physics.mechanics.pathway.LinearPathway.to_loads:\n  Loads required by the equations of motion method classes.\n\n        Explanation\n        ===========\n\n        ``KanesMethod`` requires a list of ``Point``-``Vector`` tuples to be\n        passed to the ``loads`` parameters of its ``kanes_equations`` method\n        when constructing the equations of motion. This method acts as a\n        utility to produce the correctly-structred pairs of points and vectors\n        required so that these can be easily concatenated with other items in\n        the list of loads and passed to ``KanesMethod.kanes_equations``. These\n        loads are also in the correct form to also be passed to the other\n        equations of motion method classes, e.g. ``LagrangesMethod``.\n\n        Examples\n        ========\n\n        The below example shows how to generate the loads produced in a linear\n        actuator that produces an expansile force ``F``. First, create a linear\n        actuator between two points separated by the coordinate ``q`` in the\n        ``x`` direction of the global frame ``N``.\n\n        >>> from sympy.physics.mechanics import (LinearPathway, Point,\n        ...     ReferenceFrame)\n        >>> from sympy.physics.vector import dynamicsymbols\n        >>> q = dynamicsymbols('q')\n        >>> N = ReferenceFrame('N')\n        >>> pA, pB = Point('pA'), Point('pB')\n        >>> pB.set_pos(pA, q*N.x)\n        >>> linear_pathway = LinearPathway(pA, pB)\n\n        Now create a symbol ``F`` to describe the magnitude of the (expansile)\n        force that will be produced along the pathway. The list of loads that\n        ``KanesMethod`` requires can be produced by calling the pathway's\n        ``to_loads`` method with ``F`` passed as the only argument.\n\n        >>> from sympy import symbols\n        >>> F = symbols('F')\n        >>> linear_pathway.to_loads(F)\n        [(pA, - F*q(t)/sqrt(q(t)**2)*N.x), (pB, F*q(t)/sqrt(q(t)**2)*N.x)]\n\n        Parameters\n        ==========\n\n        force : Expr\n            Magnitude of the force acting along the length of the pathway. As\n            per the sign conventions for the pathway length, pathway extension\n            velocity, and pair of point forces, if this ``Expr`` is positive\n            then the force will act to push the pair of points away from one\n            another (it is expansile).",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `to_loads` returns `Force` objects rather than tuples is not directly supported by the documentation. The documentation shows the output as tuples like `[(pA, - F*q(t)/sqrt(q(t)**2)*N.x), (pB, F*q(t)/sqrt(q(t)**2)*N.x)]` and describes them as 'Point-Vector tuples'. However, since `Force` is a namedtuple subclass, it would display as a tuple, so this is plausible but unverifiable from the doc alone.",
      "The claim about `PathwayBase` being an ABC with abstract properties, the immutability pattern using `hasattr`, the specific helper functions (`_point_pair_relative_position`, `_point_pair_length`, `_point_pair_extension_velocity`), the `LoadBase` namedtuple inheritance, and `ObstacleSetPathway` requiring 3+ points are all implementation details not mentioned in the documentation - they are plausible extrapolations from code knowledge but not verifiable from the provided doc.",
      "The claim about `WrappingPathway.to_loads` producing 3 force loads (including on geometry's center point) is not mentioned in the documentation and cannot be verified.",
      "The description of `ForceActuator`, `LinearSpring`, and `LinearDamper` behavior is not covered in the provided documentation.",
      "The force formula in point 4 states force on attachments[0] is `-force * relative_position / self.length` and on attachments[-1] is `force * relative_position / self.length`. Looking at the doc example: `[(pA, - F*q(t)/sqrt(q(t)**2)*N.x), (pB, F*q(t)/sqrt(q(t)**2)*N.x)]` where relative_position is `q(t)*N.x` and length is `sqrt(q(t)**2)`, so `F * q(t)*N.x / sqrt(q(t)**2)` matches the output for pB. This is consistent with the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor semantic descriptions and an unverifiable detail about dynamicsymbols._t being Symbol('t') needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_54",
    "repo": "sympy",
    "question": "Why does ObstacleSetPathway override the attachments setter and how does its multi-segment load generation differ from WrappingPathway's approach?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `ObstacleSetPathway` class overrides the `attachments` property setter from `PathwayBase` specifically to change the validation constraint. While `PathwayBase.attachments` setter enforces `len(attachments) != 2` (exactly 2 points required), `ObstacleSetPathway.attachments` setter enforces `len(attachments) <= 2` (3 or more points required).\n\nThe length and extension_velocity computations in `ObstacleSetPathway` use an accumulation pattern: they initialize with `S.Zero` (SymPy's singleton zero) and iterate over consecutive pairs of attachment points using `zip(self.attachments[:-1], self.attachments[1:])`, summing the individual segment lengths/velocities. Both delegate to module-level helper functions `_point_pair_length` and `_point_pair_extension_velocity`. `_point_pair_length` calls `_point_pair_relative_position(p1, p2).magnitude()`, where `magnitude()` computes `sqrt(self.dot(self))` \u2014 the Euclidean norm. `_point_pair_extension_velocity` takes the time derivative of the length via `.diff(dynamicsymbols._t)`.\n\nThe `to_loads` method generates 2 `Force` objects per segment (so 2*(N-1) loads for N attachment points). For each consecutive pair, the force on the first point is `-force * relative_position / length` and on the second is `+force * relative_position / length`. This means intermediate points appear twice in the loads list \u2014 once as the end of one segment and once as the start of the next \u2014 receiving forces from both adjacent segments. This differs from `WrappingPathway.to_loads`, which generates exactly 3 Force objects (on pA, pB, and the geometry center point pO), with the force on pO computed as `-(pA_force + pB_force)`.\n\nThe `LoadBase` class (in `sympy/physics/mechanics/loads.py`) inherits from both `ABC` and `namedtuple('LoadBase', ['location', 'vector'])`, making each load a named tuple with `location` and `vector` fields. The `Force` class inherits from `LoadBase` and additionally provides `point` and `force` as property aliases for `location` and `vector`. Its `__new__` method also accepts a `BodyBase` instance in place of a Point, automatically extracting `masscenter`.\n\nThe `ForceActuator.to_loads()` method simply delegates to `self.pathway.to_loads(self.force)`, so an `ObstacleSetPathway` used with a `ForceActuator` will automatically produce the multi-segment load structure.\n\n`ObstacleSetPathway` does not override `__repr__` \u2014 it inherits the default from `PathwayBase`, which joins all attachment names with commas. In contrast, `WrappingPathway` overrides `__repr__` to additionally include `geometry=` in its representation.",
    "rubric": [
      "Explains that ObstacleSetPathway overrides the attachments setter to change the validation from len != 2 (exactly 2 required) to len <= 2 (3 or more required)",
      "Describes the accumulation pattern for length/extension_velocity: initializes with S.Zero and iterates over consecutive pairs using zip(attachments[:-1], attachments[1:])",
      "Explains that length uses _point_pair_length (which calls magnitude() on relative position) and extension_velocity uses _point_pair_extension_velocity (time derivative of length)",
      "Describes that to_loads generates 2 Force objects per segment (2*(N-1) total for N points), with intermediate points appearing twice in the loads list",
      "Explains the force direction convention: first point gets -force*relative_position/length, second point gets +force*relative_position/length",
      "Contrasts with WrappingPathway.to_loads which generates exactly 3 Force objects (on pA, pB, and geometry point pO) with pO receiving -(pA_force + pB_force)",
      "Notes that ObstacleSetPathway inherits __repr__ from PathwayBase while WrappingPathway overrides it to include geometry information"
    ],
    "key_files": [
      "sympy/physics/mechanics/pathway.py",
      "sympy/physics/mechanics/loads.py",
      "sympy/physics/mechanics/actuator.py",
      "sympy/physics/mechanics/tests/test_pathway.py",
      "sympy/physics/vector/vector.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/pathway.py] sympy.physics.mechanics.pathway.ObstacleSetPathway\nsympy.physics.mechanics.pathway.ObstacleSetPathway:\n  Obstacle-set pathway between a set of attachment points.\n\n    Explanation\n    ===========\n\n    An obstacle-set pathway forms a series of straight-line segment between\n    pairs of consecutive points in a set of points. It is similar to multiple\n    linear pathways joined end-to-end. It will not interact with any other\n    objects in the system, i.e. an ``ObstacleSetPathway`` will intersect other\n    objects to ensure that the path between its pairs of points (its\n    attachments) is the shortest possible.\n\n    Examples\n    ========\n\n    To construct an obstacle-set pathway, three or more points are required to\n    be passed to the ``attachments`` parameter as a ``tuple``.\n\n    >>> from sympy.physics.mechanics import ObstacleSetPathway, Point\n    >>> pA, pB, pC, pD = Point('pA'), Point('pB'), Point('pC'), Point('pD')\n    >>> obstacle_set_pathway = ObstacleSetPathway(pA, pB, pC, pD)\n    >>> obstacle_set_pathway\n    ObstacleSetPathway(pA, pB, pC, pD)\n\n    The pathway created above isn't very interesting without the positions and\n    velocities of its attachment points being described. Without this its not\n    possible to describe how the pathway moves, i.e. its length or its\n    extension velocity.\n\n    >>> from sympy import cos, sin\n    >>> from sympy.physics.mechanics import ReferenceFrame\n    >>> from sympy.physics.vector import dynamicsymbols\n    >>> N = ReferenceFrame('N')\n    >>> q = dynamicsymbols('q')\n    >>> pO = Point('pO')\n    >>> pA.set_pos(pO, N.y)\n    >>> pB.set_pos(pO, -N.x)\n    >>> pC.set_pos(pA, cos(q) * N.x - (sin(q) + 1) * N.y)\n    >>> pD.set_pos(pA, sin(q) * N.x + (cos(q) - 1) * N.y)\n    >>> pB.pos_from(pA)\n    - N.x - N.y\n    >>> pC.pos_from(pA)\n    cos(q(t))*N.x + (-sin(q(t)) - 1)*N.y\n    >>> pD.pos_from(pA)\n    sin(q(t))*N.x + (cos(q(t)) - 1)*N.y\n\n    A pathway's length can be accessed via its ``length`` attribute.\n\n    >>> obstacle_set_pathway.length.simplify()\n    sqrt(2)*(sqrt(cos(q(t)) + 1) + 2)\n\n    A pathway's extension velocity can be accessed similarly via its\n    ``extension_velocity`` attribute.\n\n    >>> obstacle_set_pathway.extension_velocity.simplify()\n    -sqrt(2)*sin(q(t))*Derivative(q(t), t)/(2*sqrt(cos(q(t)) + 1))\n\n    Parameters\n    ==========\n\n    attachments : tuple[Point, ...]\n        The set of ``Point`` objects that define the segmented obstacle-set\n        pathway.\n\nsympy.physics.mechanics.pathway.ObstacleSetPathway.__init__:\n  Initializer for ``ObstacleSetPathway``.\n\n        Parameters\n        ==========\n\n        attachments : tuple[Point, ...]\n            The set of ``Point`` objects that define the segmented obstacle-set\n            pathway.\n\nsympy.physics.mechanics.pathway.ObstacleSetPathway.attachments:\n  The set of points defining a pathway's segmented path.\n\nsympy.physics.mechanics.pathway.ObstacleSetPathway.length:\n  Exact analytical expression for the pathway's length.\n\nsympy.physics.mechanics.pathway.ObstacleSetPathway.extension_velocity:\n  Exact analytical expression for the pathway's extension velocity.\n\nsympy.physics.mechanics.pathway.ObstacleSetPathway.to_loads:\n  Loads required by the equations of motion method classes.\n\n        Explanation\n        ===========\n\n        ``KanesMethod`` requires a list of ``Point``-``Vector`` tuples to be\n        passed to the ``loads`` parameters of its ``kanes_equations`` method\n        when constructing the equations of motion. This method acts as a\n        utility to produce the correctly-structred pairs of points and vectors\n        required so that these can be easily concatenated with other items in\n        the list of loads and passed to ``KanesMethod.kanes_equations``. These\n        loads are also in the correct form to also be passed to the other\n        equations of motion method classes, e.g. ``LagrangesMethod``.\n\n        Examples\n        ========\n\n        The below example shows how to generate the loads produced in an\n        actuator that follows an obstacle-set pathway between four points and\n        produces an expansile force ``F``. First, create a pair of reference\n        frames, ``A`` and ``B``, in which the four points ``pA``, ``pB``,\n        ``pC``, and ``pD`` will be located. The first two points in frame ``A``\n        and the second two in frame ``B``. Frame ``B`` will also be oriented\n        such that it relates to ``A`` via a rotation of ``q`` about an axis\n        ``N.z`` in a global frame (``N.z``, ``A.z``, and ``B.z`` are parallel).\n\n        >>> from sympy.physics.mechanics import (ObstacleSetPathway, Point,\n        ...     ReferenceFrame)\n        >>> from sympy.physics.vector import dynamicsymbols\n        >>> q = dynamicsymbols('q')\n        >>> N = ReferenceFrame('N')\n        >>> N = ReferenceFrame('N')\n        >>> A = N.orientnew('A', 'axis', (0, N.x))\n        >>> B = A.orientnew('B', 'axis', (q, N.z))\n        >>> pO = Point('pO')\n        >>> pA, pB, pC, pD = Point('pA'), Point('pB'), Point('pC'), Point('pD')\n        >>> pA.set_pos(pO, A.x)\n        >>> pB.set_pos(pO, -A.y)\n        >>> pC.set_pos(pO, B.y)\n        >>> pD.set_pos(pO, B.x)\n        >>> obstacle_set_pathway = ObstacleSetPathway(pA, pB, pC, pD)\n\n        Now create a symbol ``F`` to describe the magnitude of the (expansile)\n        force that will be produced along the pathway. The list of loads that\n        ``KanesMethod`` requires can be produced by calling the pathway's\n        ``to_loads`` method with ``F`` passed as the only argument.\n\n        >>> from sympy import Symbol\n        >>> F = Symbol('F')\n        >>> obstacle_set_pathway.to_loads(F)\n        [(pA, sqrt(2)*F/2*A.x + sqrt(2)*F/2*A.y),\n         (pB, - sqrt(2)*F/2*A.x - sqrt(2)*F/2*A.y),\n         (pB, - F/sqrt(2*cos(q(t)) + 2)*A.y - F/sqrt(2*cos(q(t)) + 2)*B.y),\n         (pC, F/sqrt(2*cos(q(t)) + 2)*A.y + F/sqrt(2*cos(q(t)) + 2)*B.y),\n         (pC, - sqrt(2)*F/2*B.x + sqrt(2)*F/2*B.y),\n         (pD, sqrt(2)*F/2*B.x - sqrt(2)*F/2*B.y)]\n\n        Parameters\n        ==========\n\n        force : Expr\n            The force acting along the length of the pathway. It is assumed\n            that this ``Expr`` represents an expansile force.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about `PathwayBase.attachments` setter enforcing `len(attachments) != 2` and `ObstacleSetPathway.attachments` setter enforcing `len(attachments) <= 2` with specific line numbers references code not present in the documentation. These are plausible code-level details but cannot be verified from the provided documentation alone.",
      "The claim about `S.Zero` initialization, `zip(self.attachments[:-1], self.attachments[1:])` iteration pattern, and helper functions `_point_pair_length` and `_point_pair_extension_velocity` are implementation details not in the documentation.",
      "The claim about `magnitude()` being defined at `sympy/physics/vector/vector.py` line 632 computing `sqrt(self.dot(self))` is a code-level detail not verifiable from the documentation.",
      "The claim about `to_loads` generating 2*(N-1) loads for N attachment points is consistent with the example in the documentation (4 points \u2192 6 loads = 2*3).",
      "The claim about `Force` class inheriting from `ABC` and `namedtuple`, and details about `WrappingPathway.to_loads` generating exactly 3 Force objects, are not verifiable from the provided documentation.",
      "The claim about `ForceActuator.to_loads()` delegating to `self.pathway.to_loads(self.force)` is not verifiable from the provided documentation.",
      "The claim about `__repr__` inheritance from `PathwayBase` with specific line numbers is not verifiable from the documentation, though the example output `ObstacleSetPathway(pA, pB, pC, pD)` is consistent with it.",
      "The documentation states attachments are passed 'as a tuple' and that 'three or more points are required', which is consistent with the answer's claim about requiring at least 3 points."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: one claim about design rationale for the 3-point minimum was not recoverable from stripped code; line number references and a 'force balance' rationale also removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_55",
    "repo": "sympy",
    "question": "Where is the RigidBody class defined, how does it relate to BodyBase and Particle through inheritance, and how does it auto-generate defaults?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `RigidBody` class is located in `sympy/physics/mechanics/rigidbody.py` and inherits from `BodyBase` (defined in `sympy/physics/mechanics/body_base.py`), which is an abstract base class (ABC). `BodyBase` defines abstract methods `kinetic_energy`, `linear_momentum`, `angular_momentum`, and `parallel_axis` that `RigidBody` must implement.\n\nKey implementation details:\n\n1. **Auto-generation of defaults**: When `RigidBody` is instantiated without `frame`, `mass`, `masscenter`, or `inertia` arguments, it auto-generates them. Specifically:\n   - If `frame` is None, a `ReferenceFrame(f'{name}_frame')` is created.\n   - If `inertia` is None, six symbols (`{name}_ixx`, `{name}_iyy`, `{name}_izz`, `{name}_izx`, `{name}_ixy`, `{name}_iyz`) are created and `Inertia.from_inertia_scalars` is used.\n   - From the parent `BodyBase`: if `mass` is None, `Symbol(f'{name}_mass')` is created; if `masscenter` is None, `Point(f'{name}_masscenter')` is created.\n   - `BodyBase.__init__` also sets `self.potential_energy = 0` and `self.points = []`.\n\n2. **Inertia setter computes central inertia**: The `inertia` setter validates that the input is a tuple of `(Dyadic, Point)`, stores it as an `Inertia` namedtuple, then computes `self._central_inertia` as `I[0] - inertia_of_point_mass(self.mass, self.masscenter.pos_from(I[1]), self.frame)`.\n\n3. **`Inertia` class is a namedtuple**: `Inertia` (in `sympy/physics/mechanics/inertia.py`) extends `namedtuple('Inertia', ['dyadic', 'point'])` and its `__new__` method will automatically swap arguments if a `Point` is passed first and a `Dyadic` second. It also overrides `__add__` and `__mul__` to raise `TypeError`.\n\n4. **`parallel_axis` differs between RigidBody and Particle**: In `RigidBody`, the `frame` parameter is optional (defaults to `self.frame`) and adds `central_inertia + inertia_of_point_mass(...)`. In `Particle`, `frame` is required and simply returns `inertia_of_point_mass(self.mass, self.point.pos_from(point), frame)`.\n\n5. **Usage in KanesMethod**: In `kane.py`'s `_form_frstar` method, `RigidBody` instances are specifically handled differently from `Particle` \u2014 for rigid bodies, both translational and rotational partial velocities are computed (`body.masscenter.vel(N)` and `body.frame.ang_vel_in(N)`), and the inertial torque includes `I.dt(body.frame).dot(omega)` plus `I.dot(ang_acc)` plus `omega.cross(I.dot(omega))`.\n\n6. **`Body` class inherits from both**: The `Body` class (in `body.py`) uses multiple inheritance: `class Body(RigidBody, Particle)`.\n\n7. **`Particle.point` is an alias**: In `Particle`, `point = BodyBase.masscenter` creates `point` as a class-level property alias to `masscenter`.",
    "rubric": [
      "Identifies RigidBody is in sympy/physics/mechanics/rigidbody.py and inherits from BodyBase (an ABC in body_base.py)",
      "Explains BodyBase defines abstract methods (kinetic_energy, linear_momentum, angular_momentum, parallel_axis) that RigidBody implements",
      "Describes auto-generation of frame (ReferenceFrame(f'{name}_frame')) when frame argument is None",
      "Describes auto-generation of inertia using six symbols (ixx, iyy, izz, izx, ixy, iyz) and Inertia.from_inertia_scalars when inertia is None",
      "Describes BodyBase auto-generating mass (Symbol(f'{name}_mass')) and masscenter (Point(f'{name}_masscenter')) when None",
      "Explains the inertia setter computes central_inertia by subtracting inertia_of_point_mass from the provided dyadic",
      "Mentions the Inertia class is a namedtuple with dyadic and point fields that swaps arguments if given in wrong order",
      "Notes that parallel_axis in RigidBody has frame as optional (defaults to self.frame) while Particle requires it",
      "Mentions Particle.point is an alias for BodyBase.masscenter",
      "Mentions Body class uses multiple inheritance from both RigidBody and Particle"
    ],
    "key_files": [
      "sympy/physics/mechanics/rigidbody.py",
      "sympy/physics/mechanics/body_base.py",
      "sympy/physics/mechanics/inertia.py",
      "sympy/physics/mechanics/kane.py",
      "sympy/physics/mechanics/particle.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/rigidbody.py] sympy.physics.mechanics.rigidbody.RigidBody\nsympy.physics.mechanics.rigidbody.RigidBody:\n  An idealized rigid body.\n\n    Explanation\n    ===========\n\n    This is essentially a container which holds the various components which\n    describe a rigid body: a name, mass, center of mass, reference frame, and\n    inertia.\n\n    All of these need to be supplied on creation, but can be changed\n    afterwards.\n\n    Attributes\n    ==========\n\n    name : string\n        The body's name.\n    masscenter : Point\n        The point which represents the center of mass of the rigid body.\n    frame : ReferenceFrame\n        The ReferenceFrame which the rigid body is fixed in.\n    mass : Sympifyable\n        The body's mass.\n    inertia : (Dyadic, Point)\n        The body's inertia about a point; stored in a tuple as shown above.\n    potential_energy : Sympifyable\n        The potential energy of the RigidBody.\n\n    Examples\n    ========\n\n    >>> from sympy import Symbol\n    >>> from sympy.physics.mechanics import ReferenceFrame, Point, RigidBody\n    >>> from sympy.physics.mechanics import outer\n    >>> m = Symbol('m')\n    >>> A = ReferenceFrame('A')\n    >>> P = Point('P')\n    >>> I = outer (A.x, A.x)\n    >>> inertia_tuple = (I, P)\n    >>> B = RigidBody('B', P, A, m, inertia_tuple)\n    >>> # Or you could change them afterwards\n    >>> m2 = Symbol('m2')\n    >>> B.mass = m2\n\nsympy.physics.mechanics.rigidbody.RigidBody.frame:\n  The ReferenceFrame fixed to the body.\n\nsympy.physics.mechanics.rigidbody.RigidBody.x:\n  The basis Vector for the body, in the x direction.\n\nsympy.physics.mechanics.rigidbody.RigidBody.y:\n  The basis Vector for the body, in the y direction.\n\nsympy.physics.mechanics.rigidbody.RigidBody.z:\n  The basis Vector for the body, in the z direction.\n\nsympy.physics.mechanics.rigidbody.RigidBody.inertia:\n  The body's inertia about a point; stored as (Dyadic, Point).\n\nsympy.physics.mechanics.rigidbody.RigidBody.central_inertia:\n  The body's central inertia dyadic.\n\nsympy.physics.mechanics.rigidbody.RigidBody.linear_momentum:\n  Linear momentum of the rigid body.\n\n        Explanation\n        ===========\n\n        The linear momentum L, of a rigid body B, with respect to frame N is\n        given by:\n\n        ``L = m * v``\n\n        where m is the mass of the rigid body, and v is the velocity of the mass\n        center of B in the frame N.\n\n        Parameters\n        ==========\n\n        frame : ReferenceFrame\n            The frame in which linear momentum is desired.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Point, ReferenceFrame, outer\n        >>> from sympy.physics.mechanics import RigidBody, dynamicsymbols\n        >>> from sympy.physics.vector import init_vprinting\n        >>> init_vprinting(pretty_print=False)\n        >>> m, v = dynamicsymbols('m v')\n        >>> N = ReferenceFrame('N')\n        >>> P = Point('P')\n        >>> P.set_vel(N, v * N.x)\n        >>> I = outer (N.x, N.x)\n        >>> Inertia_tuple = (I, P)\n        >>> B = RigidBody('B', P, N, m, Inertia_tuple)\n        >>> B.linear_momentum(N)\n        m*v*N.x\n\nsympy.physics.mechanics.rigidbody.RigidBody.angular_momentum:\n  Returns the angular momentum of the rigid body about a point in the\n        given frame.\n\n        Explanation\n        ===========\n\n        The angular momentum H of a rigid body B about some point O in a frame N\n        is given by:\n\n        ``H = dot(I, w) + cross(r, m * v)``\n\n        where I and m are the central inertia dyadic and mass of rigid body B, w\n        is the angular velocity of body B in the frame N, r is the position\n        vector from point O to the mass center of B, and v is the velocity of\n        the mass center in the frame N.\n\n        Parameters\n        ==========\n\n        point : Point\n            The point about which angular momentum is desired.\n        frame : ReferenceFrame\n            The frame in which angular momentum is desired.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Point, ReferenceFrame, outer\n        >>> from sympy.physics.mechanics import RigidBody, dynamicsymbols\n        >>> from sympy.physics.vector import init_vprinting\n        >>> init_vprinting(pretty_print=False)\n        >>> m, v, r, omega = dynamicsymbols('m v r omega')\n        >>> N = ReferenceFrame('N')\n        >>> b = ReferenceFrame('b')\n        >>> b.set_ang_vel(N, omega * b.x)\n        >>> P = Point('P')\n        >>> P.set_vel(N, 1 * N.x)\n        >>> I = outer(b.x, b.x)\n        >>> B = RigidBody('B', P, b, m, (I, P))\n        >>> B.angular_momentum(P, N)\n        omega*b.x\n\nsympy.physics.mechanics.rigidbody.RigidBody.kinetic_energy:\n  Kinetic energy of the rigid body.\n\n        Explanation\n        ===========\n\n        The kinetic energy, T, of a rigid body, B, is given by:\n\n        ``T = 1/2 * (dot(dot(I, w), w) + dot(m * v, v))``\n\n        where I and m are the central inertia dyadic and mass of rigid body B\n        respectively, w is the body's angular velocity, and v is the velocity of\n        the body's mass center in the supplied ReferenceFrame.\n\n        Parameters\n        ==========\n\n        frame : ReferenceFrame\n            The RigidBody's angular velocity and the velocity of it's mass\n            center are typically defined with respect to an inertial frame but\n            any relevant frame in which the velocities are known can be\n            supplied.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.mechanics import Point, ReferenceFrame, outer\n        >>> from sympy.physics.mechanics import RigidBody\n        >>> from sympy import symbols\n        >>> m, v, r, omega = symbols('m v r omega')\n        >>> N = ReferenceFrame('N')\n        >>> b = ReferenceFrame('b')\n        >>> b.set_ang_vel(N, omega * b.x)\n        >>> P = Point('P')\n        >>> P.set_vel(N, v * N.x)\n        >>> I = outer (b.x, b.x)\n        >>> inertia_tuple = (I, P)\n        >>> B = RigidBody('B', P, b, m, inertia_tuple)\n        >>> B.kinetic_energy(N)\n        m*v**2/2 + omega**2/2\n\nsympy.physics.mechanics.rigidbody.RigidBody.parallel_axis:\n  Returns the inertia dyadic of the body with respect to another point.\n\n        Parameters\n        ==========\n\n        point : sympy.physics.vector.Point\n            The point to express the inertia dyadic about.\n        frame : sympy.physics.vector.ReferenceFrame\n            The reference frame used to construct the dyadic.\n\n        Returns\n        =======\n\n        inertia : sympy.physics.vector.Dyadic\n            The inertia dyadic of the rigid body expressed about the provided\n            point.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims RigidBody inherits from BodyBase (ABC) - this is not mentioned in the documentation but is a plausible implementation detail from the source code.",
      "The answer claims auto-generation of defaults when arguments are None - the documentation states 'All of these need to be supplied on creation', which could be seen as a mild contradiction if the class actually allows None arguments.",
      "The answer describes internal implementation details (inertia setter computing central_inertia, Inertia namedtuple behavior, KanesMethod internals, Body class multiple inheritance, Particle.point alias) that are not in the provided documentation - these are plausible code-level details but cannot be verified from the given doc.",
      "The documentation says 'All of these need to be supplied on creation, but can be changed afterwards' which could contradict the claim about auto-generation of defaults. However, this could be outdated documentation if the code was later updated to support defaults."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale phrases ('parallel axis theorem in reverse', 'preventing accidental arithmetic') and a reference to body.py not provided; all core claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_57",
    "repo": "sympy",
    "question": "What are the key design details and dependencies of the WrappingCylinder class and its related helper functions in the mechanics wrapping geometry module?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `WrappingCylinder` class in `sympy/physics/mechanics/wrapping_geometry.py` inherits from `WrappingGeometryBase`, which is an abstract base class (ABC) that defines three abstract methods: `point` (as an abstract property), `geodesic_length`, and `geodesic_end_vectors`. Notably, `point_on_surface` is NOT declared as abstract in the base class \u2014 each subclass implements it independently without a base class contract.\n\nKey implementation details:\n\n1. **Axis normalization**: The `axis` setter automatically normalizes the provided vector via `axis.normalize()`, so any non-unit vector passed as `axis` will be stored as a unit vector. This is also true for `WrappingCone`.\n\n2. **`_directional_atan` helper function**: The `geodesic_length` method depends on a private module-level function `_directional_atan(numerator, denominator)` to compute direction-aware arc angles. This function handles four cases:\n   - Both arguments numeric: uses `atan2` and shifts negative angles by `+2*pi`\n   - Numerator numeric, denominator symbolic: raises `NotImplementedError`\n   - Numerator symbolic, denominator numeric: raises `NotImplementedError`\n   - Both symbolic: simplifies `numerator/denominator` via `trigsimp`, then checks if the result is a `tan` instance (returns its argument as the angle) or a `-1 * tan(...)` expression (returns `2*pi - angle`). Otherwise raises `NotImplementedError`.\n\n3. **`cancel` from `sympy.polys.polytools`**: Before passing arguments to `_directional_atan`, the `geodesic_length` method applies `cancel()` to both the cross-product-dot-axis (numerator) and dot-product (denominator) of the perpendicular vectors.\n\n4. **`point_on_surface` uses `trigsimp`**: Unlike `WrappingSphere.point_on_surface` which returns `Eq(point_radius_squared, self.radius**2, evaluate=False)` directly, `WrappingCylinder.point_on_surface` applies `trigsimp` to the left-hand side: `Eq(trigsimp(point_radius_squared), self.radius**2, evaluate=False)`. The cylinder version first subtracts the parallel component along the axis before computing the perpendicular distance squared.\n\n5. **`geodesic_end_vectors` special case**: When the two points have the same normal (perpendicular-to-axis) component (`point_1_normal == point_2_normal`), the perpendicular vectors are set to `Vector(0)` (zero vectors).\n\n6. **`geodesic_end_vectors` sign convention**: The perpendicular direction at point_1 is `self.axis.cross(point_1_normal).normalize()` while at point_2 it is negated: `-self.axis.cross(point_2_normal).normalize()`. The point_2 end vector also negates the parallel component (`- parallel_length * self.axis`).\n\n7. **`WrappingPathway.to_loads` reaction force**: In `pathway.py`, `WrappingPathway.to_loads` computes a reaction force on the geometry's point as `pO_force = -(pA_force + pB_force)`. Three `Force` loads are returned: one on each attachment point and one on the geometry's point.\n\n8. **`__all__` includes four classes**: The module exports `WrappingGeometryBase`, `WrappingCylinder`, `WrappingSphere`, and `WrappingCone` in its `__all__` list.\n\n9. **The `WrappingGeometryBase.point` abstract method** uses `cls` as its parameter name rather than `self`, although it's decorated with `@property` \u2014 this is an inconsistency in the code but doesn't affect functionality.\n\n10. **`_directional_atan` is used by both `WrappingCylinder` and `WrappingCone`**: It's called in `WrappingCylinder.geodesic_length`, `WrappingCone.geodesic_length`, and `WrappingCone.geodesic_end_vectors`.",
    "rubric": [
      "WrappingCylinder inherits from WrappingGeometryBase, which is an ABC defining abstract methods: point (property), geodesic_length, and geodesic_end_vectors",
      "point_on_surface is NOT abstract in the base class \u2014 subclasses implement it independently",
      "The axis setter normalizes the provided vector automatically via axis.normalize()",
      "_directional_atan is a module-level helper that handles four cases: both numeric (atan2 + shift negative by 2*pi), mixed numeric/symbolic (NotImplementedError), and both symbolic (trigsimp + checks for tan instance or -1*tan pattern)",
      "geodesic_length applies cancel() from sympy.polys.polytools to the numerator and denominator before passing them to _directional_atan",
      "point_on_surface applies trigsimp to the perpendicular distance squared (after subtracting parallel component) before constructing the Eq",
      "geodesic_end_vectors returns zero vectors when both points have the same normal component perpendicular to axis",
      "The sign convention: point_1 perpendicular vector is axis.cross(point_1_normal).normalize(), point_2 is negated",
      "_directional_atan is used by both WrappingCylinder.geodesic_length and WrappingCone (geodesic_length and geodesic_end_vectors)",
      "The module's __all__ exports four classes: WrappingGeometryBase, WrappingCylinder, WrappingSphere, and WrappingCone"
    ],
    "key_files": [
      "sympy/physics/mechanics/wrapping_geometry.py",
      "sympy/physics/mechanics/pathway.py",
      "sympy/physics/mechanics/tests/test_wrapping_geometry.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/wrapping_geometry.py] sympy.physics.mechanics.wrapping_geometry.WrappingCylinder\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder:\n  A solid (infinite) cylindrical object.\n\n    Explanation\n    ===========\n\n    A wrapping geometry that allows for circular arcs to be defined between\n    pairs of points. These paths are always geodetic (the shortest possible) in\n    the sense that they will be a straight line on the unwrapped cylinder's\n    surface. However, it is also possible for a direction to be specified, i.e.\n    paths can be influenced such that they either wrap along the shortest side\n    or the longest side of the cylinder. To define these directions, rotations\n    are in the positive direction following the right-hand rule.\n\n    Examples\n    ========\n\n    To create a ``WrappingCylinder`` instance, a ``Symbol`` denoting its\n    radius, a ``Vector`` defining its axis, and a ``Point`` through which its\n    axis passes are needed:\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import (Point, ReferenceFrame,\n    ...     WrappingCylinder)\n    >>> N = ReferenceFrame('N')\n    >>> r = symbols('r')\n    >>> pO = Point('pO')\n    >>> ax = N.x\n\n    A cylinder with radius ``r``, and axis parallel to ``N.x`` passing through\n    ``pO`` can be instantiated with:\n\n    >>> WrappingCylinder(r, pO, ax)\n    WrappingCylinder(radius=r, point=pO, axis=N.x)\n\n    Parameters\n    ==========\n\n    radius : Symbol\n        The radius of the cylinder.\n    point : Point\n        A point through which the cylinder's axis passes.\n    axis : Vector\n        The axis along which the cylinder is aligned.\n\n    See Also\n    ========\n\n    WrappingSphere: Spherical geometry where the wrapping direction is always\n        geodetic.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.__init__:\n  Initializer for ``WrappingCylinder``.\n\n        Parameters\n        ==========\n\n        radius : Symbol\n            The radius of the cylinder. This symbol must represent a value that\n            is positive and constant, i.e. it cannot be a dynamic symbol.\n        point : Point\n            A point through which the cylinder's axis passes.\n        axis : Vector\n            The axis along which the cylinder is aligned.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.radius:\n  Radius of the cylinder.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.point:\n  A point through which the cylinder's axis passes.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.axis:\n  Axis along which the cylinder is aligned.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.point_on_surface:\n  Returns a symbolic equality for whether a point is on the cylinder's\n        surface.\n\n        Parameters\n        ==========\n\n        point : Point\n            The point for which the expression is to be generated.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.geodesic_length:\n  The shortest distance between two points on a geometry's surface.\n\n        Explanation\n        ===========\n\n        The geodesic length, i.e. the shortest arc along the surface of a\n        cylinder, connecting two points. It can be calculated using Pythagoras'\n        theorem. The first short side is the distance between the two points on\n        the cylinder's surface parallel to the cylinder's axis. The second\n        short side is the arc of a circle between the two points of the\n        cylinder's surface perpendicular to the cylinder's axis. The resulting\n        hypotenuse is the geodesic length.\n\n        Examples\n        ========\n\n        A geodesic length can only be calculated between two points on the\n        cylinder's surface. Firstly, a ``WrappingCylinder`` instance must be\n        created along with two points that will lie on its surface:\n\n        >>> from sympy import symbols, cos, sin\n        >>> from sympy.physics.mechanics import (Point, ReferenceFrame,\n        ...     WrappingCylinder, dynamicsymbols)\n        >>> N = ReferenceFrame('N')\n        >>> r = symbols('r')\n        >>> pO = Point('pO')\n        >>> pO.set_vel(N, 0)\n        >>> cylinder = WrappingCylinder(r, pO, N.x)\n        >>> p1 = Point('p1')\n        >>> p2 = Point('p2')\n\n        Let's assume that ``p1`` is located at ``N.x + r*N.y`` relative to\n        ``pO`` and that ``p2`` is located at ``r*(cos(q)*N.y + sin(q)*N.z)``\n        relative to ``pO``, where ``q(t)`` is a generalized coordinate\n        specifying the angle rotated around the ``N.x`` axis according to the\n        right-hand rule where ``N.y`` is zero. These positions can be set with:\n\n        >>> q = dynamicsymbols('q')\n        >>> p1.set_pos(pO, N.x + r*N.y)\n        >>> p1.pos_from(pO)\n        N.x + r*N.y\n        >>> p2.set_pos(pO, r*(cos(q)*N.y + sin(q)*N.z).normalize())\n        >>> p2.pos_from(pO).simplify()\n        r*cos(q(t))*N.y + r*sin(q(t))*N.z\n\n        The geodesic length, which is in this case a is the hypotenuse of a\n        right triangle where the other two side lengths are ``1`` (parallel to\n        the cylinder's axis) and ``r*q(t)`` (parallel to the cylinder's cross\n        section), can be calculated using the ``geodesic_length`` method:\n\n        >>> cylinder.geodesic_length(p1, p2).simplify()\n        sqrt(r**2*q(t)**2 + 1)\n\n        If the ``geodesic_length`` method is passed an argument ``Point`` that\n        doesn't lie on the cylinder's surface then unexpected results may occur.\n        It is the user's responsibility to ensure that the points lie on the\n        cylinder's surface.\n\n        Parameters\n        ==========\n\n        point_1 : Point\n            Point from which the geodesic length should be calculated.\n        point_2 : Point\n            Point to which the geodesic length should be calculated.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.geodesic_end_vectors:\n  The vectors parallel to the geodesic at the two end points.\n\n        Parameters\n        ==========\n\n        point_1 : Point\n            The point from which the geodesic originates.\n        point_2 : Point\n            The point at which the geodesic terminates.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingCylinder.__repr__:\n  Representation of a ``WrappingCylinder``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer (points 1-10) go far beyond what the documentation covers, referencing specific line numbers, internal helper functions, and implementation details not present in the provided documentation. These cannot be verified from the documentation alone.",
      "The answer's description of `WrappingCylinder` as inheriting from `WrappingGeometryBase` is not stated in the documentation but is a plausible code-level detail.",
      "The claim about `point_on_surface` NOT being declared as abstract in the base class cannot be verified from the documentation.",
      "The claim about axis normalization cannot be verified from the documentation (the documentation only says axis is 'The axis along which the cylinder is aligned').",
      "The detailed description of `_directional_atan` helper function cannot be verified from the documentation.",
      "The claim about `WrappingPathway.to_loads` reaction force references code from a different module (`pathway.py`) entirely outside the scope of the provided documentation.",
      "The claim about `__all__` including `WrappingCone` cannot be verified; `WrappingCone` is not mentioned in the documentation.",
      "No claims directly contradict the provided documentation - the basic class description, parameters (radius, point, axis), and geodesic_length behavior are consistent with the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only minor design rationale phrases ('ensuring force balance', 'making the geodesic a straight line') needed trimming.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_56",
    "repo": "sympy",
    "question": "How does WrappingSphere compute geodesic lengths and tangent end vectors, and how does WrappingPathway use those results to build force loads?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `WrappingSphere` class implements geodesic calculations on a spherical surface using the following algorithms:\n\n**`geodesic_length` implementation:**\n1. Computes unit vectors from the sphere center to each point by calling `point.pos_from(self.point).normalize()` on both points\n2. Calculates the central angle as `acos(point_2_vector.dot(point_1_vector))` \u2014 the dot product of the two unit vectors\n3. Returns `self.radius * central_angle`\n\nThe `normalize()` method (in `sympy/physics/vector/vector.py`) divides the vector by `sqrt(self.dot(self))`, so any point at distance `r` from center gets a unit vector.\n\n**`geodesic_end_vectors` implementation:**\n1. Gets position vectors `pA_vec` and `pB_vec` from the sphere center to the two points\n2. Checks if `pA_vec.cross(pB_vec) == 0` to detect coincident OR diametrically opposed points \u2014 both cases raise `ValueError` with message referencing 'diametrically opposed'\n3. Uses a **double cross product** formula to compute tangent vectors:\n   - For point_1: `pA_vec.cross(pB.pos_from(pA)).cross(pA_vec).normalize()`\n   - For point_2: `pB_vec.cross(pA.pos_from(pB)).cross(pB_vec).normalize()`\n   Note that the intermediate vector uses `pB.pos_from(pA)` (position of B relative to A), NOT relative to center.\n\n**`point_on_surface` implementation:**\nChecks if the point vector is a `Vector` instance (vs. a scalar 0) and uses the module-level `dot` function (from `sympy.physics.vector`) when it is a Vector, otherwise squares the scalar directly. Returns `Eq(point_radius_squared, self.radius**2, evaluate=False)`.\n\n**Integration with `WrappingPathway` (pathway.py):**\nThe `to_loads` method in `WrappingPathway` uses `geodesic_end_vectors` to get force directions, and additionally computes a reaction force on the geometry's center point as `pO_force = -(pA_force + pB_force)`, creating three `Force` loads: one on each attachment point and one on the geometry's center point (`self.geometry.point`).\n\n**Inheritance:** `WrappingSphere` inherits from `WrappingGeometryBase` (ABC) which defines three abstract methods: `point` (property), `geodesic_length`, and `geodesic_end_vectors`. Unlike `WrappingCylinder`, the sphere has no `axis` property. The base class provides a default `__repr__` returning just the class name with empty parens, which `WrappingSphere` overrides.\n\n**Vector equality check detail:** The `Vector.__eq__` method when comparing to 0 converts 0 to `Vector(0)`, then if both have empty args returns True; otherwise iterates over frame basis vectors checking `expand((self - other).dot(v)) != 0`.",
    "rubric": [
      "geodesic_length normalizes position vectors from sphere center to each point, computes central angle via acos of their dot product, and multiplies by the radius",
      "geodesic_end_vectors checks if the cross product of the two center-to-point vectors is zero and raises ValueError (mentioning diametrically opposed) for both coincident and antipodal cases",
      "geodesic_end_vectors uses a double cross product formula: pA_vec.cross(pB.pos_from(pA)).cross(pA_vec).normalize() (and analogously for point_2), where the intermediate vector uses the relative position between points, not from center",
      "point_on_surface checks if the position vector is a Vector instance and dispatches to either the dot function or scalar squaring, returning an unevaluated Eq comparing to radius squared",
      "WrappingPathway.to_loads computes a reaction force on the geometry's center point as the negative sum of the two endpoint force vectors, producing three Force loads total"
    ],
    "key_files": [
      "sympy/physics/mechanics/wrapping_geometry.py",
      "sympy/physics/vector/vector.py",
      "sympy/physics/mechanics/pathway.py"
    ],
    "source_doc": "[docstring: sympy/physics/mechanics/wrapping_geometry.py] sympy.physics.mechanics.wrapping_geometry.WrappingSphere\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere:\n  A solid spherical object.\n\n    Explanation\n    ===========\n\n    A wrapping geometry that allows for circular arcs to be defined between\n    pairs of points. These paths are always geodetic (the shortest possible).\n\n    Examples\n    ========\n\n    To create a ``WrappingSphere`` instance, a ``Symbol`` denoting its radius\n    and ``Point`` at which its center will be located are needed:\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.mechanics import Point, WrappingSphere\n    >>> r = symbols('r')\n    >>> pO = Point('pO')\n\n    A sphere with radius ``r`` centered on ``pO`` can be instantiated with:\n\n    >>> WrappingSphere(r, pO)\n    WrappingSphere(radius=r, point=pO)\n\n    Parameters\n    ==========\n\n    radius : Symbol\n        Radius of the sphere. This symbol must represent a value that is\n        positive and constant, i.e. it cannot be a dynamic symbol, nor can it\n        be an expression.\n    point : Point\n        A point at which the sphere is centered.\n\n    See Also\n    ========\n\n    WrappingCylinder: Cylindrical geometry where the wrapping direction can be\n        defined.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.__init__:\n  Initializer for ``WrappingSphere``.\n\n        Parameters\n        ==========\n\n        radius : Symbol\n            The radius of the sphere.\n        point : Point\n            A point on which the sphere is centered.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.radius:\n  Radius of the sphere.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.point:\n  A point on which the sphere is centered.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.point_on_surface:\n  Returns a symbolic equality for whether a point is on the sphere's\n        surface.\n\n        Parameters\n        ==========\n\n        point : Point\n            The point for which the expression is to be generated.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.geodesic_length:\n  Returns the shortest distance between two points on the sphere's\n        surface.\n\n        Explanation\n        ===========\n\n        The geodesic length, i.e. the shortest arc along the surface of a\n        sphere, connecting two points can be calculated using the formula:\n\n        .. math::\n\n           l = \\arccos\\left(\\mathbf{v}_1 \\cdot \\mathbf{v}_2\\right)\n\n        where $\\mathbf{v}_1$ and $\\mathbf{v}_2$ are the unit vectors from the\n        sphere's center to the first and second points on the sphere's surface\n        respectively. Note that the actual path that the geodesic will take is\n        undefined when the two points are directly opposite one another.\n\n        Examples\n        ========\n\n        A geodesic length can only be calculated between two points on the\n        sphere's surface. Firstly, a ``WrappingSphere`` instance must be\n        created along with two points that will lie on its surface:\n\n        >>> from sympy import symbols\n        >>> from sympy.physics.mechanics import (Point, ReferenceFrame,\n        ...     WrappingSphere)\n        >>> N = ReferenceFrame('N')\n        >>> r = symbols('r')\n        >>> pO = Point('pO')\n        >>> pO.set_vel(N, 0)\n        >>> sphere = WrappingSphere(r, pO)\n        >>> p1 = Point('p1')\n        >>> p2 = Point('p2')\n\n        Let's assume that ``p1`` lies at a distance of ``r`` in the ``N.x``\n        direction from ``pO`` and that ``p2`` is located on the sphere's\n        surface in the ``N.y + N.z`` direction from ``pO``. These positions can\n        be set with:\n\n        >>> p1.set_pos(pO, r*N.x)\n        >>> p1.pos_from(pO)\n        r*N.x\n        >>> p2.set_pos(pO, r*(N.y + N.z).normalize())\n        >>> p2.pos_from(pO)\n        sqrt(2)*r/2*N.y + sqrt(2)*r/2*N.z\n\n        The geodesic length, which is in this case is a quarter of the sphere's\n        circumference, can be calculated using the ``geodesic_length`` method:\n\n        >>> sphere.geodesic_length(p1, p2)\n        pi*r/2\n\n        If the ``geodesic_length`` method is passed an argument that doesn't\n        lie on the sphere's surface then unexpected results may be obtained.\n        It is the user's responsibility to ensure that the points lie on the\n        sphere's surface.\n\n        Parameters\n        ==========\n\n        point_1 : Point\n            Point from which the geodesic length should be calculated.\n        point_2 : Point\n            Point to which the geodesic length should be calculated.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.geodesic_end_vectors:\n  The vectors parallel to the geodesic at the two end points.\n\n        Parameters\n        ==========\n\n        point_1 : Point\n            The point from which the geodesic originates.\n        point_2 : Point\n            The point at which the geodesic terminates.\n\nsympy.physics.mechanics.wrapping_geometry.WrappingSphere.__repr__:\n  Representation of a ``WrappingSphere``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The geodesic_length formula in the documentation is stated as l = arccos(v1 \u00b7 v2) without explicitly multiplying by radius, but the answer claims it returns self.radius * central_angle. The documentation example shows sphere.geodesic_length(p1, p2) returning pi*r/2, which is consistent with radius * arccos(dot product), so the answer's claim is consistent with the example but the formula shown in the doc omits the radius multiplication (likely a doc oversight or the formula shows the angle not the length).",
      "Line numbers (e.g., lines 239-243, lines 257-273, lines 163-168) are not verifiable from the documentation provided and may be fabricated.",
      "Details about geodesic_end_vectors implementation (double cross product formula, ValueError for diametrically opposed points) are not mentioned in the documentation and cannot be verified - they are plausible but unverifiable code-level claims.",
      "Details about point_on_surface implementation (Vector instance check, evaluate=False) go beyond what the documentation states and cannot be verified.",
      "Claims about WrappingPathway integration (pathway.py, to_loads method, reaction force computation) are entirely outside the scope of the provided documentation.",
      "Claims about WrappingGeometryBase ABC and its abstract methods are not in the provided documentation.",
      "Claims about Vector.__eq__ method implementation details are not in the provided documentation.",
      "The documentation formula l = arccos(v1\u00b7v2) appears to give the arc angle, not the arc length. The answer correctly notes that the result should be r*arccos(v1\u00b7v2) based on the example, suggesting the doc formula may be incomplete, but the answer's description is consistent with the example output."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only line number references and a minor explanation about evaluate=False semantics needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_58",
    "repo": "sympy",
    "question": "Why does `RayTransferMatrix.__mul__` cast both operands to plain `Matrix` objects before performing multiplication?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `RayTransferMatrix.__mul__` method uses explicit `isinstance` type-checking dispatch. When multiplying two `RayTransferMatrix` objects or a `RayTransferMatrix` with a `GeometricRay`, it first casts both operands to plain `Matrix` objects via `Matrix(self)` and `Matrix(other)` before performing the multiplication, and then wraps the result back in the appropriate domain type. Casting to `Matrix` prevents re-entering the same `__mul__` and ensures the result preserves the correct domain type (e.g., `RayTransferMatrix * RayTransferMatrix` \u2192 `RayTransferMatrix`, not a plain `Matrix`).\n\nFor `BeamParameter` multiplication, the code constructs a 2\u00d71 column vector `Matrix(((other.q,), (1,)))` from the complex beam parameter `q`, performs the ABCD matrix multiplication, then uses `.expand(complex=True)` followed by `re()` and `im()` to separate real and imaginary parts, and wraps them with `together()` (from `sympy.polys.rationaltools`).\n\nKey design details:\n1. `RayTransferMatrix` inherits from `MutableDenseMatrix` (not `ImmutableDenseMatrix`), which itself inherits from both `DenseMatrix` and `MutableRepMatrix`. `MutableDenseMatrix` has `_op_priority = 10.01` (set in `DenseMatrix` in `sympy/matrices/dense.py`). Since `BeamParameter` inherits from `Expr` (not `Matrix`), the matrix's higher `_op_priority` ensures `RayTransferMatrix.__mul__` is called first when using the `call_highest_priority` dispatch mechanism visible in `sympy/core/decorators.py`.\n2. The `__new__` constructor only accepts exactly 4 arguments (A, B, C, D) or exactly 1 argument that must be a `Matrix` with shape `(2, 2)` \u2014 any other input raises a `ValueError` using `filldedent()` (from `sympy.utilities.misc`).\n3. The fallback `Matrix.__mul__(self, other)` delegates to the parent class's multiplication logic. The `_dotprodsimp` utility is imported in `sympy/matrices/common.py` for use in matrix multiplication.\n4. Subclasses like `FlatRefraction` and `CurvedRefraction` call `map(sympify, ...)` on their arguments before passing to `RayTransferMatrix.__new__`, ensuring proper symbolic conversion, while `FreeSpace` does not sympify its `d` argument.\n5. `FlatMirror` is the only subclass whose `__new__` takes no parameters \u2014 it is always the 2\u00d72 identity matrix `[[1,0],[0,1]]`.",
    "rubric": [
      "Explains that casting to plain Matrix avoids infinite recursion by preventing re-entry into the custom __mul__ method",
      "Notes that the result is wrapped back into the appropriate domain type (RayTransferMatrix, GeometricRay, or BeamParameter) after multiplication",
      "Mentions the isinstance-based dispatch that handles RayTransferMatrix, GeometricRay, and BeamParameter operands differently",
      "Describes how BeamParameter multiplication constructs a 2\u00d71 column vector from the complex beam parameter q, performs matrix multiplication, then extracts real/imaginary parts using expand(complex=True), re(), im(), and together()",
      "Notes that RayTransferMatrix inherits from MutableDenseMatrix (not ImmutableDenseMatrix), and BeamParameter inherits from Expr, so the matrix's higher _op_priority ensures RayTransferMatrix.__mul__ is called when multiplying by a BeamParameter"
    ],
    "key_files": [
      "sympy/physics/optics/gaussopt.py",
      "sympy/matrices/dense.py",
      "sympy/matrices/common.py",
      "sympy/core/decorators.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/gaussopt.py] sympy.physics.optics.gaussopt.RayTransferMatrix\nsympy.physics.optics.gaussopt.RayTransferMatrix:\n  Base class for a Ray Transfer Matrix.\n\n    It should be used if there is not already a more specific subclass mentioned\n    in See Also.\n\n    Parameters\n    ==========\n\n    parameters :\n        A, B, C and D or 2x2 matrix (Matrix(2, 2, [A, B, C, D]))\n\n    Examples\n    ========\n\n    >>> from sympy.physics.optics import RayTransferMatrix, ThinLens\n    >>> from sympy import Symbol, Matrix\n\n    >>> mat = RayTransferMatrix(1, 2, 3, 4)\n    >>> mat\n    Matrix([\n    [1, 2],\n    [3, 4]])\n\n    >>> RayTransferMatrix(Matrix([[1, 2], [3, 4]]))\n    Matrix([\n    [1, 2],\n    [3, 4]])\n\n    >>> mat.A\n    1\n\n    >>> f = Symbol('f')\n    >>> lens = ThinLens(f)\n    >>> lens\n    Matrix([\n    [   1, 0],\n    [-1/f, 1]])\n\n    >>> lens.C\n    -1/f\n\n    See Also\n    ========\n\n    GeometricRay, BeamParameter,\n    FreeSpace, FlatRefraction, CurvedRefraction,\n    FlatMirror, CurvedMirror, ThinLens\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Ray_transfer_matrix_analysis\n\nsympy.physics.optics.gaussopt.RayTransferMatrix.A:\n  The A parameter of the Matrix.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import RayTransferMatrix\n        >>> mat = RayTransferMatrix(1, 2, 3, 4)\n        >>> mat.A\n        1\n\nsympy.physics.optics.gaussopt.RayTransferMatrix.B:\n  The B parameter of the Matrix.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import RayTransferMatrix\n        >>> mat = RayTransferMatrix(1, 2, 3, 4)\n        >>> mat.B\n        2\n\nsympy.physics.optics.gaussopt.RayTransferMatrix.C:\n  The C parameter of the Matrix.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import RayTransferMatrix\n        >>> mat = RayTransferMatrix(1, 2, 3, 4)\n        >>> mat.C\n        3\n\nsympy.physics.optics.gaussopt.RayTransferMatrix.D:\n  The D parameter of the Matrix.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import RayTransferMatrix\n        >>> mat = RayTransferMatrix(1, 2, 3, 4)\n        >>> mat.D\n        4",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of claims in the answer (about __mul__ dispatch, isinstance checking, Matrix casting, BeamParameter handling, together(), MutableDenseMatrix inheritance, _op_priority, __new__ constructor details, FlatRefraction/CurvedRefraction/FreeSpace/FlatMirror implementation details) are not supported by the provided documentation. The documentation only covers the basic RayTransferMatrix class, its parameters A/B/C/D, constructor examples, and related classes.",
      "The answer discusses internal implementation details (line numbers, specific method dispatch, inheritance chains, _op_priority values) that cannot be verified from the provided documentation chunk.",
      "The claim that __new__ 'only accepts exactly 4 arguments or exactly 1 argument that must be a Matrix with shape (2,2)' is consistent with the documentation examples showing both RayTransferMatrix(1,2,3,4) and RayTransferMatrix(Matrix([[1,2],[3,4]])), though the ValueError behavior is not documented.",
      "The claim about FlatMirror being the identity matrix, FreeSpace not sympifying, etc. are all beyond the scope of the provided documentation and cannot be verified."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Minor doc leakage in descriptions of together() behavior, filldedent wrapping details, and some design rationale; most claims verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_59",
    "repo": "sympy",
    "question": "Where is `FreeSpace` defined and how does its class hierarchy and multiplication behavior work in the optics module?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `FreeSpace` class is defined at line 205 in `sympy/physics/optics/gaussopt.py`. It inherits from `RayTransferMatrix`, which is defined at line 60 of the same file and inherits from `MutableDenseMatrix` (defined at line 123 of `sympy/matrices/dense.py`). The full class hierarchy is: FreeSpace \u2192 RayTransferMatrix \u2192 MutableDenseMatrix \u2192 (DenseMatrix, MutableRepMatrix) \u2192 RepMatrix \u2192 MatrixBase.\n\nThe `FreeSpace.__new__` method (line 230-231) simply calls `RayTransferMatrix.__new__(cls, 1, d, 0, 1)`, constructing a 2\u00d72 matrix with elements [1, d, 0, 1]. Unlike other subclasses like `CurvedRefraction` and `ThinLens`, `FreeSpace` does NOT call `sympify()` on its input parameter `d` before passing it to the parent constructor.\n\nThe `RayTransferMatrix.__new__` method (line 116-128) handles either 4 individual arguments or a single 2\u00d72 Matrix, packaging 4 arguments as `((args[0], args[1]), (args[2], args[3]))` and then calling `Matrix.__new__(cls, temp)`.\n\nWhen `FreeSpace` is multiplied with a `GeometricRay`, the `RayTransferMatrix.__mul__` method (line 130-142) returns a `GeometricRay` object. When multiplied with a `BeamParameter`, it performs the ABCD transformation: constructs a column vector `(q, 1)`, multiplies the matrix by it, then computes the new q parameter as `temp[0]/temp[1]` expanded with `complex=True`, and constructs a new `BeamParameter` with the real part as the new z and imaginary part as the new z_r.",
    "rubric": [
      "FreeSpace is defined in sympy/physics/optics/gaussopt.py and inherits from RayTransferMatrix",
      "RayTransferMatrix inherits from MutableDenseMatrix (or Matrix)",
      "FreeSpace.__new__ calls RayTransferMatrix.__new__(cls, 1, d, 0, 1) to construct a 2x2 matrix [1, d; 0, 1]",
      "FreeSpace does NOT call sympify() on its input parameter d (unlike some other subclasses like CurvedRefraction or ThinLens)",
      "RayTransferMatrix.__new__ packages 4 arguments as ((args[0], args[1]), (args[2], args[3])) before passing to Matrix.__new__",
      "RayTransferMatrix.__mul__ returns a GeometricRay when multiplied with a GeometricRay",
      "When multiplied with a BeamParameter, __mul__ performs the ABCD transformation computing new q as temp[0]/temp[1]"
    ],
    "key_files": [
      "sympy/physics/optics/gaussopt.py",
      "sympy/matrices/dense.py",
      "sympy/physics/optics/tests/test_gaussopt.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/gaussopt.py] sympy.physics.optics.gaussopt.FreeSpace\nsympy.physics.optics.gaussopt.FreeSpace:\n  Ray Transfer Matrix for free space.\n\n    Parameters\n    ==========\n\n    distance\n\n    See Also\n    ========\n\n    RayTransferMatrix\n\n    Examples\n    ========\n\n    >>> from sympy.physics.optics import FreeSpace\n    >>> from sympy import symbols\n    >>> d = symbols('d')\n    >>> FreeSpace(d)\n    Matrix([\n    [1, d],\n    [0, 1]])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "No doc-leakage found; only line number references needed adjustment since they may shift when docstrings are stripped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_61",
    "repo": "sympy",
    "question": "What functions does the polarization optics module provide, and how is it structured in terms of classes, imports, and key implementation details?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `sympy/physics/optics/polarization.py` module is a purely functional module (no classes) that provides 11 public functions. Here is its detailed architecture:\n\n**Functions (all return `sympy.matrices.dense.Matrix` instances):**\n1. `jones_vector(psi, chi)` \u2014 returns a 2\u00d71 Matrix\n2. `stokes_vector(psi, chi, p=1, I=1)` \u2014 returns a 4\u00d71 Matrix (note: the parameter `I` shadows the imported imaginary unit `I` from `sympy.core.numbers`)\n3. `jones_2_stokes(e)` \u2014 converts a Jones vector to a Stokes vector using `Abs`, `re`, `im`, and `conjugate()`; the S3 component uses a negative sign: `-2*im(ex*ey.conjugate())`\n4. `linear_polarizer(theta=0)` \u2014 2\u00d72 Matrix\n5. `phase_retarder(theta=0, delta=0)` \u2014 2\u00d72 Matrix; the internal matrix R is multiplied by a global phase factor `exp(-I*delta/2)` before returning\n6. `half_wave_retarder(theta)` \u2014 delegates entirely to `phase_retarder(theta, pi)`\n7. `quarter_wave_retarder(theta)` \u2014 delegates entirely to `phase_retarder(theta, pi/2)`\n8. `transmissive_filter(T)` \u2014 diagonal 2\u00d72 matrix with `sqrt(T)` entries\n9. `reflective_filter(R)` \u2014 diagonal 2\u00d72 matrix with `sqrt(R)` and `-sqrt(R)` (negative sign on the (1,1) entry)\n10. `mueller_matrix(J)` \u2014 converts a 2\u00d72 Jones matrix to a 4\u00d74 Mueller matrix using the formula `A * TensorProduct(J, J.conjugate()) * A.inv()` wrapped in `simplify()`. The transformation matrix A is `Matrix([[1,0,0,1],[1,0,0,-1],[0,1,1,0],[0,-I,I,0]])`.\n11. `polarizing_beam_splitter(Tp=1, Rs=1, Ts=0, Rp=0, phia=0, phib=0)` \u2014 returns a 4\u00d74 Matrix\n\n**Key imports and dependencies:**\n- The module imports from 7 distinct sympy subpackages: `core.numbers`, `functions.elementary.complexes`, `functions.elementary.exponential`, `functions.elementary.miscellaneous`, `functions.elementary.trigonometric`, `matrices.dense`, `simplify.simplify`, and `physics.quantum`.\n- `TensorProduct` is imported from `sympy.physics.quantum`, which re-exports it from `sympy.physics.quantum.tensorproduct`. The `TensorProduct` class extends `sympy.core.expr.Expr` and its `__new__` method detects when the first argument is a `Matrix` (or `ImmutableMatrix`, numpy array, or scipy sparse matrix), and in that case delegates to `matrix_tensor_product()` from `sympy.physics.quantum.matrixutils`.\n- `matrix_tensor_product` for SymPy matrices delegates to `_sympy_tensor_product`, which in turn calls `matrix_kronecker_product` from `sympy.matrices.expressions.kronecker`. The Kronecker product is computed iteratively from right to left, building the result by `row_join` and `col_join` operations.\n\n**Test structure:**\n- Tests are in `sympy/physics/optics/tests/test_polarization.py` with a single test function `test_polarization()` that verifies all 11 functions. For example, it asserts `half_wave_retarder(0) == Matrix([[-I, 0], [0, I]])` and `polarizing_beam_splitter()` with default arguments equals `Matrix([[1, 0, 0, 0], [0, 0, 0, -I], [0, 0, 1, 0], [0, -I, 0, 0]])`.\n\n**Module exposure:**\n- All 11 functions are listed in `sympy/physics/optics/__init__.py`'s `__all__` and imported there, making them accessible via `from sympy.physics.optics import ...`.",
    "rubric": [
      "States the module is purely functional with no classes (11 public functions)",
      "Lists or describes the main functions: jones_vector, stokes_vector, jones_2_stokes, linear_polarizer, phase_retarder, half_wave_retarder, quarter_wave_retarder, transmissive_filter, reflective_filter, mueller_matrix, polarizing_beam_splitter",
      "Mentions that half_wave_retarder and quarter_wave_retarder delegate to phase_retarder with pi and pi/2 respectively",
      "Explains that mueller_matrix uses TensorProduct (from sympy.physics.quantum) with J and J.conjugate(), and the transformation matrix A",
      "Notes that phase_retarder multiplies by a global phase factor exp(-I*delta/2)",
      "Mentions reflective_filter has a negative sign on one diagonal entry (-sqrt(R))",
      "Notes jones_2_stokes uses a negative sign in the S3 component: -2*im(ex*ey.conjugate())",
      "Mentions all 11 functions are exported via the optics package __init__.py __all__ list"
    ],
    "key_files": [
      "sympy/physics/optics/polarization.py",
      "sympy/physics/quantum/tensorproduct.py",
      "sympy/physics/quantum/matrixutils.py",
      "sympy/matrices/expressions/kronecker.py",
      "sympy/physics/optics/tests/test_polarization.py",
      "sympy/physics/optics/__init__.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/polarization.py] sympy.physics.optics.polarization\nThe module implements routines to model the polarization of optical fields\nand can be used to calculate the effects of polarization optical elements on\nthe fields.\n\n- Jones vectors.\n\n- Stokes vectors.\n\n- Jones matrices.\n\n- Mueller matrices.\n\nExamples\n========\n\nWe calculate a generic Jones vector:\n\n>>> from sympy import symbols, pprint, zeros, simplify\n>>> from sympy.physics.optics.polarization import (jones_vector, stokes_vector,\n...     half_wave_retarder, polarizing_beam_splitter, jones_2_stokes)\n\n>>> psi, chi, p, I0 = symbols(\"psi, chi, p, I0\", real=True)\n>>> x0 = jones_vector(psi, chi)\n>>> pprint(x0, use_unicode=True)\n\u23a1-\u2148\u22c5sin(\u03c7)\u22c5sin(\u03c8) + cos(\u03c7)\u22c5cos(\u03c8)\u23a4\n\u23a2                                \u23a5\n\u23a3\u2148\u22c5sin(\u03c7)\u22c5cos(\u03c8) + sin(\u03c8)\u22c5cos(\u03c7) \u23a6\n\nAnd the more general Stokes vector:\n\n>>> s0 = stokes_vector(psi, chi, p, I0)\n>>> pprint(s0, use_unicode=True)\n\u23a1          I\u2080          \u23a4\n\u23a2                      \u23a5\n\u23a2I\u2080\u22c5p\u22c5cos(2\u22c5\u03c7)\u22c5cos(2\u22c5\u03c8)\u23a5\n\u23a2                      \u23a5\n\u23a2I\u2080\u22c5p\u22c5sin(2\u22c5\u03c8)\u22c5cos(2\u22c5\u03c7)\u23a5\n\u23a2                      \u23a5\n\u23a3    I\u2080\u22c5p\u22c5sin(2\u22c5\u03c7)     \u23a6\n\nWe calculate how the Jones vector is modified by a half-wave plate:\n\n>>> alpha = symbols(\"alpha\", real=True)\n>>> HWP = half_wave_retarder(alpha)\n>>> x1 = simplify(HWP*x0)\n\nWe calculate the very common operation of passing a beam through a half-wave\nplate and then through a polarizing beam-splitter. We do this by putting this\nJones vector as the first entry of a two-Jones-vector state that is transformed\nby a 4x4 Jones matrix modelling the polarizing beam-splitter to get the\ntransmitted and reflected Jones vectors:\n\n>>> PBS = polarizing_beam_splitter()\n>>> X1 = zeros(4, 1)\n>>> X1[:2, :] = x1\n>>> X2 = PBS*X1\n>>> transmitted_port = X2[:2, :]\n>>> reflected_port = X2[2:, :]\n\nThis allows us to calculate how the power in both ports depends on the initial\npolarization:\n\n>>> transmitted_power = jones_2_stokes(transmitted_port)[0]\n>>> reflected_power = jones_2_stokes(reflected_port)[0]\n>>> print(transmitted_power)\ncos(-2*alpha + chi + psi)**2/2 + cos(2*alpha + chi - psi)**2/2\n\n\n>>> print(reflected_power)\nsin(-2*alpha + chi + psi)**2/2 + sin(2*alpha + chi - psi)**2/2\n\nPlease see the description of the individual functions for further\ndetails and examples.\n\nReferences\n==========\n\n.. [1] https://en.wikipedia.org/wiki/Jones_calculus\n.. [2] https://en.wikipedia.org/wiki/Mueller_calculus\n.. [3] https://en.wikipedia.org/wiki/Stokes_parameters",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `jones_2_stokes` uses a negative sign `-2*im(ex*ey.conjugate())` for S3 cannot be verified from the documentation alone - it's an implementation detail that could be a hallucination.",
      "The claim about `reflective_filter` having a negative sign on the (1,1) entry cannot be verified from the documentation.",
      "The claim about the transformation matrix A in `mueller_matrix` being `Matrix([[1,0,0,1],[1,0,0,-1],[0,1,1,0],[0,-I,I,0]])` cannot be verified from the documentation.",
      "The claim about `polarizing_beam_splitter()` with default arguments equaling `Matrix([[1, 0, 0, 0], [0, 0, 0, -I], [0, 0, 1, 0], [0, -I, 0, 0]])` cannot be verified from the documentation.",
      "The claim about `half_wave_retarder(0) == Matrix([[-I, 0], [0, I]])` cannot be verified from the documentation.",
      "The claim about 11 public functions cannot be verified - only 5 functions are explicitly mentioned in the documentation (jones_vector, stokes_vector, half_wave_retarder, polarizing_beam_splitter, jones_2_stokes).",
      "The claim that `jones_vector` uses the parameter name `chi` is supported by the Stokes vector output showing `\u03c7`, and the Jones vector example confirms both `psi` and `chi` parameters.",
      "Many implementation details (imports from 7 subpackages, TensorProduct delegation, test file structure) are plausible but unverifiable from the provided documentation."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor descriptive phrases were slightly adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_62",
    "repo": "sympy",
    "question": "Why does the `jones_vector` function use `-I*sin(chi)` in the first component and `+I*sin(chi)` in the second, and how does this relate to `jones_2_stokes`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `jones_vector` function uses the specific formula `Matrix([-I*sin(chi)*sin(psi) + cos(chi)*cos(psi), I*sin(chi)*cos(psi) + sin(psi)*cos(chi)])`. A key mathematical property of this formula is that when `jones_2_stokes` is applied to a Jones vector produced by `jones_vector(psi, chi)`, the result equals `stokes_vector(psi, chi, 1, 1)`. This can be verified from the test cases and works because:\n\n1. The `jones_2_stokes` function uses S3 = `-2*im(ex*ey.conjugate())` (note the negative sign). From the test, `R = jones_vector(0, pi/4)` maps via `jones_2_stokes` to `[1, 0, 0, 1]` (positive S3), while `L = jones_vector(0, -pi/4)` maps to `[1, 0, 0, -1]` (negative S3).\n\n2. The Jones vector uses `+I*sin(chi)` in the second component and `-I*sin(chi)` in the first component. When chi > 0, the y-component has a positive imaginary coefficient while the x-component has a negative one, establishing a specific phase relationship between components.\n\n3. The `stokes_vector` uses doubled angles (2*psi, 2*chi) in its formula. The Jones vector parameterization implicitly uses single angles (psi, chi) that map to the same polarization state.\n\nThe function performs no input validation or normalization \u2014 it directly returns the trigonometric expression. This means it always produces a unit-norm Jones vector (the sum of squared magnitudes of components simplifies to 1), matching the defaults in `stokes_vector` where `p=1` and `I=1`.",
    "rubric": [
      "Explains that the formula is `Matrix([-I*sin(chi)*sin(psi) + cos(chi)*cos(psi), I*sin(chi)*cos(psi) + sin(psi)*cos(chi)])`",
      "Notes that `jones_2_stokes` uses S3 = `-2*im(ex*ey.conjugate())` with the negative sign, so positive chi yields positive S3",
      "Shows that `jones_vector(0, pi/4)` (right-hand circular) maps via `jones_2_stokes` to `[1, 0, 0, 1]` and `jones_vector(0, -pi/4)` maps to `[1, 0, 0, -1]`",
      "Explains that `jones_2_stokes(jones_vector(psi, chi))` equals `stokes_vector(psi, chi, 1, 1)` which uses doubled angles (2*psi, 2*chi)",
      "Notes the Jones vector always produces unit norm (no normalization needed), matching the defaults p=1 and I=1 in stokes_vector"
    ],
    "key_files": [
      "sympy/physics/optics/polarization.py",
      "sympy/physics/optics/tests/test_polarization.py",
      "sympy/physics/optics/__init__.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/polarization.py] sympy.physics.optics.polarization.jones_vector\nA Jones vector corresponding to a polarization ellipse with `psi` tilt,\n    and `chi` circularity.\n\n    Parameters\n    ==========\n\n    psi : numeric type or SymPy Symbol\n        The tilt of the polarization relative to the `x` axis.\n\n    chi : numeric type or SymPy Symbol\n        The angle adjacent to the mayor axis of the polarization ellipse.\n\n\n    Returns\n    =======\n\n    Matrix :\n        A Jones vector.\n\n    Examples\n    ========\n\n    The axes on the Poincar\u00e9 sphere.\n\n    >>> from sympy import pprint, symbols, pi\n    >>> from sympy.physics.optics.polarization import jones_vector\n    >>> psi, chi = symbols(\"psi, chi\", real=True)\n\n    A general Jones vector.\n\n    >>> pprint(jones_vector(psi, chi), use_unicode=True)\n    \u23a1-\u2148\u22c5sin(\u03c7)\u22c5sin(\u03c8) + cos(\u03c7)\u22c5cos(\u03c8)\u23a4\n    \u23a2                                \u23a5\n    \u23a3\u2148\u22c5sin(\u03c7)\u22c5cos(\u03c8) + sin(\u03c8)\u22c5cos(\u03c7) \u23a6\n\n    Horizontal polarization.\n\n    >>> pprint(jones_vector(0, 0), use_unicode=True)\n    \u23a11\u23a4\n    \u23a2 \u23a5\n    \u23a30\u23a6\n\n    Vertical polarization.\n\n    >>> pprint(jones_vector(pi/2, 0), use_unicode=True)\n    \u23a10\u23a4\n    \u23a2 \u23a5\n    \u23a31\u23a6\n\n    Diagonal polarization.\n\n    >>> pprint(jones_vector(pi/4, 0), use_unicode=True)\n    \u23a1\u221a2\u23a4\n    \u23a2\u2500\u2500\u23a5\n    \u23a22 \u23a5\n    \u23a2  \u23a5\n    \u23a2\u221a2\u23a5\n    \u23a2\u2500\u2500\u23a5\n    \u23a32 \u23a6\n\n    Anti-diagonal polarization.\n\n    >>> pprint(jones_vector(-pi/4, 0), use_unicode=True)\n    \u23a1 \u221a2 \u23a4\n    \u23a2 \u2500\u2500 \u23a5\n    \u23a2 2  \u23a5\n    \u23a2    \u23a5\n    \u23a2-\u221a2 \u23a5\n    \u23a2\u2500\u2500\u2500\u2500\u23a5\n    \u23a3 2  \u23a6\n\n    Right-hand circular polarization.\n\n    >>> pprint(jones_vector(0, pi/4), use_unicode=True)\n    \u23a1 \u221a2 \u23a4\n    \u23a2 \u2500\u2500 \u23a5\n    \u23a2 2  \u23a5\n    \u23a2    \u23a5\n    \u23a2\u221a2\u22c5\u2148\u23a5\n    \u23a2\u2500\u2500\u2500\u2500\u23a5\n    \u23a3 2  \u23a6\n\n    Left-hand circular polarization.\n\n    >>> pprint(jones_vector(0, -pi/4), use_unicode=True)\n    \u23a1  \u221a2  \u23a4\n    \u23a2  \u2500\u2500  \u23a5\n    \u23a2  2   \u23a5\n    \u23a2      \u23a5\n    \u23a2-\u221a2\u22c5\u2148 \u23a5\n    \u23a2\u2500\u2500\u2500\u2500\u2500\u2500\u23a5\n    \u23a3  2   \u23a6",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about `jones_2_stokes` using `S3 = -2*im(ex*ey.conjugate())` with a negative sign is not supported by the provided documentation \u2014 it references behavior of other functions not documented here.",
      "The claim about `stokes_vector(psi, chi, 1, 1)` and the relationship between `jones_vector` and `stokes_vector` via `jones_2_stokes` is not documented here \u2014 this is an extrapolation from knowledge of related functions.",
      "The claim about `stokes_vector` using doubled angles (2*psi, 2*chi) is not documented in the provided chunk.",
      "The claim that the function 'performs no input validation or normalization' is plausible but not explicitly stated in the documentation.",
      "The claim that it 'always produces a unit-norm Jones vector' is plausible given the trigonometric formula but not explicitly stated in the documentation.",
      "The core formula claim `Matrix([-I*sin(chi)*sin(psi) + cos(chi)*cos(psi), I*sin(chi)*cos(psi) + sin(psi)*cos(chi)])` is correctly supported by the documentation's general Jones vector output.",
      "The description of the parameter `chi` as 'The angle adjacent to the mayor axis' matches the documentation (noting the doc has 'mayor' which appears to be a typo for 'major')."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several physics interpretation claims (Poincar\u00e9 sphere, right-hand circular polarization, fully polarized light) rely on documentation rather than stripped code, but the core mathematical relationships are all verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_60",
    "repo": "sympy",
    "question": "How does the `Medium` class use its `__new__` method to dispatch to subclasses, and how do those subclasses avoid infinite recursion?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Medium` class in `sympy/physics/optics/medium.py` implements a factory pattern via its `__new__` method. Rather than always creating a `Medium` instance directly, `Medium.__new__` dispatches to one of two concrete subclasses\u2014`MediumN` or `MediumPP`\u2014depending on which parameters are provided:\n\n**Factory dispatching logic in `Medium.__new__`:**\n1. If `n` is provided alone (no permittivity/permeability), it returns a `MediumN(name, n)` instance.\n2. If `n` is provided with permittivity but not permeability, it computes `permeability = n**2/(c**2*permittivity)` and returns `MediumPP(name, permittivity, permeability)`.\n3. If `n` is provided with permeability but not permittivity, it computes `permittivity = n**2/(c**2*permeability)` and returns `MediumPP(name, permittivity, permeability)`.\n4. If both permittivity and permeability are provided (without `n`), returns `MediumPP(name, permittivity, permeability)`.\n5. If neither permittivity, permeability, nor `n` is provided (default case), it uses vacuum defaults `_e0mksa` and `_u0mksa` (pre-converted at module level) and returns `MediumPP(name, _e0mksa, _u0mksa)`.\n6. Providing all three raises `ValueError`.\n\n**Class hierarchy:**\n- `Medium` inherits from `sympy.core.basic.Basic` (not `Expr`).\n- `MediumN` and `MediumPP` both inherit from `Medium`.\n- Both subclasses' `__new__` methods call `super(Medium, cls).__new__(cls, ...)`, which skips `Medium.__new__` (the factory) and goes directly to `Basic.__new__`, avoiding infinite recursion.\n- The name argument is wrapped in `sympy.core.symbol.Str` (an `Atom` subclass, not `Symbol`).\n\n**Key property implementations:**\n- `MediumPP.n` computes refractive index as `c * sqrt(self.permittivity * self.permeability)` (stored in args[1] and args[2]).\n- `MediumN.n` simply returns `self.args[1]`.\n- `Medium.speed` is `c / self.n` (uses the `n` property from whichever subclass).\n- `Medium.refractive_index` is `c / self.speed`, which simplifies to `self.n`.\n- `MediumPP.intrinsic_impedance` is `sqrt(self.permeability / self.permittivity)` \u2014 this property only exists on `MediumPP`, not on `MediumN`.\n\n**Module-level constants:**\n- `c = speed_of_light.convert_to(meter/second)` converts the speed of light to meter/second units.\n- `_e0mksa = e0.convert_to(ampere**2*second**4/(kilogram*meter**3))` \u2014 where `e0` is `vacuum_permittivity` (a `PhysicalConstant` instance, visible in unit_definitions.py).\n- `_u0mksa = u0.convert_to(meter*kilogram/(ampere**2*second**2))` \u2014 where `u0` is `vacuum_permeability` (a `PhysicalConstant` instance, visible in unit_definitions.py).\n- `e0` and `u0` are `PhysicalConstant` instances (a subclass of `Quantity`).\n\n**Integration with optics utilities:**\n- The helper function `refractive_index_of_medium(medium)` in `utils.py` checks `isinstance(medium, Medium)` and accesses `medium.refractive_index`; otherwise it `sympify`s the input. This is used by `refraction_angle`, `fresnel_coefficients`, `brewster_angle`, and `critical_angle`. Additionally, `lens_makers_formula` directly checks `isinstance(n_lens, Medium)` and accesses `.refractive_index`.\n\n**Notable bug on line 88:** The condition `elif permittivity is not None and permittivity is not None:` checks `permittivity` twice instead of checking `permittivity is not None and permeability is not None`. This means when both permittivity and permeability are provided along with `n`, the error-raising branch may not trigger correctly if `permittivity` is `None` (though in practice this branch is only reached when `n is not None` and neither of the first two conditions matched, so `permittivity` would always be non-None at that point, making the bug latent).\n\n**Contrast with TWave:** `TWave` in `waves.py` inherits from `Expr` (not `Basic`), takes a scalar `n` parameter (defaulting to `Symbol('n')`), and computes wavelength as `c/(self.frequency*self.n)` with speed as `self.wavelength*self.frequency` (simplifying to `c/self.n`), while `Medium` encapsulates the physical constants permittivity/permeability and computes speed from them.",
    "rubric": [
      "Explains that Medium.__new__ acts as a factory, returning MediumN or MediumPP instances based on which parameters (n, permittivity, permeability) are provided",
      "Describes the dispatching logic: n alone \u2192 MediumN; n + one of permittivity/permeability \u2192 computes the missing one and returns MediumPP; both permittivity and permeability \u2192 MediumPP; no args \u2192 MediumPP with vacuum defaults",
      "Explains that MediumN and MediumPP call super(Medium, cls).__new__(cls, ...) which skips Medium.__new__ and goes to Basic.__new__, avoiding infinite recursion",
      "Notes that Medium inherits from Basic (not Expr), and both MediumN and MediumPP inherit from Medium",
      "Mentions that MediumPP.n computes refractive index as c * sqrt(permittivity * permeability), while MediumN.n simply returns self.args[1]",
      "Mentions the bug on line 88 where permittivity is checked twice instead of checking permeability"
    ],
    "key_files": [
      "sympy/physics/optics/medium.py",
      "sympy/physics/optics/utils.py",
      "sympy/physics/optics/waves.py",
      "sympy/core/symbol.py",
      "sympy/physics/units/definitions/unit_definitions.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/medium.py] sympy.physics.optics.medium.Medium\nsympy.physics.optics.medium.Medium:\n  This class represents an optical medium. The prime reason to implement this is\n    to facilitate refraction, Fermat's principle, etc.\n\n    Explanation\n    ===========\n\n    An optical medium is a material through which electromagnetic waves propagate.\n    The permittivity and permeability of the medium define how electromagnetic\n    waves propagate in it.\n\n\n    Parameters\n    ==========\n\n    name: string\n        The display name of the Medium.\n\n    permittivity: Sympifyable\n        Electric permittivity of the space.\n\n    permeability: Sympifyable\n        Magnetic permeability of the space.\n\n    n: Sympifyable\n        Index of refraction of the medium.\n\n\n    Examples\n    ========\n\n    >>> from sympy.abc import epsilon, mu\n    >>> from sympy.physics.optics import Medium\n    >>> m1 = Medium('m1')\n    >>> m2 = Medium('m2', epsilon, mu)\n    >>> m1.intrinsic_impedance\n    149896229*pi*kilogram*meter**2/(1250000*ampere**2*second**3)\n    >>> m2.refractive_index\n    299792458*meter*sqrt(epsilon*mu)/second\n\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Optical_medium\n\nsympy.physics.optics.medium.Medium.speed:\n  Returns speed of the electromagnetic wave travelling in the medium.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import Medium\n        >>> m = Medium('m')\n        >>> m.speed\n        299792458*meter/second\n        >>> m2 = Medium('m2', n=1)\n        >>> m.speed == m2.speed\n        True\n\nsympy.physics.optics.medium.Medium.refractive_index:\n  Returns refractive index of the medium.\n\n        Examples\n        ========\n\n        >>> from sympy.physics.optics import Medium\n        >>> m = Medium('m')\n        >>> m.refractive_index\n        1",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most claims about internal implementation details (factory pattern, MediumN/MediumPP subclasses, module-level constants, bug on line 88, TWave contrast) go well beyond what the documentation provides and cannot be verified from it alone, but they are plausible code-level details rather than contradictions.",
      "The claim that Medium.refractive_index is 'c / self.speed, which simplifies to self.n' is consistent with the documentation showing m.refractive_index returning 1 for a default medium and '299792458*meter*sqrt(epsilon*mu)/second' for a medium with custom epsilon/mu.",
      "The claim that Medium.speed returns 'c / self.n' is consistent with the documentation showing m.speed as '299792458*meter/second' for default medium (n=1).",
      "The claim about intrinsic_impedance returning '149896229*pi*kilogram*meter**2/(1250000*ampere**2*second**3)' for default medium is consistent with the documentation example."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor adjustments needed for documentation-derived explanations about Str's purpose and exact numeric values.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_63",
    "repo": "sympy",
    "question": "Where does data flow through the `refraction_angle` function, and how does it determine what type of result to return?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `refraction_angle` function in `sympy/physics/optics/utils.py` follows a specific data/control flow path:\n\n1. **Entry point and medium handling**: It first calls `refractive_index_of_medium(medium1)` and `refractive_index_of_medium(medium2)` (a helper which checks if the input is a `Medium` instance (returning `.refractive_index`) or else calls `sympify()` on it.\n\n2. **Angle vs. ray branching**: The function attempts `float(incident)` in a try/except block. If this succeeds (incident is a number), it treats it as an angle of incidence and uses Snell's law via `asin(n1*sin(angle_of_incidence)/n2)`. In this path, it first computes the critical angle by calling `critical_angle(medium1, medium2)` (which itself raises `ValueError` if n2 > n1, caught here) and raises `ValueError('Ray undergoes total internal reflection')` if the angle exceeds critical angle.\n\n3. **Vector/Ray path**: When incident is a ray/vector, the function computes the refracted direction using the vector form of Snell's law:\n   - `eta = n1/n2` (relative refractive index)\n   - `c1 = -_incident.dot(_normal)` (cosine of incidence angle)\n   - `cs2 = 1 - eta**2*(1 - c1**2)` (cosine squared of refraction angle)\n   - If `cs2.is_negative` (total internal reflection), returns `S.Zero`\n   - Otherwise: `drs = eta*_incident + (eta*c1 - sqrt(cs2))*_normal`\n\n4. **Output type determination**: A `return_ray` flag is set to `True` only when: (a) a `Plane` is provided and the incident is a `Ray3D`, or (b) the normal is a `Ray3D` that intersects the incident ray. When `return_ray` is True, the function returns `Ray3D(intersection_pt, direction_ratio=drs)`. Otherwise it returns the raw `Matrix` `drs`.\n\n5. **Intersection point computation**: When a `Plane` is provided, it uses `plane.intersection(incident)[0]` from `sympy.geometry.plane.Plane.intersection`. When the normal is a `Ray3D`, it uses `intersection(incident, normal)` from `sympy.geometry.util.intersection` which delegates to `entities[0].intersection(entities[1])`.\n\n6. **Downstream consumer**: The `deviation` function calls `refraction_angle` and implicitly returns `None` when TIR occurs (when `refracted == 0`/`S.Zero`), since the `if refracted != 0:` block is not entered.\n\n7. **Medium class hierarchy**: The `Medium` base class delegates to `MediumN` (when only `n` is specified) or `MediumPP` (when permittivity/permeability are given). `MediumN.n` returns `self.args[1]` directly, while `MediumPP.n` computes `c*sqrt(self.permittivity*self.permeability)`. The base class `refractive_index` property computes `c/self.speed` where `speed = c/self.n`, effectively returning `self.n`.\n\n8. **Ray3D construction**: When `refraction_angle` returns a `Ray3D`, it uses `Ray3D(intersection_pt, direction_ratio=drs)` where `drs` is a 3\u00d71 Matrix. The `Ray3D.__new__` constructor checks `len(direction_ratio) == 3` and creates the second point as `Point3D(p1.x + direction_ratio[0], p1.y + direction_ratio[1], p1.z + direction_ratio[2])`. For a 3\u00d71 Matrix, `len()` returns `rows*cols = 3`.",
    "rubric": [
      "Explains that medium inputs are converted to refractive indices via refractive_index_of_medium helper (which checks for Medium instance or sympifies)",
      "Describes the angle vs. ray/vector branching based on whether float(incident) succeeds or raises an exception",
      "For the angle path: mentions Snell's law via asin(n1*sin(angle)/n2) and critical_angle check that raises ValueError for TIR",
      "For the vector path: describes the vector Snell's law formula using eta=n1/n2, c1=-incident.dot(normal), cs2=1-eta\u00b2(1-c1\u00b2), and returns S.Zero if cs2 is negative",
      "Explains the return_ray flag logic: True when a Plane is provided with a Ray3D incident, or when normal is a Ray3D intersecting the incident",
      "When return_ray is True, returns Ray3D(intersection_pt, direction_ratio=drs); otherwise returns the raw Matrix drs",
      "Mentions how intersection points are computed: via plane.intersection(incident)[0] or geometry util intersection for Ray3D normals",
      "Notes that the deviation function downstream treats a return of S.Zero (TIR case) by implicitly returning None"
    ],
    "key_files": [
      "sympy/physics/optics/utils.py",
      "sympy/physics/optics/medium.py",
      "sympy/geometry/line.py",
      "sympy/geometry/util.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/utils.py] sympy.physics.optics.utils.refraction_angle\nThis function calculates transmitted vector after refraction at planar\n    surface. ``medium1`` and ``medium2`` can be ``Medium`` or any sympifiable object.\n    If ``incident`` is a number then treated as angle of incidence (in radians)\n    in which case refraction angle is returned.\n\n    If ``incident`` is an object of `Ray3D`, `normal` also has to be an instance\n    of `Ray3D` in order to get the output as a `Ray3D`. Please note that if\n    plane of separation is not provided and normal is an instance of `Ray3D`,\n    ``normal`` will be assumed to be intersecting incident ray at the plane of\n    separation. This will not be the case when `normal` is a `Matrix` or\n    any other sequence.\n    If ``incident`` is an instance of `Ray3D` and `plane` has not been provided\n    and ``normal`` is not `Ray3D`, output will be a `Matrix`.\n\n    Parameters\n    ==========\n\n    incident : Matrix, Ray3D, sequence or a number\n        Incident vector or angle of incidence\n    medium1 : sympy.physics.optics.medium.Medium or sympifiable\n        Medium 1 or its refractive index\n    medium2 : sympy.physics.optics.medium.Medium or sympifiable\n        Medium 2 or its refractive index\n    normal : Matrix, Ray3D, or sequence\n        Normal vector\n    plane : Plane\n        Plane of separation of the two media.\n\n    Returns\n    =======\n\n    Returns an angle of refraction or a refracted ray depending on inputs.\n\n    Examples\n    ========\n\n    >>> from sympy.physics.optics import refraction_angle\n    >>> from sympy.geometry import Point3D, Ray3D, Plane\n    >>> from sympy.matrices import Matrix\n    >>> from sympy import symbols, pi\n    >>> n = Matrix([0, 0, 1])\n    >>> P = Plane(Point3D(0, 0, 0), normal_vector=[0, 0, 1])\n    >>> r1 = Ray3D(Point3D(-1, -1, 1), Point3D(0, 0, 0))\n    >>> refraction_angle(r1, 1, 1, n)\n    Matrix([\n    [ 1],\n    [ 1],\n    [-1]])\n    >>> refraction_angle(r1, 1, 1, plane=P)\n    Ray3D(Point3D(0, 0, 0), Point3D(1, 1, -1))\n\n    With different index of refraction of the two media\n\n    >>> n1, n2 = symbols('n1, n2')\n    >>> refraction_angle(r1, n1, n2, n)\n    Matrix([\n    [                                n1/n2],\n    [                                n1/n2],\n    [-sqrt(3)*sqrt(-2*n1**2/(3*n2**2) + 1)]])\n    >>> refraction_angle(r1, n1, n2, plane=P)\n    Ray3D(Point3D(0, 0, 0), Point3D(n1/n2, n1/n2, -sqrt(3)*sqrt(-2*n1**2/(3*n2**2) + 1)))\n    >>> round(refraction_angle(pi/6, 1.2, 1.5), 5)\n    0.41152",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not provide source code details like line numbers, try/except blocks, or internal variable names (eta, c1, cs2, drs, return_ray flag). These are plausible implementation details but cannot be verified from the documentation alone.",
      "The claim about `critical_angle(medium1, medium2)` raising ValueError if n2 > n1 is an implementation detail not mentioned in the documentation.",
      "The claim that total internal reflection returns `S.Zero` is not mentioned in the documentation but is a plausible implementation detail.",
      "The Medium class hierarchy details (MediumN, MediumPP, args[1], permittivity*permeability) are not in the documentation and cannot be verified.",
      "The Ray3D constructor details (direction_ratio parameter, len() behavior for 3\u00d71 Matrix) are implementation details not in the documentation.",
      "The claim about the `deviation` function's behavior with TIR is not covered in this documentation chunk.",
      "The overall behavior described (Snell's law for angles, vector form for rays, returning Matrix vs Ray3D depending on inputs) is consistent with the documentation's description and examples."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped source code; only a minor line number reference was adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_64",
    "repo": "sympy",
    "question": "How does the `fresnel_coefficients` function handle total internal reflection differently from normal refraction?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `fresnel_coefficients` function in `sympy/physics/optics/utils.py` implements the Fresnel equations through a two-branch algorithm that distinguishes between normal refraction and total internal reflection (TIR).\n\n**Input validation:** The function validates the angle of incidence using the condition `not 0 <= 2*angle_of_incidence < pi`, which is equivalent to checking that the angle is in `[0, pi/2)`. This differs from the `refraction_angle` function, which uses `not 0.0 <= angle_of_incidence < pi*0.5` \u2014 a float comparison rather than symbolic multiplication by 2.\n\n**Refractive index extraction:** Both `medium1` and `medium2` are passed through the helper function `refractive_index_of_medium()`, which checks `isinstance(medium, Medium)` and extracts `medium.refractive_index` if true, otherwise calls `sympify(medium)` on the raw value. The `Medium` class is actually a factory: its `__new__` method dispatches to either `MediumN` (when only `n` is specified) or `MediumPP` (when permittivity/permeability are specified). The `refractive_index` property in the base `Medium` class computes `c/self.speed`, while `MediumN` stores `n` directly as `self.args[1]` and `MediumPP` computes it as `c*sqrt(self.permittivity*self.permeability)`.\n\n**TIR detection:** The function calls `critical_angle(n1, n2)` inside a try/except block. The `critical_angle` function raises `ValueError('Total internal reflection impossible for n1 < n2')` when `n2 > n1`, which the `fresnel_coefficients` function catches and sets `angle_of_total_internal_reflection_onset = None`. If a valid critical angle exists AND the incidence angle exceeds it, the TIR branch is executed.\n\n**Normal refraction branch:** The refraction angle is computed via `asin(n1*sin(angle_of_incidence)/n2)`. The four coefficients are then computed using trigonometric formulas:\n- `R_p = tan(\u03b8_i - \u03b8_r) / tan(\u03b8_i + \u03b8_r)`\n- `R_s = -sin(\u03b8_i - \u03b8_r) / sin(\u03b8_i + \u03b8_r)`\n- `T_p = 2*sin(\u03b8_r)*cos(\u03b8_i) / (sin(\u03b8_i + \u03b8_r)*cos(\u03b8_i - \u03b8_r))`\n- `T_s = 2*sin(\u03b8_r)*cos(\u03b8_i) / sin(\u03b8_i + \u03b8_r)`\nReturned as `[R_p, R_s, T_p, T_s]`.\n\n**TIR branch:** When total internal reflection occurs, the ratio `n = n2/n1` is computed. Two complex reflection coefficients are calculated using the imaginary unit `I` (imported from `sympy.core.numbers`) and `sympy.polys.polytools.cancel` is applied to simplify the complex fractions:\n- `R_s = cancel((cos(\u03b8_i) - I*sqrt(sin\u00b2(\u03b8_i) - n\u00b2)) / (cos(\u03b8_i) + I*sqrt(sin\u00b2(\u03b8_i) - n\u00b2)))`\n- `R_p = cancel((n\u00b2*cos(\u03b8_i) - I*sqrt(sin\u00b2(\u03b8_i) - n\u00b2)) / (n\u00b2*cos(\u03b8_i) + I*sqrt(sin\u00b2(\u03b8_i) - n\u00b2)))`\nReturned as `[R_p, R_s]` (only two coefficients, no transmission).\n\n**Return order:** In both branches, `R_p` comes first, then `R_s`, matching the return order `[R_p, R_s, ...]`.\n\n**Test patterns:** The test file uses a helper `ae = lambda a, b, n: comp(a, b, 10**-n)` for approximate equality checking. The TIR test case decomposes complex results using `.as_real_imag()` and compares real and imaginary parts separately at 5 decimal places of precision.",
    "rubric": [
      "Explains that TIR detection uses critical_angle() inside a try/except block, catching ValueError when n2 > n1",
      "Describes that in the normal refraction branch, four coefficients are returned: R_p, R_s, T_p, T_s using trigonometric Fresnel formulas",
      "Describes that in the TIR branch, only two complex reflection coefficients (R_p, R_s) are returned with no transmission coefficients",
      "Mentions that the TIR branch uses the imaginary unit I and cancel() from sympy.polys.polytools to simplify the complex expressions",
      "Notes the TIR branch computes n = n2/n1 and uses sqrt(sin\u00b2(\u03b8) - n\u00b2) in the numerator and denominator of the reflection coefficients",
      "Mentions the angle validation checks that 0 <= 2*angle_of_incidence < pi (equivalent to [0, pi/2))"
    ],
    "key_files": [
      "sympy/physics/optics/utils.py",
      "sympy/physics/optics/medium.py",
      "sympy/physics/optics/tests/test_utils.py"
    ],
    "source_doc": "[docstring: sympy/physics/optics/utils.py] sympy.physics.optics.utils.fresnel_coefficients\nThis function uses Fresnel equations to calculate reflection and\n    transmission coefficients. Those are obtained for both polarisations\n    when the electric field vector is in the plane of incidence (labelled 'p')\n    and when the electric field vector is perpendicular to the plane of\n    incidence (labelled 's'). There are four real coefficients unless the\n    incident ray reflects in total internal in which case there are two complex\n    ones. Angle of incidence is the angle between the incident ray and the\n    surface normal. ``medium1`` and ``medium2`` can be ``Medium`` or any\n    sympifiable object.\n\n    Parameters\n    ==========\n\n    angle_of_incidence : sympifiable\n\n    medium1 : Medium or sympifiable\n        Medium 1 or its refractive index\n\n    medium2 : Medium or sympifiable\n        Medium 2 or its refractive index\n\n    Returns\n    =======\n\n    Returns a list with four real Fresnel coefficients:\n    [reflection p (TM), reflection s (TE),\n    transmission p (TM), transmission s (TE)]\n    If the ray is undergoes total internal reflection then returns a\n    list of two complex Fresnel coefficients:\n    [reflection p (TM), reflection s (TE)]\n\n    Examples\n    ========\n\n    >>> from sympy.physics.optics import fresnel_coefficients\n    >>> fresnel_coefficients(0.3, 1, 2)\n    [0.317843553417859, -0.348645229818821,\n            0.658921776708929, 0.651354770181179]\n    >>> fresnel_coefficients(0.6, 2, 1)\n    [-0.235625382192159 - 0.971843958291041*I,\n             0.816477005968898 - 0.577377951366403*I]\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Fresnel_equations",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Almost all claims are directly verifiable from the stripped source code; only minor physics terminology labels (TM/TE) were slightly adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_65",
    "repo": "sympy",
    "question": "What is the architecture of the AntiCommutator class \u2014 its inheritance, evaluation logic, and how it dispatches to operator-specific implementations?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `AntiCommutator` class in `sympy/physics/quantum/anticommutator.py` represents the anticommutator `{A, B} = A*B + B*A` (as seen in its `doit()` fallback) that extends `sympy.core.expr.Expr` (not a quantum-specific base class like `QExpr`). It stores the anticommutator in an unevaluated form, with evaluation deferred to the `.doit()` method.\n\n**Key architectural details:**\n\n1. **Class hierarchy**: `AntiCommutator` inherits directly from `Expr`, the general SymPy expression class \u2014 it does NOT inherit from `QExpr` (the quantum expression base class). The class has `is_commutative = False` set as a class attribute.\n\n2. **`eval()` classmethod simplification logic**: The `eval()` method (called during `__new__`) handles several special cases:\n   - Returns `S.Zero` if either argument is falsy (zero/empty).\n   - Returns `Integer(2)*a**2` when both arguments are equal (`a == b`).\n   - Returns `Integer(2)*a*b` when either argument is commutative.\n   - Factors out commutative parts using `args_cnc()` (which separates commutative and non-commutative factors), then reconstructs as `Mul(*c_part) * cls(nc_a, nc_b)`.\n   - Applies canonical ordering using `a.compare(b)`: if `a.compare(b) == 1`, it swaps the arguments and returns `cls(b, a)`.\n\n3. **`doit()` method dispatch mechanism**: The `doit()` method first checks if both arguments are `Operator` instances (imported locally). If so, it calls `A._eval_anticommutator(B)`, and if that raises `NotImplementedError`, tries `B._eval_anticommutator(A)`. This dispatch is handled by `dispatch_method()` in `sympy/physics/quantum/qexpr.py`, which constructs a method name like `_eval_anticommutator_ClassName` (e.g., `_eval_anticommutator_FermionOp`) and calls it via `getattr`. If no specialized method exists or both raise `NotImplementedError`, it falls back to computing `(A*B + B*A).doit()`.\n\n4. **Kind system**: `AntiCommutator` has a `_kind_dispatcher` (a `KindDispatcher` with `commutative=True`) and a registered handler `find_op_kind` that returns `OperatorKind` when both arguments are `_OperatorKind`.\n\n5. **Adjoint handling**: The `_eval_adjoint` method creates a new `AntiCommutator(Dagger(self.args[0]), Dagger(self.args[1]))`. Since `Dagger` extends `adjoint` (from `sympy.functions.elementary.complexes`) and calls `arg.adjoint()` during construction (if `evaluate=True`), applying `Dagger` to an `AntiCommutator` triggers this `_eval_adjoint` method.\n\n6. **Concrete implementations**: Among the provided code:\n   - `IdentityOperator._eval_anticommutator`: Always returns `2 * other`.",
    "rubric": [
      "AntiCommutator inherits from Expr (not QExpr) and has is_commutative = False",
      "eval() classmethod returns S.Zero if either argument is zero/falsy",
      "eval() returns Integer(2)*a**2 when both arguments are equal",
      "eval() returns Integer(2)*a*b when either argument is commutative",
      "eval() separates commutative/non-commutative parts using args_cnc() and factors out commutative parts",
      "eval() applies canonical ordering by swapping arguments when a.compare(b) == 1",
      "doit() checks if both arguments are Operator instances and dispatches via _eval_anticommutator methods",
      "dispatch_method() constructs method names like _eval_anticommutator_ClassName and calls via getattr",
      "doit() falls back to computing (A*B + B*A).doit() if no specialized method handles the case",
      "_eval_adjoint returns AntiCommutator(Dagger(self.args[0]), Dagger(self.args[1]))"
    ],
    "key_files": [
      "sympy/physics/quantum/anticommutator.py",
      "sympy/physics/quantum/operator.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/kind.py",
      "sympy/physics/quantum/dagger.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/anticommutator.py] sympy.physics.quantum.anticommutator.AntiCommutator\nsympy.physics.quantum.anticommutator.AntiCommutator:\n  The standard anticommutator, in an unevaluated state.\n\n    Explanation\n    ===========\n\n    Evaluating an anticommutator is defined [1]_ as: ``{A, B} = A*B + B*A``.\n    This class returns the anticommutator in an unevaluated form.  To evaluate\n    the anticommutator, use the ``.doit()`` method.\n\n    Canonical ordering of an anticommutator is ``{A, B}`` for ``A < B``. The\n    arguments of the anticommutator are put into canonical order using\n    comparison operators. If ``B < A``, then ``{A, B}`` is returned as ``{B, A}``.\n\n    Parameters\n    ==========\n\n    A : Expr\n        The first argument of the anticommutator {A,B}.\n    B : Expr\n        The second argument of the anticommutator {A,B}.\n\n    Examples\n    ========\n\n    >>> from sympy import symbols\n    >>> from sympy.physics.quantum import AntiCommutator\n    >>> from sympy.physics.quantum import Operator, Dagger\n    >>> x, y = symbols('x,y')\n    >>> A = Operator('A')\n    >>> B = Operator('B')\n\n    Create an anticommutator and use ``doit()`` to multiply them out.\n\n    >>> ac = AntiCommutator(A,B); ac\n    {A,B}\n    >>> ac.doit()\n    A*B + B*A\n\n    The commutator orders it arguments in canonical order:\n\n    >>> ac = AntiCommutator(B,A); ac\n    {A,B}\n\n    Commutative constants are factored out:\n\n    >>> AntiCommutator(3*x*A,x*y*B)\n    3*x**2*y*{A,B}\n\n    Adjoint operations applied to the anticommutator are properly applied to\n    the arguments:\n\n    >>> Dagger(AntiCommutator(A,B))\n    {Dagger(A),Dagger(B)}\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Commutator\n\nsympy.physics.quantum.anticommutator.AntiCommutator.doit:\n  Evaluate anticommutator",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states canonical ordering swaps when 'B < A' (returning {B, A}), while the answer describes the mechanism as using 'a.compare(b) == 1' which is a plausible implementation detail but not explicitly stated in the docs.",
      "The answer contains extensive implementation details (class hierarchy, eval() internals, kind system, dispatch_method, FermionOp specifics, represent/qapply integration, etc.) that go far beyond what the documentation provides. These are plausible code-level details but cannot be verified from the given documentation alone.",
      "The documentation says 'The commutator orders it arguments in canonical order' which appears to be a typo for 'anticommutator' - the answer correctly describes this as anticommutator canonical ordering.",
      "Claim that AntiCommutator inherits from Expr and NOT QExpr cannot be verified from the documentation provided.",
      "Claims about FermionOp, IdentityOperator, OneQubitGate, Pauli matrices anticommutator implementations are not in the documentation.",
      "The detailed dispatch mechanism description (dispatch_method, _eval_anticommutator_ClassName pattern) is not mentioned in the documentation."
    ],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "Several claims about external files (FermionOp, Pauli, represent.py, qapply.py) and minor design rationales were removed; core code-verifiable claims retained.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_66",
    "repo": "sympy",
    "question": "Why does BosonOp only implement the [a\u2020, a] = -1 case in its commutator method and not the [a, a\u2020] = 1 direction?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `BosonOp` class's `_eval_commutator_BosonOp` method only handles the case `[a\u2020, a] = -1` (when `self` is a creation operator and `other` is an annihilation operator with the same name), returning `S.NegativeOne`. It does NOT explicitly handle `[a, a\u2020] = 1`. This works because of the **dual-dispatch fallback pattern** in `Commutator.doit()`: when `A._eval_commutator(B)` raises `NotImplementedError`, it tries `-1 * B._eval_commutator(A)`, which gives `-1 * (-1) = 1`. Additionally, `Commutator.eval()` enforces **canonical ordering** \u2014 if `a.compare(b) == 1`, it rewrites `[A, B]` as `-[B, A]`, so the stored commutator may already be in the form where the creation operator comes first.\n\nThis design has several observable characteristics:\n\n1. **Only one direction implemented**: By only implementing one direction of the commutation relation (`[a\u2020, a] = -1`), `BosonOp` relies on the fallback in `Commutator.doit()` for the symmetric case: first try `A._eval_commutator(B)`, and if that raises `NotImplementedError`, compute `-1 * B._eval_commutator(A)`.\n\n2. **Name-based dispatch via `dispatch_method`**: The `Operator._eval_commutator` method delegates to `dispatch_method` in `qexpr.py`, which dynamically constructs a method name like `_eval_commutator_BosonOp` by appending the class name of `other`. This allows type-specific commutation rules without explicit type-checking \u2014 a pattern used by both `BosonOp` and `FermionOp`.\n\n3. **Cross-species commutation**: `BosonOp._eval_commutator_FermionOp` returns `S.Zero`, and symmetrically `FermionOp._eval_commutator_BosonOp` also returns `S.Zero`, meaning bosonic and fermionic operators commute.\n\n4. **The `independent` hints pattern**: For operators with different names, the commutator returns `None` by default, but if `hints['independent']` is `True`, it returns `S.Zero`. When `None` is returned, the `doit()` method falls through to computing `A*B - B*A` directly.\n\n5. **`FermionOp` has `_eval_power` but `BosonOp` does not**: `FermionOp` implements `_eval_power` to return `S.Zero` for integer `n > 1`, while `BosonOp` has no such method, meaning bosonic operators can be raised to any power.\n\n6. **Dagger integration**: `Dagger.__new__` calls `arg.adjoint()` if available, and `BosonOp._eval_adjoint` creates a new `BosonOp` with the `is_annihilation` flag negated. The annihilation flag is stored as `Integer(1)` or `Integer(0)` in `args[1]` (converted in `__new__` via `Integer(args[1])`).",
    "rubric": [
      "Explains that Commutator.doit() has a dual-dispatch fallback: it tries A._eval_commutator(B), and if NotImplementedError is raised, computes -1 * B._eval_commutator(A), yielding -1 * (-1) = 1",
      "Mentions that Commutator.eval() enforces canonical ordering via a.compare(b), rewriting [A, B] as -[B, A] when A > B, so the creation operator may already come first",
      "Describes the dispatch_method mechanism in qexpr.py that dynamically constructs method names like _eval_commutator_BosonOp from the class name of the other operand",
      "Notes that when _eval_commutator_BosonOp returns None (e.g., different-named operators without the 'independent' hint), doit() falls through to computing A*B - B*A directly",
      "Mentions the cross-species behavior: BosonOp._eval_commutator_FermionOp and FermionOp._eval_commutator_BosonOp both return Zero"
    ],
    "key_files": [
      "sympy/physics/quantum/boson.py",
      "sympy/physics/quantum/commutator.py",
      "sympy/physics/quantum/operator.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/fermion.py",
      "sympy/physics/quantum/dagger.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/boson.py] sympy.physics.quantum.boson.BosonOp\nsympy.physics.quantum.boson.BosonOp:\n  A bosonic operator that satisfies [a, Dagger(a)] == 1.\n\n    Parameters\n    ==========\n\n    name : str\n        A string that labels the bosonic mode.\n\n    annihilation : bool\n        A bool that indicates if the bosonic operator is an annihilation (True,\n        default value) or creation operator (False)\n\n    Examples\n    ========\n\n    >>> from sympy.physics.quantum import Dagger, Commutator\n    >>> from sympy.physics.quantum.boson import BosonOp\n    >>> a = BosonOp(\"a\")\n    >>> Commutator(a, Dagger(a)).doit()\n    1",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes BosonOp's basic interface (parameters: name and annihilation) and shows that Commutator(a, Dagger(a)).doit() == 1. The generated answer makes extensive claims about internal implementation details (specific line numbers, method dispatch patterns, _eval_commutator_BosonOp returning S.NegativeOne, _eval_commutator_FermionOp, independent hints, _eval_power absence, Dagger.__new__ behavior, Integer conversion in args[1]) that cannot be verified from the provided documentation alone.",
      "The core claim that [a, Dagger(a)] == 1 is consistent with the documentation's example showing Commutator(a, Dagger(a)).doit() == 1.",
      "The claim about dual-dispatch fallback pattern and canonical ordering are plausible implementation details but cannot be verified from the given documentation.",
      "Line number references (e.g., 'lines 218-224 of commutator.py', 'line 164-166', 'line 398-410', 'line 83', 'line 87-88', 'line 89-90', 'line 92-93') are very specific claims that could be hallucinated since they cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals needed for physics rationale descriptions and line number references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_67",
    "repo": "sympy",
    "question": "Where are Wigner 3-j symbols implemented in the codebase, and how do the different implementations relate to each other?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `Wigner3j` class in `sympy/physics/quantum/cg.py` inherits from `sympy.core.expr.Expr`, and has `is_commutative = True` set as a class attribute. There are actually **two distinct `Wigner3j` classes** in the codebase:\n\n1. **`sympy.physics.quantum.cg.Wigner3j`** \u2014 inherits from `Expr`, takes 6 args (j1, m1, j2, m2, j3, m3), and its `doit()` raises a `ValueError(\"Coefficients must be numerical\")` if `self.is_symbolic` is True.\n\n2. **`sympy.physics.wigner.Wigner3j`** \u2014 inherits from `Function`, also takes 6 args, but its `doit()` does NOT raise an error for symbolic parameters; instead it simply returns `self`.\n\nThe `Wigner3j.doit()` method (in `cg.py`) delegates computation to the `wigner_3j()` function in `sympy/physics/wigner.py`. This function first doubles all inputs via `_doubled_int()`, then applies several zero-checks: `m1+m2+m3 != 0`, triangle inequality failure via `_check_dj_couple()`, `|mi| > ji`, and parity check `(dji - dmi) % 2 != 0`. The actual computation iterates with `math.comb` (binomial coefficients from Python's math module) in an alternating sum `sumres = ti - sumres`.\n\nThe `CG` (Clebsch-Gordan) class is a subclass of the `quantum.cg.Wigner3j` class, inheriting its `__new__`, property accessors, and `is_symbolic` check. `CG.doit()` calls `clebsch_gordan()` from `sympy.physics.wigner`, which internally calls `wigner_3j()` with `-m_3` (negated third m parameter) and multiplies the result by `(-1)**(j_1 - j_2 + m_3) * sqrt(2*j_3 + 1)`. CG also sets `precedence = PRECEDENCE[\"Pow\"] - 1` (which equals 59) for printing purposes.\n\nThe `cg_simp()` function, also defined in the same module, simplifies sums/products of CG coefficients by checking against identities encoded in helper functions `_check_varsh_871_1`, `_check_varsh_871_2`, and `_check_varsh_872_9`, using Wild pattern matching.\n\nThe `Wigner3j._pretty()` method renders a 2\u00d73 matrix-like layout in parentheses: top row shows j1, j2, j3 and bottom row shows m1, m2, m3. Note that `_latex()` reorders the arguments, printing j1, j2, j3 on the first row and m1, m2, m3 on the second row (using the order j1, j2, j3, m1, m2, m3 when passed to the format string).\n\nThe `dot_rot_grad_Ynm()` function in `sympy/physics/wigner.py` uses the `wigner.Wigner3j(Function)` class symbolically within a `Sum` over spherical harmonics.",
    "rubric": [
      "Identifies two distinct Wigner3j classes: one in sympy/physics/quantum/cg.py (inherits from Expr) and one in sympy/physics/wigner.py (inherits from Function)",
      "Notes that the quantum.cg.Wigner3j.doit() delegates to the wigner_3j() function in sympy/physics/wigner.py",
      "Explains that CG (Clebsch-Gordan) class is a subclass of quantum.cg.Wigner3j",
      "Describes that CG.doit() calls clebsch_gordan() which internally calls wigner_3j() with negated m3 and multiplies by (-1)**(j1-j2+m3)*sqrt(2*j3+1)",
      "Notes that quantum.cg.Wigner3j.doit() raises ValueError for symbolic coefficients, while wigner.Wigner3j.doit() simply returns self",
      "Mentions that wigner_3j() uses _doubled_int() and applies zero/triangle/parity checks before computing an alternating sum with math.comb"
    ],
    "key_files": [
      "sympy/physics/quantum/cg.py",
      "sympy/physics/wigner.py",
      "sympy/printing/precedence.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/cg.py] sympy.physics.quantum.cg.Wigner3j\nsympy.physics.quantum.cg.Wigner3j:\n  Class for the Wigner-3j symbols.\n\n    Explanation\n    ===========\n\n    Wigner 3j-symbols are coefficients determined by the coupling of\n    two angular momenta. When created, they are expressed as symbolic\n    quantities that, for numerical parameters, can be evaluated using the\n    ``.doit()`` method [1]_.\n\n    Parameters\n    ==========\n\n    j1, m1, j2, m2, j3, m3 : Number, Symbol\n        Terms determining the angular momentum of coupled angular momentum\n        systems.\n\n    Examples\n    ========\n\n    Declare a Wigner-3j coefficient and calculate its value\n\n        >>> from sympy.physics.quantum.cg import Wigner3j\n        >>> w3j = Wigner3j(6,0,4,0,2,0)\n        >>> w3j\n        Wigner3j(6, 0, 4, 0, 2, 0)\n        >>> w3j.doit()\n        sqrt(715)/143\n\n    See Also\n    ========\n\n    CG: Clebsch-Gordan coefficients\n\n    References\n    ==========\n\n    .. [1] Varshalovich, D A, Quantum Theory of Angular Momentum. 1988.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage: module description as 'Clebsch-Gordan coefficients module', reference to [Wei99] algorithm, and 'half-integer quantum numbers' rationale are documentation-only; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_68",
    "repo": "sympy",
    "question": "How is the CG (Clebsch-Gordan) class implemented, and what happens when you call doit() on it?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `CG` class (Clebsch-Gordan coefficient) is implemented in `sympy/physics/quantum/cg.py` as a subclass of `Wigner3j`, which itself inherits from `sympy.core.expr.Expr`. Here's how the implementation works:\n\n**Class Hierarchy & Construction:**\n- `CG` inherits from `Wigner3j` and does NOT define its own `__new__` method. It reuses `Wigner3j.__new__`, which calls `sympify` on all 6 arguments (j1, m1, j2, m2, j3, m3) and delegates to `Expr.__new__`.\n- `Wigner3j` sets `is_commutative = True`, which CG inherits.\n- CG sets `precedence = PRECEDENCE[\"Pow\"] - 1`, which equals `59` (since `PRECEDENCE[\"Pow\"]` is `60`). This precedence attribute is used by the printing system for parenthesization.\n- The `is_symbolic` property (inherited from `Wigner3j`) checks `not all(arg.is_number for arg in self.args)` \u2014 if any argument is symbolic, `doit()` raises a `ValueError(\"Coefficients must be numerical\")`.\n\n**CG.doit() Algorithm:**\n1. `CG.doit()` delegates to `clebsch_gordan(j1, j2, j3, m1, m2, m3)` from `sympy.physics.wigner`.\n2. `clebsch_gordan()` converts the CG coefficient to a Wigner 3j symbol using the relation: `(-1)^(j1-j2+m3) * sqrt(2*j3+1) * wigner_3j(j1, j2, j3, m1, m2, -m3)` \u2014 note the negated `m3` passed to `wigner_3j`.\n3. `wigner_3j()` implementation:\n   - All arguments are first doubled to integers via `_doubled_int()` (which multiplies by 2 and converts to Python int).\n   - Four early-return zero conditions are checked: (a) `m1+m2+m3 \u2260 0`, (b) triangle inequality failure via `_check_dj_couple`, (c) `|mi| > ji` for any i, (d) `ji-mi` not even (in doubled representation).\n   - The core sum uses `math.comb` (Python's built-in combinatorial function) in an alternating-sign accumulation pattern: `sumres = ti - sumres` (not `sumres += (-1)**i * ti`), which implements the alternating sum without explicit sign tracking.\n   - The result is factored as `phase * sqrt(res_num/res_den) * sumres` where `phase = (-1)**(dj1 + (dj3+dm3)//2 + imax)`.\n   - If `prec` is given, `res.evalf(prec)` is called to return a floating-point result.\n\n**In contrast, `Wigner3j.doit()`** calls `wigner_3j(j1, j2, j3, m1, m2, m3)` directly without any sign/factor transformation.\n\n**CG Simplification (`cg_simp`):**\n- `cg_simp` recursively simplifies expressions containing CG coefficients.\n- For `Add` expressions, it separates CG and non-CG terms, then applies three functions: `_check_varsh_871_1`, `_check_varsh_871_2`, and `_check_varsh_872_9`.\n- For `Sum` expressions, it applies `_check_varsh_sum_871_1`, `_check_varsh_sum_871_2`, and `_check_varsh_sum_872_4`.\n- There is a likely bug in `_check_varsh_872_9`: the variable `other3` is computed but omitted from the return value (line 596 returns `other1 + other2 + other4`, skipping `other3`).\n\n**Usage in the broader quantum module:**\n- In `sympy/physics/quantum/spin.py`, CG is imported and used in angular momentum coupling/uncoupling computations.\n\n**Printing:**\n- LaTeX: CG renders as `C^{j3,m3}_{j1,m1,j2,m2}` \u2014 note the argument reordering: superscript has (j3, m3), subscript has (j1, m1, j2, m2).\n- Pretty printing: CG shows 'C' with (j3, m3) above and (j1, m1, j2, m2) below.",
    "rubric": [
      "Explains that CG inherits from Wigner3j which inherits from Expr, and reuses Wigner3j.__new__ which sympifies all 6 arguments",
      "Mentions that CG.doit() delegates to the clebsch_gordan() function from sympy.physics.wigner",
      "Explains that clebsch_gordan() converts to wigner_3j using (-1)^(j1-j2+m3) * sqrt(2*j3+1) * wigner_3j(j1,j2,j3,m1,m2,-m3) (note the negated m3)",
      "Describes the wigner_3j algorithm: arguments are doubled to integers, early-return zero conditions are checked (m1+m2+m3\u22600, triangle inequality, |mi|>ji, parity), then an alternating sum using sumres = ti - sumres pattern with math.comb",
      "Mentions the phase calculation: (-1)^(dj1 + (dj3+dm3)//2 + imax) multiplied with sqrt(res_num/res_den) * sumres",
      "Notes the is_symbolic property check that raises ValueError if any argument is not numerical",
      "Mentions the precedence attribute set to PRECEDENCE['Pow'] - 1 or the is_commutative = True inheritance"
    ],
    "key_files": [
      "sympy/physics/quantum/cg.py",
      "sympy/physics/wigner.py",
      "sympy/physics/quantum/spin.py",
      "sympy/printing/precedence.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/cg.py] sympy.physics.quantum.cg.CG\nsympy.physics.quantum.cg.CG:\n  Class for Clebsch-Gordan coefficient.\n\n    Explanation\n    ===========\n\n    Clebsch-Gordan coefficients describe the angular momentum coupling between\n    two systems. The coefficients give the expansion of a coupled total angular\n    momentum state and an uncoupled tensor product state. The Clebsch-Gordan\n    coefficients are defined as [1]_:\n\n    .. math ::\n        C^{j_3,m_3}_{j_1,m_1,j_2,m_2} = \\left\\langle j_1,m_1;j_2,m_2 | j_3,m_3\\right\\rangle\n\n    Parameters\n    ==========\n\n    j1, m1, j2, m2 : Number, Symbol\n        Angular momenta of states 1 and 2.\n\n    j3, m3: Number, Symbol\n        Total angular momentum of the coupled system.\n\n    Examples\n    ========\n\n    Define a Clebsch-Gordan coefficient and evaluate its value\n\n        >>> from sympy.physics.quantum.cg import CG\n        >>> from sympy import S\n        >>> cg = CG(S(3)/2, S(3)/2, S(1)/2, -S(1)/2, 1, 1)\n        >>> cg\n        CG(3/2, 3/2, 1/2, -1/2, 1, 1)\n        >>> cg.doit()\n        sqrt(3)/2\n        >>> CG(j1=S(1)/2, m1=-S(1)/2, j2=S(1)/2, m2=+S(1)/2, j3=1, m3=0).doit()\n        sqrt(2)/2\n\n\n    Compare [2]_.\n\n    See Also\n    ========\n\n    Wigner3j: Wigner-3j symbols\n\n    References\n    ==========\n\n    .. [1] Varshalovich, D A, Quantum Theory of Angular Momentum. 1988.\n    .. [2] `Clebsch-Gordan Coefficients, Spherical Harmonics, and d Functions\n        <https://pdg.lbl.gov/2020/reviews/rpp2020-rev-clebsch-gordan-coefs.pdf>`_\n        in P.A. Zyla *et al.* (Particle Data Group), Prog. Theor. Exp. Phys.\n        2020, 083C01 (2020).",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many implementation details (Wei99 algorithm, _doubled_int, math.comb, alternating sum pattern, cg_simp internals, _check_varsh_872_9 bug, precedence value of 59, etc.) are not verifiable from the provided documentation alone, but they are plausible code-level details that don't contradict the documentation.",
      "The LaTeX rendering claim (C^{j3,m3}_{j1,m1,j2,m2}) is consistent with the mathematical notation shown in the documentation (C^{j_3,m_3}_{j_1,m_1,j_2,m_2}).",
      "The CG.doit() examples are consistent: CG(3/2, 3/2, 1/2, -1/2, 1, 1).doit() = sqrt(3)/2 matches the documentation.",
      "The parameter description (j1, m1, j2, m2, j3, m3) matches the documentation."
    ],
    "strip_verify_leakage": 0.09,
    "strip_verify_summary": "Minor doc leakage: algorithm name 'Wei99' and physics identity descriptions came from documentation; nearly all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_69",
    "repo": "sympy",
    "question": "What algorithm does `find_subcircuit` use, and what are its key dependencies including helper functions, input handling, and consumers?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `find_subcircuit` function in `sympy/physics/quantum/circuitutils.py` uses the KMP (Knuth-Morris-Pratt) algorithm pattern (as indicated by the `kmp_table` helper function name) to find a subcircuit within a circuit. It depends on several key components:\n\n1. **`kmp_table` function** (same file): Builds the table used by the KMP-style search algorithm. It initializes the table with `[-1, 0]` and then iterates through the word/subcircuit to fill in match values.\n\n2. **Input handling**: Both `circuit` and `subcircuit` parameters accept `Mul` objects (from `sympy.core.mul`). When a `Mul` is received, the function extracts its `.args` tuple to get the individual gate sequence.\n\n3. **Early termination conditions**: Returns -1 immediately if `subcircuit` is empty or longer than `circuit`. If `end < 1`, it defaults `end` to `len(circuit)`.\n\n4. **The Gate class hierarchy**: The gates being searched (X, Y, Z, H, etc.) inherit from `Gate \u2192 UnitaryOperator \u2192 Operator`. `Gate` is defined in `sympy/physics/quantum/gate.py`. The convenience names `X`, `Y`, `Z`, `H` correspond to `XGate`, `YGate`, `ZGate`, `HadamardGate` as seen from usage in tests and the `__all__` list. These all inherit from `OneQubitGate(Gate)`, which sets `nqubits = _S.One`.\n\n5. **Consumers of find_subcircuit**: \n   - `replace_subcircuit` calls `find_subcircuit` to locate the position before splicing out and replacing the subcircuit.\n   - `random_reduce` uses `find_subcircuit` to check if any identity (from the flattened gate_ids list) exists in the circuit before removing it.\n\n6. **`flatten_ids` function**: Used by `random_reduce` to flatten `GateIdentity` objects into a sorted list of equivalent identity tuples. It uses `functools.reduce` with a lambda that accesses each `GateIdentity`'s `equivalent_ids` property and sorts them using `default_sort_key`.\n\n7. **`GateIdentity` class** (in `sympy/physics/quantum/identitysearch.py`): Has an `equivalent_ids` property that returns `self._eq_ids`, which is computed via `generate_equivalent_ids(args)` in the `__new__` method. `generate_equivalent_ids` produces a set of gate sequence tuples derived from gate rules.\n\n8. **Module exports**: The `__all__` list in `circuitutils.py` exports 7 functions: `kmp_table`, `find_subcircuit`, `replace_subcircuit`, `convert_to_symbolic_indices`, `convert_to_real_indices`, `random_reduce`, and `random_insert`.\n\n9. **The `convert_to_symbolic_indices` function** uses `numbered_symbols` with `prefix='i'` and `start=-1` to generate symbolic indices (i0, i1, ...) for gates.",
    "rubric": [
      "Identifies that find_subcircuit uses the KMP (Knuth-Morris-Pratt) algorithm, evidenced by the kmp_table helper function",
      "Explains that kmp_table builds a prefix table initialized with [-1, 0] and iterates through the subcircuit to fill match values",
      "Notes that both circuit and subcircuit parameters accept Mul objects, and .args is extracted to get the gate sequence",
      "Mentions early termination: returns -1 if subcircuit is empty or longer than circuit, and defaults end to len(circuit) if end < 1",
      "Identifies that gates like X, Y, Z, H are aliases for XGate, YGate, ZGate, HadamardGate which inherit from OneQubitGate(Gate)",
      "Names replace_subcircuit as a consumer that uses find_subcircuit to locate position before splicing/replacing",
      "Names random_reduce as a consumer that uses find_subcircuit to check if identity sequences exist in the circuit",
      "Explains that flatten_ids flattens GateIdentity objects into sorted tuples using equivalent_ids property",
      "Notes that GateIdentity's equivalent_ids comes from generate_equivalent_ids called in __new__"
    ],
    "key_files": [
      "sympy/physics/quantum/circuitutils.py",
      "sympy/physics/quantum/gate.py",
      "sympy/physics/quantum/identitysearch.py",
      "sympy/physics/quantum/tests/test_circuitutils.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/circuitutils.py] sympy.physics.quantum.circuitutils.find_subcircuit\nFinds the subcircuit in circuit, if it exists.\n\n    Explanation\n    ===========\n\n    If the subcircuit exists, the index of the start of\n    the subcircuit in circuit is returned; otherwise,\n    -1 is returned.  The algorithm that is implemented\n    is the Knuth-Morris-Pratt algorithm.\n\n    Parameters\n    ==========\n\n    circuit : tuple, Gate or Mul\n        A tuple of Gates or Mul representing a quantum circuit\n    subcircuit : tuple, Gate or Mul\n        A tuple of Gates or Mul to find in circuit\n    start : int\n        The location to start looking for subcircuit.\n        If start is the same or past end, -1 is returned.\n    end : int\n        The last place to look for a subcircuit.  If end\n        is less than 1 (one), then the length of circuit\n        is taken to be end.\n\n    Examples\n    ========\n\n    Find the first instance of a subcircuit:\n\n    >>> from sympy.physics.quantum.circuitutils import find_subcircuit\n    >>> from sympy.physics.quantum.gate import X, Y, Z, H\n    >>> circuit = X(0)*Z(0)*Y(0)*H(0)\n    >>> subcircuit = Z(0)*Y(0)\n    >>> find_subcircuit(circuit, subcircuit)\n    1\n\n    Find the first instance starting at a specific position:\n\n    >>> find_subcircuit(circuit, subcircuit, start=1)\n    1\n\n    >>> find_subcircuit(circuit, subcircuit, start=2)\n    -1\n\n    >>> circuit = circuit*subcircuit\n    >>> find_subcircuit(circuit, subcircuit, start=2)\n    4\n\n    Find the subcircuit within some interval:\n\n    >>> find_subcircuit(circuit, subcircuit, start=2, end=2)\n    -1",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor documentation-derived purpose descriptions and unverifiable line numbers were adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_70",
    "repo": "sympy",
    "question": "Why does `replace_subcircuit` use KMP instead of a naive search approach, and how does `random_reduce` benefit from this?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `replace_subcircuit` function uses the Knuth-Morris-Pratt (KMP) algorithm for subcircuit searching instead of a naive O(n*m) approach, which provides O(n+m) time complexity for pattern matching.\n\n1. **KMP avoids redundant comparisons**: The `find_subcircuit` function (called by `replace_subcircuit`) implements KMP, which builds a 'partial match' table via `kmp_table()` before searching. This table allows the search to skip positions that cannot possibly match, avoiding backtracking \u2014 visible in the code where `pos = pos + index - table[index]` advances the position.\n\n2. **The primary consumer is `random_reduce`**, which calls `find_subcircuit` in a loop to check if each identity from a potentially large list of flattened gate identities exists in the circuit, then calls `replace_subcircuit` to remove the found identity. The `flatten_ids` function expands `GateIdentity` objects into all their `equivalent_ids` (permutations/rotations of the identity circuit). This means `find_subcircuit` may be called many times per `random_reduce` invocation.\n\n3. **Tuple-based operations**: All functions (`replace_subcircuit`, `find_subcircuit`, `random_reduce`, `random_insert`) immediately convert `Mul` inputs to tuples via `.args`. The replacement itself is just tuple concatenation: `circuit = left + replace + right`.\n\n4. **Default `replace=None` maps to empty tuple**: When `replace` is `None` (the default), it's set to `()`, so the function acts as a removal operation. This is the exact behavior needed by `random_reduce`, which calls `replace_subcircuit(circuit, id)` without a replacement argument to remove identity subcircuits.\n\n5. **Gate equality relies on SymPy structural equality**: The KMP comparisons (`subcircuit[index] == circuit[pos + index]`) use SymPy's `Basic.__eq__`, inherited through `Gate -> UnitaryOperator -> Operator -> QExpr -> Expr -> Basic`. Two gates are equal when they have the same class and args. The `kmp_table` function also uses `==` for comparing word elements (`if word[pos - 1] == word[cnd]`), which works correctly because Gate objects support structural equality.",
    "rubric": [
      "Explains that KMP provides O(n+m) time complexity instead of naive O(n*m) by building a partial match table via kmp_table() that avoids redundant comparisons/backtracking",
      "Describes that the position advances via `pos = pos + index - table[index]` to skip impossible positions",
      "Explains that random_reduce calls find_subcircuit in a loop for each identity from a potentially large flattened list of gate identities (via flatten_ids expanding equivalent_ids), so efficient searching matters",
      "Mentions that replace_subcircuit with default replace=None maps to empty tuple, effectively removing the subcircuit \u2014 which is how random_reduce uses it",
      "Notes that Gate equality comparison relies on SymPy's structural equality (same class and args) for the KMP comparisons"
    ],
    "key_files": [
      "sympy/physics/quantum/circuitutils.py",
      "sympy/physics/quantum/identitysearch.py",
      "sympy/physics/quantum/gate.py",
      "sympy/physics/quantum/tests/test_circuitutils.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/circuitutils.py] sympy.physics.quantum.circuitutils.replace_subcircuit\nReplaces a subcircuit with another subcircuit in circuit,\n    if it exists.\n\n    Explanation\n    ===========\n\n    If multiple instances of subcircuit exists, the first instance is\n    replaced.  The position to being searching from (if different from\n    0) may be optionally given.  If subcircuit cannot be found, circuit\n    is returned.\n\n    Parameters\n    ==========\n\n    circuit : tuple, Gate or Mul\n        A quantum circuit.\n    subcircuit : tuple, Gate or Mul\n        The circuit to be replaced.\n    replace : tuple, Gate or Mul\n        The replacement circuit.\n    pos : int\n        The location to start search and replace\n        subcircuit, if it exists.  This may be used\n        if it is known beforehand that multiple\n        instances exist, and it is desirable to\n        replace a specific instance.  If a negative number\n        is given, pos will be defaulted to 0.\n\n    Examples\n    ========\n\n    Find and remove the subcircuit:\n\n    >>> from sympy.physics.quantum.circuitutils import replace_subcircuit\n    >>> from sympy.physics.quantum.gate import X, Y, Z, H\n    >>> circuit = X(0)*Z(0)*Y(0)*H(0)*X(0)*H(0)*Y(0)\n    >>> subcircuit = Z(0)*Y(0)\n    >>> replace_subcircuit(circuit, subcircuit)\n    (X(0), H(0), X(0), H(0), Y(0))\n\n    Remove the subcircuit given a starting search point:\n\n    >>> replace_subcircuit(circuit, subcircuit, pos=1)\n    (X(0), H(0), X(0), H(0), Y(0))\n\n    >>> replace_subcircuit(circuit, subcircuit, pos=2)\n    (X(0), Z(0), Y(0), H(0), X(0), H(0), Y(0))\n\n    Replace the subcircuit:\n\n    >>> replacement = H(0)*Z(0)\n    >>> replace_subcircuit(circuit, subcircuit, replace=replacement)\n    (X(0), H(0), Z(0), H(0), X(0), H(0), Y(0))",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provides only the docstring for `replace_subcircuit`, covering its parameters, behavior, and examples. The generated answer makes extensive claims about internal implementation details (KMP algorithm, line numbers, `kmp_table()`, `flatten_ids`, `random_reduce`, `random_insert`, tuple-based operations, SymPy's `Basic.__eq__` inheritance chain, etc.) that are not present in or verifiable from the provided documentation.",
      "While these implementation claims are plausible and likely accurate based on knowledge of the SymPy codebase, they cannot be verified against the provided doc chunk alone \u2014 they are extrapolations beyond the documentation scope.",
      "The core documented behavior (replacing first instance of subcircuit, pos parameter behavior, default removal when no replacement given, returning original circuit if subcircuit not found) is consistent with the answer's claims, though the answer focuses primarily on implementation details rather than the documented API behavior.",
      "Claim that 'Default replace=None maps to empty tuple' is consistent with the documented examples showing removal behavior when no replace argument is given."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "A few claims about design rationale/motivation (why KMP was chosen, why tuples are used instead of Mul) are not recoverable from stripped code, but most algorithmic and structural claims are directly verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_71",
    "repo": "sympy",
    "question": "Where is the quantum Commutator class defined and how does its `doit()` method dispatch evaluation to specific operator subclasses?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Commutator` class in `sympy/physics/quantum/commutator.py` inherits directly from `sympy.core.expr.Expr` (not from `QExpr` like `Operator` does). Its parent class `Expr` provides the `compare` method, which is used in the `eval` classmethod (`if a.compare(b) == 1`) to swap argument order \u2014 when `compare` returns 1, the result is `S.NegativeOne*cls(b, a)`.\n\nThe `doit()` method imports `Operator` from `sympy.physics.quantum.operator` locally, then uses a dispatch mechanism: it first calls `A._eval_commutator(B)`, and if that raises `NotImplementedError`, tries `-1*B._eval_commutator(A)`. The `Operator._eval_commutator` method calls `dispatch_method(self, '_eval_commutator', other, **options)` from `sympy/physics/quantum/qexpr.py`. This `dispatch_method` function constructs a method name by appending the class name of `other` (e.g., `_eval_commutator_BosonOp`) and looks for it via `hasattr`/`getattr` on `self`.\n\nThe quantum `Commutator` class from `commutator.py` inherits from `Expr`. There are also other `Commutator` classes elsewhere in the codebase (in `sympy/physics/secondquant.py` and `sympy/diffgeom/diffgeom.py`).\n\nThe quantum `Commutator` is used in `qapply.py` where calling `.doit()` is triggered when it appears as a left-hand side operator in `qapply_Mul` (`if isinstance(lhs, (Commutator, AntiCommutator)): comm = lhs.doit()`), and in `represent.py` where it is expanded as `represent(Mul(A, B) - Mul(B, A), **options)`.",
    "rubric": [
      "States that the quantum Commutator class is defined in sympy/physics/quantum/commutator.py and inherits from Expr (not QExpr)",
      "Explains that doit() first tries A._eval_commutator(B), and on NotImplementedError, falls back to -1*B._eval_commutator(A)",
      "Describes the dispatch_method function in qexpr.py that constructs method names like _eval_commutator_BosonOp by appending the other class's name, then looks it up via hasattr/getattr",
      "Mentions that there are other Commutator classes in sympy/physics/secondquant.py and sympy/diffgeom/diffgeom.py",
      "Notes how Commutator is consumed in qapply.py (lhs.doit() when it appears as left-hand side) and/or represent.py (expanded as represent(A*B - B*A))",
      "Mentions that eval() uses a.compare(b) == 1 to canonically order arguments, returning -1 * cls(b, a) when needed"
    ],
    "key_files": [
      "sympy/physics/quantum/commutator.py",
      "sympy/physics/quantum/operator.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/qapply.py",
      "sympy/physics/quantum/represent.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/commutator.py] sympy.physics.quantum.commutator.Commutator\nsympy.physics.quantum.commutator.Commutator:\n  The standard commutator, in an unevaluated state.\n\n    Explanation\n    ===========\n\n    Evaluating a commutator is defined [1]_ as: ``[A, B] = A*B - B*A``. This\n    class returns the commutator in an unevaluated form. To evaluate the\n    commutator, use the ``.doit()`` method.\n\n    Canonical ordering of a commutator is ``[A, B]`` for ``A < B``. The\n    arguments of the commutator are put into canonical order using comparison operators.\n    If ``B < A``, then ``[B, A]`` is returned as ``-[A, B]``.\n\n    Parameters\n    ==========\n\n    A : Expr\n        The first argument of the commutator [A,B].\n    B : Expr\n        The second argument of the commutator [A,B].\n\n    Examples\n    ========\n\n    >>> from sympy.physics.quantum import Commutator, Dagger, Operator\n    >>> from sympy.abc import x, y\n    >>> A = Operator('A')\n    >>> B = Operator('B')\n    >>> C = Operator('C')\n\n    Create a commutator and use ``.doit()`` to evaluate it:\n\n    >>> comm = Commutator(A, B)\n    >>> comm\n    [A,B]\n    >>> comm.doit()\n    A*B - B*A\n\n    The commutator orders it arguments in canonical order:\n\n    >>> comm = Commutator(B, A); comm\n    -[A,B]\n\n    Commutative constants are factored out:\n\n    >>> Commutator(3*x*A, x*y*B)\n    3*x**2*y*[A,B]\n\n    Using ``.expand(commutator=True)``, the standard commutator expansion rules\n    can be applied:\n\n    >>> Commutator(A+B, C).expand(commutator=True)\n    [A,C] + [B,C]\n    >>> Commutator(A, B+C).expand(commutator=True)\n    [A,B] + [A,C]\n    >>> Commutator(A*B, C).expand(commutator=True)\n    [A,C]*B + A*[B,C]\n    >>> Commutator(A, B*C).expand(commutator=True)\n    [A,B]*C + B*[A,C]\n\n    Adjoint operations applied to the commutator are properly applied to the\n    arguments:\n\n    >>> Dagger(Commutator(A, B))\n    -[Dagger(A),Dagger(B)]\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Commutator\n\nsympy.physics.quantum.commutator.Commutator.doit:\n  Evaluate commutator",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with exact line numbers, design rationale for local imports, and unshown files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sympy_gen_72",
    "repo": "sympy",
    "question": "How does the `Dagger` class dispatch and evaluate the Hermitian conjugate for different types of arguments?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Dagger` class in `sympy/physics/quantum/dagger.py` implements the Hermitian conjugate operation by subclassing `adjoint` (from `sympy.functions.elementary.complexes`), which itself inherits from `DefinedFunction` \u2192 `Function` \u2192 `Application` \u2192 `Expr`. Here is how `Dagger` works at the implementation level:\n\n**1. `__new__` method (construction/evaluation):**\n`Dagger.__new__` completely overrides the parent `DefinedFunction.__new__`. It follows a three-step dispatch:\n- If the argument has an `adjoint` method and `evaluate=True`, it calls `arg.adjoint()` directly.\n- Otherwise, if the argument has both `conjugate` and `transpose` methods and `evaluate=True`, it calls `arg.conjugate().transpose()`.\n- If neither condition applies (or `evaluate=False`), it falls through to `Expr.__new__(cls, sympify(arg))`, creating an unevaluated `Dagger` wrapper.\n\n**2. Dispatch through `_eval_adjoint`:**\nWhen `arg.adjoint()` is called (the first path), it ultimately results in the argument's `_eval_adjoint()` method being invoked. Many quantum objects implement this method:\n- `State._eval_adjoint()` returns `self.dual`, which constructs the dual class (e.g., `Ket` \u2192 `Bra`).\n- `QExpr._eval_adjoint()` falls back to `Expr._eval_adjoint()` and if that returns None, wraps the result in `Expr.__new__(Dagger, self)`, also preserving the `hilbert_space` attribute.\n- `HermitianOperator` has `is_hermitian = True`, so `Expr._eval_adjoint()` returns `self` (the operator itself).\n- `UnitaryOperator._eval_adjoint()` returns `self._eval_inverse()`.\n- `OuterProduct._eval_adjoint()` returns `OuterProduct(Dagger(self.bra), Dagger(self.ket))`.\n\n**3. Core expression adjoint handling:**\nFor non-quantum expressions, `Expr._eval_adjoint()` checks `is_hermitian` (returns self) and `is_antihermitian` (returns -self). The `is_hermitian` property in `Expr` returns True when `self.is_real` is True, and `is_antihermitian` returns True when `self.is_imaginary` or `self.is_zero` is True.\n\n**4. Monkey-patching at module level:**\nAfter the class definition, two lines modify the parent `adjoint` class globally:\n- `adjoint.__name__ = \"Dagger\"` \u2014 changes the display name of the parent class.\n- `adjoint._sympyrepr = lambda a, b: \"Dagger(%s)\" % b._print(a.args[0])` \u2014 overrides repr printing so that unevaluated `adjoint` instances also print as `Dagger(...)`. This means importing `dagger.py` has the side effect of modifying `adjoint`'s string representation.\n\n**5. `kind` property:**\nThe `kind` property simply delegates to `self.args[0].kind`, returning the kind of the inner expression.",
    "rubric": [
      "Explains that Dagger subclasses `adjoint` from `sympy.functions.elementary.complexes`",
      "Describes the three-step dispatch in `__new__`: first checks for `adjoint` method, then checks for `conjugate`+`transpose` methods, then falls through to `Expr.__new__` for unevaluated wrapping",
      "Mentions the `evaluate` parameter controls whether eager evaluation happens or an unevaluated wrapper is returned",
      "Explains that `arg.adjoint()` triggers `_eval_adjoint()` on the argument, with examples like `State._eval_adjoint()` returning `self.dual`, `UnitaryOperator._eval_adjoint()` returning `self._eval_inverse()`, or `QExpr._eval_adjoint()` falling back to `Expr._eval_adjoint()` and wrapping in Dagger if None",
      "Describes that `Expr._eval_adjoint()` handles Hermitian (returns self) and anti-Hermitian (returns -self) cases, and also tries conjugate+transpose fallback",
      "Mentions the module-level monkey-patching: `adjoint.__name__` is set to 'Dagger' and `adjoint._sympyrepr` is overridden so adjoint instances print as `Dagger(...)`"
    ],
    "key_files": [
      "sympy/physics/quantum/dagger.py",
      "sympy/functions/elementary/complexes.py",
      "sympy/core/expr.py",
      "sympy/physics/quantum/qexpr.py",
      "sympy/physics/quantum/state.py",
      "sympy/physics/quantum/operator.py",
      "sympy/core/mul.py"
    ],
    "source_doc": "[docstring: sympy/physics/quantum/dagger.py] sympy.physics.quantum.dagger.Dagger\nsympy.physics.quantum.dagger.Dagger:\n  General Hermitian conjugate operation.\n\n    Explanation\n    ===========\n\n    Take the Hermetian conjugate of an argument [1]_. For matrices this\n    operation is equivalent to transpose and complex conjugate [2]_.\n\n    Parameters\n    ==========\n\n    arg : Expr\n        The SymPy expression that we want to take the dagger of.\n    evaluate : bool\n        Whether the resulting expression should be directly evaluated.\n\n    Examples\n    ========\n\n    Daggering various quantum objects:\n\n        >>> from sympy.physics.quantum.dagger import Dagger\n        >>> from sympy.physics.quantum.state import Ket, Bra\n        >>> from sympy.physics.quantum.operator import Operator\n        >>> Dagger(Ket('psi'))\n        <psi|\n        >>> Dagger(Bra('phi'))\n        |phi>\n        >>> Dagger(Operator('A'))\n        Dagger(A)\n\n    Inner and outer products::\n\n        >>> from sympy.physics.quantum import InnerProduct, OuterProduct\n        >>> Dagger(InnerProduct(Bra('a'), Ket('b')))\n        <b|a>\n        >>> Dagger(OuterProduct(Ket('a'), Bra('b')))\n        |b><a|\n\n    Powers, sums and products::\n\n        >>> A = Operator('A')\n        >>> B = Operator('B')\n        >>> Dagger(A*B)\n        Dagger(B)*Dagger(A)\n        >>> Dagger(A+B)\n        Dagger(A) + Dagger(B)\n        >>> Dagger(A**2)\n        Dagger(A)**2\n\n    Dagger also seamlessly handles complex numbers and matrices::\n\n        >>> from sympy import Matrix, I\n        >>> m = Matrix([[1,I],[2,I]])\n        >>> m\n        Matrix([\n        [1, I],\n        [2, I]])\n        >>> Dagger(m)\n        Matrix([\n        [ 1,  2],\n        [-I, -I]])\n\n    References\n    ==========\n\n    .. [1] https://en.wikipedia.org/wiki/Hermitian_adjoint\n    .. [2] https://en.wikipedia.org/wiki/Hermitian_transpose\n\nsympy.physics.quantum.dagger.Dagger.kind:\n  Find the kind of a dagger of something (just the kind of the something).",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims Dagger subclasses `adjoint` from `sympy.functions.elementary.complexes`, which is plausible but not explicitly stated in the documentation provided.",
      "The answer describes detailed internal dispatch mechanisms (`__new__`, `_eval_adjoint`, monkey-patching) that are not mentioned in the documentation - these are plausible implementation details but cannot be verified from the given doc.",
      "The answer claims `adjoint` inherits from `DefinedFunction \u2192 Function \u2192 Application \u2192 Expr` - the documentation does not mention this inheritance chain, and `DefinedFunction` is not a standard SymPy class name (it's typically `Function \u2192 Application \u2192 Basic`), which may be a hallucination.",
      "The answer states that `Commutator._eval_adjoint()` reverses operand order - plausible but unverifiable from the given documentation.",
      "The monkey-patching claims (`adjoint.__name__ = 'Dagger'` and `adjoint._sympyrepr` override) are specific implementation details not mentioned in the documentation and could be hallucinated.",
      "The core behavioral claims (Dagger of Ket gives Bra, Dagger of product reverses order, Dagger of matrix gives conjugate transpose) are all consistent with the documentation's examples."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims about files not provided (Commutator, AntiCommutator, TensorProduct, BosonOp, InnerProduct, Mul._eval_adjoint) and one about DefinedFunction.__new__ internals were removed; most core claims about dagger.py, qexpr.py, state.py, and operator.py are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_04",
    "repo": "pytest",
    "question": "How does pytest handle finalizer failures during fixture teardown, and how are multiple exceptions from different teardown stages combined?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "Here's how the key features are implemented in the current codebase:\n\n**Finalizer handling:** The `FixtureDef.finish()` method in `src/_pytest/fixtures.py` runs ALL finalizers even if some fail, collecting exceptions in a list. Finalizers are popped in LIFO order (using `.pop()`). If multiple exceptions occur, they are wrapped in a `BaseExceptionGroup` with exceptions reversed (`exceptions[::-1]`). This pattern is clearly demonstrated in `SetupState.teardown_exact()` in runner.py. After finalization, `self.cached_result` is set to `None` and `self._finalizers.clear()` is called even if finalization fails.\n\n**Lazy finalization at setup time:** In `FixtureDef.execute()`, parametrized fixture dependencies are resolved first. If a cached result exists but with a different parameter (cache miss), `self.finish(request)` is called immediately during setup to tear down the old value before creating a new one. Finalizers are also registered on parent fixtures via `functools.partial(self.finish, request=request)`.\n\n**--color option:** Defined in `src/_pytest/terminal.py` with `choices=[\"yes\", \"no\", \"auto\"]` and `default=\"auto\"`, stored in dest `\"color\"`.\n\n**--runxfail:** In `src/_pytest/skipping.py`, `pytest_configure` replaces `pytest.xfail` with a no-op function `nop` that does nothing. `nop.Exception = xfail.Exception` is set on the replacement function. The original is restored via `config.add_cleanup(lambda: setattr(pytest, \"xfail\", old))`.\n\n**pytestconfig fixture is session-scoped:** Defined in `src/_pytest/fixtures.py` with `@fixture(scope=\"session\")` decorator, returning `request.config`.\n\n**Version comparison in importorskip:** In `src/_pytest/outcomes.py`, version comparison uses `packaging.version.Version` (imported inside the function), comparing `Version(verattr) < Version(minversion)`.\n\n**Mock patch ordering fix:** `num_mock_patch_args()` in `src/_pytest/compat.py` counts mock patchings by checking `function.patchings` attribute and filtering for patches without `attribute_name` where `p.new` is the mock sentinel from either `mock.DEFAULT` or `unittest.mock.DEFAULT`. The `getfuncargnames()` function then strips these mock arguments with `arg_names[num_mock_patch_args(function):]`.\n\n**-k expression handling:** The expression grammar in `src/_pytest/mark/expression.py` defines identifiers as `(:?\\w|:|\\+|-|\\.|\\[|\\]|\\\\|/)+` regex pattern, which allows dots, hyphens, etc. in identifiers. This means expressions like \"-k 1.3\" or \"-k None\" are valid identifiers that don't raise syntax errors.\n\n**SetupState teardown:** `SetupState.teardown_exact()` in `src/_pytest/runner.py` pops nodes from the stack (which uses dict insertion order via `popitem()`) and runs all finalizers for each node even if some fail. Multiple exceptions per node are grouped into a `BaseExceptionGroup`, and multiple node-level exceptions are grouped into an outer `BaseExceptionGroup(\"errors during test teardown\", ...)`.",
    "rubric": [
      "Explains that FixtureDef.finish() runs ALL finalizers even if some fail, collecting exceptions in a list",
      "Mentions finalizers are popped in LIFO order (using .pop())",
      "Notes that multiple exceptions are wrapped in a BaseExceptionGroup with exceptions reversed (exceptions[::-1])",
      "Describes that SetupState.teardown_exact() pops nodes from the stack using popitem() and runs finalizers for each",
      "Mentions that multiple node-level exceptions are grouped into an outer BaseExceptionGroup('errors during test teardown', ...)",
      "Notes that cached_result is set to None and _finalizers.clear() is called even if finalization fails",
      "Mentions that parametrized fixture cache misses trigger self.finish(request) during setup to tear down old values before creating new ones"
    ],
    "key_files": [
      "src/_pytest/fixtures.py",
      "src/_pytest/runner.py",
      "src/_pytest/skipping.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/compat.py",
      "src/_pytest/mark/expression.py",
      "src/_pytest/terminal.py",
      "src/_pytest/scope.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.5.0.rst] 2.5.0\n- dropped python2.5 from automated release testing of pytest itself\n  which means it's probably going to break soon (but still works\n  with this release we believe).\n\n- simplified and fixed implementation for calling finalizers when\n  parametrized fixtures or function arguments are involved.  finalization\n  is now performed lazily at setup time instead of in the \"teardown phase\".\n  While this might sound odd at first, it helps to ensure that we are\n  correctly handling setup/teardown even in complex code.  User-level code\n  should not be affected unless it's implementing the pytest_runtest_teardown\n  hook and expecting certain fixture instances are torn down within (very\n  unlikely and would have been unreliable anyway).\n\n- PR90: add --color=yes|no|auto option to force terminal coloring\n  mode (\"auto\" is default).  Thanks Marc Abramowitz.\n\n- fix issue319 - correctly show unicode in assertion errors.  Many\n  thanks to Floris Bruynooghe for the complete PR.  Also means\n  we depend on py>=1.4.19 now.\n\n- fix issue396 - correctly sort and finalize class-scoped parametrized\n  tests independently from number of methods on the class.\n\n- refix issue323 in a better way -- parametrization should now never\n  cause Runtime Recursion errors because the underlying algorithm\n  for re-ordering tests per-scope/per-fixture is not recursive\n  anymore (it was tail-call recursive before which could lead\n  to problems for more than >966 non-function scoped parameters).\n\n- fix issue290 - there is preliminary support now for parametrizing\n  with repeated same values (sometimes useful to test if calling\n  a second time works as with the first time).\n\n- close issue240 - document precisely how pytest module importing\n  works, discuss the two common test directory layouts, and how it\n  interacts with PEP420-namespace packages.\n\n- fix issue246 fix finalizer order to be LIFO on independent fixtures\n  depending on a parametrized higher-than-function scoped fixture.\n  (was quite some effort so please bear with the complexity of this sentence :)\n  Thanks Ralph Schmitt for the precise failure example.\n\n- fix issue244 by implementing special index for parameters to only use\n  indices for parametrized test ids\n\n- fix issue287 by running all finalizers but saving the exception\n  from the first failing finalizer and re-raising it so teardown will\n  still have failed.  We reraise the first failing exception because\n  it might be the cause for other finalizers to fail.\n\n- fix ordering when mock.patch or other standard decorator-wrappings\n  are used with test methods.  This fixes issue346 and should\n  help with random \"xdist\" collection failures.  Thanks to\n  Ronny Pfannschmidt and Donald Stufft for helping to isolate it.\n\n- fix issue357 - special case \"-k\" expressions to allow for\n  filtering with simple strings that are not valid python expressions.\n  Examples: \"-k 1.3\" matches all tests parametrized with 1.3.\n  \"-k None\" filters all tests that have \"None\" in their name\n  and conversely \"-k 'not None'\".\n  Previously these examples would raise syntax errors.\n\n- fix issue384 by removing the trial support code\n  since the unittest compat enhancements allow\n  trial to handle it on its own\n\n- don't hide an ImportError when importing a plugin produces one.\n  fixes issue375.\n\n- fix issue275 - allow usefixtures and autouse fixtures\n  for running doctest text files.\n\n- fix issue380 by making --resultlog only rely on longrepr instead\n  of the \"reprcrash\" attribute which only exists sometimes.\n\n- address issue122: allow @pytest.fixture(params=iterator) by exploding\n  into a list early on.\n\n- fix pexpect-3.0 compatibility for pytest's own tests.\n  (fixes issue386)\n\n- allow nested parametrize-value markers, thanks James Lan for the PR.\n\n- fix unicode handling with new monkeypatch.setattr(import_path, value)\n  API.  Thanks Rob Dennis.  Fixes issue371.\n\n- fix unicode handling with junitxml, fixes issue368.\n\n- In assertion rewriting mode on Python 2, fix the detection of coding\n  cookies. See issue #330.\n\n- make \"--runxfail\" turn imperative pytest.xfail calls into no ops\n  (it already did neutralize pytest.mark.xfail markers)\n\n- refine pytest / pkg_resources interactions: The AssertionRewritingHook\n  PEP302 compliant loader now registers itself with setuptools/pkg_resources\n  properly so that the pkg_resources.resource_stream method works properly.\n  Fixes issue366.  Thanks for the investigations and full PR to Jason R. Coombs.\n\n- pytestconfig fixture is now session-scoped as it is the same object during the\n  whole test run.  Fixes issue370.\n\n- avoid one surprising case of marker malfunction/confusion::\n\n      @pytest.mark.some(lambda arg: ...)\n      def test_function():\n\n  would not work correctly because pytest assumes @pytest.mark.some\n  gets a function to be decorated already.  We now at least detect if this\n  arg is a lambda and thus the example will work.  Thanks Alex Gaynor\n  for bringing it up.\n\n- xfail a test on pypy that checks wrong encoding/ascii (pypy does\n  not error out). fixes issue385.\n\n- internally make varnames() deal with classes's __init__,\n  although it's not needed by pytest itself atm.  Also\n  fix caching.  Fixes issue376.\n\n- fix issue221 - handle importing of namespace-package with no\n  __init__.py properly.\n\n- refactor internal FixtureRequest handling to avoid monkeypatching.\n  One of the positive user-facing effects is that the \"request\" object\n  can now be used in closures.\n\n- fixed version comparison in pytest.importskip(modname, minverstring)\n\n- fix issue377 by clarifying in the nose-compat docs that pytest\n  does not duplicate the unittest-API into the \"plain\" namespace.\n\n- fix verbose reporting for @mock'd test functions",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code, but issue references, some line numbers, and design rationale comments (like 'to avoid keeping instances alive via bound methods' and 'imported lazily for start-up time') are documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_01",
    "repo": "pytest",
    "question": "What is the architecture of pytest's Config class, including its inner classes, verbosity system, plugin manager, and INI configuration handling?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "Here's what the code reveals about the `Config` class and related infrastructure in `src/_pytest/config/__init__.py`:\n\n**Architecture & Internal Structure:**\n- `Config` registers itself as a plugin named `\"pytestconfig\"` during `__init__`\n- It uses `contextlib.ExitStack` for cleanup management (`_cleanup_stack`), where `add_cleanup` calls `self._cleanup_stack.callback(func)` and `_ensure_unconfigure` closes the stack and replaces it with a fresh one\n- The `_inicache` dictionary caches resolved ini values, keyed by canonical name\n- The `Config` class directly implements hook methods (`pytest_cmdline_parse`, `pytest_load_initial_conftests` with `@hookimpl(trylast=True)`, `pytest_collection` with `@hookimpl(wrapper=True)`)\n\n**Key Inner Classes:**\n- `InvocationParams` is a frozen dataclass with `args` (tuple[str, ...]), `plugins`, and `dir` fields. Its custom `__init__` uses `object.__setattr__` to set the frozen fields.\n- `ArgsSource` is an enum with values `ARGS`, `INVOCATION_DIR`, `TESTPATHS`, and a backwards-compatibility alias `INCOVATION_DIR = INVOCATION_DIR`\n\n**Verbosity System:**\n- Three fine-grained verbosity types are defined as Final class variables: `VERBOSITY_ASSERTIONS = \"assertions\"`, `VERBOSITY_TEST_CASES = \"test_cases\"`, `VERBOSITY_SUBTESTS = \"subtests\"`\n- `_VERBOSITY_INI_DEFAULT` is the sentinel string `\"auto\"` used to indicate the global verbosity level should be used\n- `_verbosity_ini_name()` is a static method that computes the ini name as `f\"verbosity_{verbosity_type}\"`\n- `_add_verbosity_ini` registers the ini option with type=\"string\" and default=`Config._VERBOSITY_INI_DEFAULT`\n\n**Plugin Manager:**\n- `PytestPluginManager` extends pluggy's `PluginManager` with project name `\"pytest\"`\n- It initializes a `DummyRewriteHook` as `rewrite_hook` that gets replaced by a real one if assertion rewriting is enabled\n- It caches `_get_directory` with `lru_cache(256)`\n- `parse_hookimpl_opts` filters out any attribute not starting with `\"pytest_\"` and ignores `\"pytest_plugins\"` specifically\n\n**INI Configuration:**\n- `ConfigValue` dataclass in `findpaths.py` tracks `value`, `origin` (\"file\" or \"override\"), and `mode` (\"ini\" or \"toml\")\n- In `_getini()`, when multiple candidates exist (canonical + aliases), precedence is: CLI override > file, then canonical > alias, determined by `max(candidates, key=lambda x: (x[0].origin == \"override\", x[1]))`\n- The `Parser.addini` method supports an `aliases` parameter\n- `get_ini_default_for_type()` in argparsing.py returns `[]` for paths/pathlist/args/linelist, `False` for bool, `0` for int, `0.0` for float, and `\"\"` for string\n\n**Legacy Path Integration:**\n- The `legacypath` plugin monkeypatches `Config._getini_unknown_type` to support the `\"pathlist\"` type, returning `py.path.local` objects instead of `pathlib.Path`\n\n**getoption Implementation:**\n- `getoption` resolves option names through `self._parser._opt2dest` mapping (e.g. `\"--version\"` \u2192 `\"version\"`), allowing callers to use either the literal `--OPT` form or the dest name\n- When `skip=True` and value is `None`, it raises `AttributeError` internally then catches it to call `pytest.skip()`\n\n**Essential vs Default Plugins:**\n- Essential plugins tuple contains: `mark`, `main`, `runner`, `fixtures`, `helpconfig`\n- Default plugins include 23+ additional modules including `subtests`, `faulthandler`, `reports`\n- `builtin_plugins` adds `pytester`, `pytester_assertions`, and `terminalprogress` on top of default_plugins",
    "rubric": [
      "Config registers itself as a plugin named 'pytestconfig' during __init__",
      "Uses contextlib.ExitStack (_cleanup_stack) for cleanup management, with _ensure_unconfigure closing and replacing the stack",
      "InvocationParams is a frozen dataclass with args (tuple), plugins, and dir fields, using object.__setattr__ to set frozen fields",
      "ArgsSource enum has ARGS, INVOCATION_DIR, TESTPATHS values and a backwards-compatibility alias INCOVATION_DIR = INVOCATION_DIR",
      "Three fine-grained verbosity types defined as Final class variables: VERBOSITY_ASSERTIONS='assertions', VERBOSITY_TEST_CASES='test_cases', VERBOSITY_SUBTESTS='subtests'",
      "_VERBOSITY_INI_DEFAULT sentinel is 'auto' indicating global verbosity should be used",
      "PytestPluginManager extends pluggy's PluginManager with project name 'pytest' and initializes DummyRewriteHook as rewrite_hook",
      "parse_hookimpl_opts filters out attributes not starting with 'pytest_' and ignores 'pytest_plugins' specifically",
      "_getini() resolves multiple candidates (canonical + aliases) with precedence: CLI override > file, then canonical > alias",
      "getoption resolves option names through self._parser._opt2dest mapping, allowing --OPT form or dest name",
      "Essential plugins tuple contains mark, main, runner, fixtures, helpconfig",
      "Config implements hook methods directly including pytest_cmdline_parse and pytest_load_initial_conftests with @hookimpl(trylast=True)"
    ],
    "key_files": [
      "src/_pytest/config/__init__.py",
      "src/_pytest/config/argparsing.py",
      "src/_pytest/config/findpaths.py",
      "src/_pytest/stash.py",
      "src/_pytest/legacypath.py"
    ],
    "source_doc": "[docstring: src/_pytest/config/__init__.py] src._pytest.config.Config\nsrc._pytest.config.Config:\n  Access to configuration values, pluginmanager and plugin hooks.\n\n    :param PytestPluginManager pluginmanager:\n        A pytest PluginManager.\n\n    :param InvocationParams invocation_params:\n        Object containing parameters regarding the :func:`pytest.main`\n        invocation.\n\nsrc._pytest.config.Config.rootpath:\n  The path to the :ref:`rootdir <rootdir>`.\n\n        .. versionadded:: 6.1\n\nsrc._pytest.config.Config.inipath:\n  The path to the :ref:`configfile <configfiles>`.\n\n        .. versionadded:: 6.1\n\nsrc._pytest.config.Config.add_cleanup:\n  Add a function to be called when the config object gets out of\n        use (usually coinciding with pytest_unconfigure).\n\nsrc._pytest.config.Config.fromdictargs:\n  Constructor usable for subprocesses.\n\nsrc._pytest.config.Config._consider_importhook:\n  Install the PEP 302 import hook if using assertion rewriting.\n\n        Needs to parse the --assert=<mode> option from the commandline\n        and find all the installed plugins to mark them for rewriting\n        by the importhook.\n\nsrc._pytest.config.Config._mark_plugins_for_rewrite:\n  Given an importhook, mark for rewrite any top-level\n        modules or packages in the distribution package for\n        all pytest plugins.\n\nsrc._pytest.config.Config._validate_args:\n  Validate known args.\n\nsrc._pytest.config.Config._decide_args:\n  Decide the args (initial paths/nodeids) to use given the relevant inputs.\n\n        :param warn: Whether can issue warnings.\n\n        :returns: The args and the args source. Guaranteed to be non-empty.\n\nsrc._pytest.config.Config.issue_config_time_warning:\n  Issue and handle a warning during the \"configure\" stage.\n\n        During ``pytest_configure`` we can't capture warnings using the ``catch_warnings_for_item``\n        function because it is not possible to have hook wrappers around ``pytest_configure``.\n\n        This function is mainly intended for plugins that need to issue warnings during\n        ``pytest_configure`` (or similar stages).\n\n        :param warning: The warning instance.\n        :param stacklevel: stacklevel forwarded to warnings.warn.\n\nsrc._pytest.config.Config.addinivalue_line:\n  Add a line to a configuration option. The option must have been\n        declared but might not yet be set in which case the line becomes\n        the first line in its value.\n\nsrc._pytest.config.Config.getini:\n  Return configuration value the an :ref:`configuration file <configfiles>`.\n\n        If a configuration value is not defined in a\n        :ref:`configuration file <configfiles>`, then the ``default`` value\n        provided while registering the configuration through\n        :func:`parser.addini <pytest.Parser.addini>` will be returned.\n        Please note that you can even provide ``None`` as a valid\n        default value.\n\n        If ``default`` is not provided while registering using\n        :func:`parser.addini <pytest.Parser.addini>`, then a default value\n        based on the ``type`` parameter passed to\n        :func:`parser.addini <pytest.Parser.addini>` will be returned.\n        The default values based on ``type`` are:\n        ``paths``, ``pathlist``, ``args`` and ``linelist`` : empty list ``[]``\n        ``bool`` : ``False``\n        ``string`` : empty string ``\"\"``\n        ``int`` : ``0``\n        ``float`` : ``0.0``\n\n        If neither the ``default`` nor the ``type`` parameter is passed\n        while registering the configuration through\n        :func:`parser.addini <pytest.Parser.addini>`, then the configuration\n        is treated as a string and a default empty string '' is returned.\n\n        If the specified name hasn't been registered through a prior\n        :func:`parser.addini <pytest.Parser.addini>` call (usually from a\n        plugin), a ValueError is raised.\n\nsrc._pytest.config.Config._getini_ini:\n  Handle config values read in INI mode.\n\n        In INI mode, values are stored as str or list[str] only, and coerced\n        from string based on the registered type.\n\nsrc._pytest.config.Config._getini_toml:\n  Handle TOML config values with strict type validation and no coercion.\n\n        In TOML mode, values already have native types from TOML parsing.\n        We validate types match expectations exactly, including list items.\n\nsrc._pytest.config.Config.getoption:\n  Return command line option value.\n\n        :param name: Name of the option. You may also specify\n            the literal ``--OPT`` option instead of the \"dest\" option name.\n        :param default: Fallback value if no option of that name is **declared** via :hook:`pytest_addoption`.\n            Note this parameter will be ignored when the option is **declared** even if the option's value is ``None``.\n        :param skip: If ``True``, raise :func:`pytest.skip` if option is undeclared or has a ``None`` value.\n            Note that even if ``True``, if a default was specified it will be returned instead of a skip.\n\nsrc._pytest.config.Config.getvalue:\n  Deprecated, use getoption() instead.\n\nsrc._pytest.config.Config.getvalueorskip:\n  Deprecated, use getoption(skip=True) instead.\n\nsrc._pytest.config.Config.get_verbosity:\n  Retrieve the verbosity level for a fine-grained verbosity type.\n\n        :param verbosity_type: Verbosity type to get level for. If a level is\n            configured for the given type, that value will be returned. If the\n            given type is not a known verbosity type, the global verbosity\n            level will be returned. If the given type is None (default), the\n            global verbosity level will be returned.\n\n        To configure a level for a fine-grained verbosity type, the\n        configuration file should have a setting for the configuration name\n        and a numeric value for the verbosity level. A special value of \"auto\"\n        can be used to explicitly use the global verbosity level.\n\n        Example:\n\n        .. tab:: toml\n\n            .. code-block:: toml\n\n                [tool.pytest]\n                verbosity_assertions = 2\n\n        .. tab:: ini\n\n            .. code-block:: ini\n\n                [pytest]\n                verbosity_assertions = 2\n\n        .. code-block:: console\n\n            pytest -v\n\n        .. code-block:: python\n\n            print(config.get_verbosity())  # 1\n            print(config.get_verbosity(Config.VERBOSITY_ASSERTIONS))  # 2\n\nsrc._pytest.config.Config._add_verbosity_ini:\n  Add a output verbosity configuration option for the given output type.\n\n        :param parser: Parser for command line arguments and config-file values.\n        :param verbosity_type: Fine-grained verbosity category.\n        :param help: Description of the output this type controls.\n\n        The value should be retrieved via a call to\n        :py:func:`config.get_verbosity(type) <pytest.Config.get_verbosity>`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims `Config.getini` documentation says 'Return configuration value the an configuration file' which matches the doc's apparent typo 'Return configuration value the an :ref:`configuration file`'",
      "The answer states getoption 'raises AttributeError internally then catches it to call pytest.skip()' - this is a plausible implementation detail but not verifiable from the documentation alone",
      "The answer claims three verbosity types including 'VERBOSITY_SUBTESTS = \"subtests\"' - the documentation only mentions VERBOSITY_ASSERTIONS explicitly in the example, the others are plausible but unverifiable from this doc",
      "The answer mentions 'aliases' parameter added in 9.0 and xfail_strict usage - not mentioned in the documentation provided, could be accurate code-level detail or hallucination",
      "The answer claims Config registers itself as 'pytestconfig' plugin at line 1105 - specific line number is unverifiable from the documentation",
      "The answer mentions ArgsSource enum with 'INCOVATION_DIR = INVOCATION_DIR' typo alias - plausible but unverifiable from documentation",
      "The answer's claim about 'legacypath plugin monkeypatches Config._getini_unknown_type' is unverifiable from this documentation",
      "The answer mentions 23+ default plugins including 'subtests' - this is unverifiable and potentially inaccurate as subtests is typically a third-party plugin",
      "The getoption documentation says 'default' is the fallback when option is not declared, but the answer says it resolves through '_parser._opt2dest' mapping - the mechanism is plausible but unverifiable"
    ],
    "strip_verify_leakage": 0.14,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few relied on documentation for rationale ('stat storms'), version info ('added in 9.0'), cross-file references, and high-level descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_02",
    "repo": "pytest",
    "question": "Why does `ParameterSet.extract_from` have a `force_tuple` parameter, and how does the parametrize logic decide when to enable it?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `force_tuple` parameter in `ParameterSet.extract_from` exists to solve an ambiguity problem when parametrizing with a single argument name. The core design issue is:\n\nWhen `@pytest.mark.parametrize(\"arg\", [(1,), (2,)])` is used with a single string argument name (no comma), the tuple values `(1,)` and `(2,)` are ambiguous \u2014 should they be treated as the raw values for `arg` (i.e., `arg=(1,)`) or should they be unpacked as a single-element parameter set (i.e., `arg=1`)?\n\nThe `force_tuple` mechanism resolves this by detecting the argnames format:\n\n1. **`force_tuple=True`**: When argnames is a string with exactly one name and NO trailing comma (e.g., `\"arg\"`), `extract_from` wraps the value via `cls.param(parameterset)`, which uses `*values` to create `values=(parameterset,)`. This treats each argvalue as a raw value to pass to the test, preventing tuple decomposition. So `parametrize(\"arg\", [(1,2)])` gives `arg=(1,2)`.\n\n2. **`force_tuple=False`**: When argnames is a tuple/sequence (e.g., `(\"arg\",)`) or has a trailing comma (e.g., `\"arg,\"`), or has multiple names, `extract_from` directly assigns the parameterset as the `values` field: `cls(parameterset, marks=[], id=None)`. This treats each argvalue as a sequence to be unpacked into multiple arguments. So `parametrize((\"arg\",), [(1,)])` gives `arg=1`.\n\nThe trailing comma detection (`has_trailing_comma = argnames.rstrip().endswith(\",\")`) ensures that `\"arg,\"` behaves like `(\"arg\",)` \u2014 both treat argvalues as lists of tuples to unpack.\n\nAdditional design decisions in the `ParameterSet` class:\n- **`usefixtures` rejection**: `ParameterSet.param()` explicitly checks if any mark has `name == \"usefixtures\"` and raises a `ValueError`.\n- **Empty parameter set handling**: `_for_parametrize` handles empty argvalues by creating a synthetic `ParameterSet` with `NOTSET` sentinel values (one per argname) and applying a configurable mark (skip/xfail/fail_at_collect) based on the `empty_parameter_set_mark` ini option.\n- **`HIDDEN_PARAM` id support**: The `param()` method validates that `id` is either a string, `None`, or the `HIDDEN_PARAM` sentinel (a `_HiddenParam` enum), raising `TypeError` otherwise.",
    "rubric": [
      "Explains the ambiguity: with a single argname, tuple argvalues like (1,) could be interpreted either as a raw value or as a single-element parameter set to unpack",
      "Explains that force_tuple=True wraps the value via cls.param(parameterset), using *values to create values=(parameterset,), preventing tuple decomposition",
      "Explains that force_tuple=False directly assigns parameterset as the values field in the ParameterSet",
      "Explains that force_tuple is set to True when argnames is a string with exactly one name and no trailing comma",
      "Explains the trailing comma distinction: 'arg,' or ('arg',) results in force_tuple=False, treating argvalues as tuples to unpack",
      "Mentions at least one additional design detail: usefixtures rejection, empty parameter set handling with NOTSET sentinels, or HIDDEN_PARAM id validation"
    ],
    "key_files": [
      "src/_pytest/mark/structures.py",
      "src/_pytest/python.py",
      "testing/python/metafunc.py"
    ],
    "source_doc": "[docstring: src/_pytest/mark/structures.py] src._pytest.mark.structures.ParameterSet\nsrc._pytest.mark.structures.ParameterSet:\n  A set of values for a set of parameters along with associated marks and\n    an optional ID for the set.\n\n    Examples::\n\n        pytest.param(1, 2, 3)\n        # ParameterSet(values=(1, 2, 3), marks=(), id=None)\n\n        pytest.param(\"hello\", id=\"greeting\")\n        # ParameterSet(values=(\"hello\",), marks=(), id=\"greeting\")\n\n        # Parameter set with marks\n        pytest.param(42, marks=pytest.mark.xfail)\n        # ParameterSet(values=(42,), marks=(MarkDecorator(...),), id=None)\n\n        # From parametrize mark (parameter names + list of parameter sets)\n        pytest.mark.parametrize(\n            (\"a\", \"b\", \"expected\"),\n            [\n                (1, 2, 3),\n                pytest.param(40, 2, 42, id=\"everything\"),\n            ],\n        )\n        # ParameterSet(values=(1, 2, 3), marks=(), id=None)\n        # ParameterSet(values=(40, 2, 42), marks=(), id=\"everything\")\n\nsrc._pytest.mark.structures.ParameterSet.extract_from:\n  Extract from an object or objects.\n\n        :param parameterset:\n            A legacy style parameterset that may or may not be a tuple,\n            and may or may not be wrapped into a mess of mark objects.\n\n        :param force_tuple:\n            Enforce tuple wrapping so single argument tuple values\n            don't get decomposed and break tests.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a brief description of `force_tuple`: 'Enforce tuple wrapping so single argument tuple values don't get decomposed and break tests.' The answer elaborates extensively on the implementation details (e.g., `cls.param(parameterset)` using `*values`, trailing comma detection, `_parse_parametrize_args`, issue #719, `usefixtures` rejection, `HIDDEN_PARAM`, `NOTSET` sentinels, `empty_parameter_set_mark` ini option) which are not present in the provided documentation.",
      "The detailed implementation claims about `force_tuple=True` vs `force_tuple=False` behavior, while plausible based on pytest source code knowledge, go far beyond what the documentation states.",
      "Claims about `usefixtures` rejection, `HIDDEN_PARAM`, `NOTSET` sentinels, and `empty_parameter_set_mark` are entirely unsupported by the provided documentation chunk.",
      "The reference to issue #719 and `has_trailing_comma` detection is not mentioned in the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Two claims rely on stripped comments (issue #719 reference and TODO comment); the rest are verifiable from code and tests.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_03",
    "repo": "pytest",
    "question": "Where does the `pytest_exception_interact` hook get triggered, and how does data flow to its primary consumer?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `pytest_exception_interact` hook's data flows through two primary call sites in `src/_pytest/runner.py`:\n\n1. **During test execution** (setup/call/teardown phases): The `call_and_report()` function creates a `CallInfo` object from the test phase hook, generates a `TestReport` via `pytest_runtest_makereport`, then calls `check_interactive_exception(call, report)`. If that returns True, it invokes `ihook.pytest_exception_interact(node=item, call=call, report=report)`.\n\n2. **During collection errors**: The `collect_one_node()` function retrieves the `CallInfo` from the `CollectReport`'s `__dict__` (it was stashed via `rep.call = call` in `pytest_make_collect_report`), then pops it with `call = rep.__dict__.pop(\"call\", None)` and similarly checks `check_interactive_exception(call, rep)` before dispatching `ihook.pytest_exception_interact(node=collector, call=call, report=rep)`.\n\nThe `check_interactive_exception()` function filters out exceptions:\n- Returns False if `call.excinfo is None` (no exception)\n- Returns False if `report` has `wasxfail` attribute\n- Returns False if the exception is `Skipped`, `bdb.BdbQuit`, or `unittest.SkipTest`\n\nThe primary consumer of this hook is `PdbInvoke` class in `src/_pytest/debugging.py`, which is registered as a plugin when `--pdb` is passed (registered via `config.pluginmanager.register(PdbInvoke(), \"pdbinvoke\")`). Its `pytest_exception_interact` method suspends global capture, writes captured stdout/stderr to sys.stdout, then calls `_enter_pdb(node, call.excinfo, report)` which displays the traceback and enters post-mortem debugging via `post_mortem()`.\n\nThe `node.addfinalizer()` method (in `src/_pytest/nodes.py`) delegates to `self.session._setupstate.addfinalizer(fin, self)`, which appends the finalizer to the node's entry in the `SetupState.stack` dictionary. The stack stores each node as a key mapped to a tuple of `(finalizers_list, exception_tuple_or_None)`. When setup fails, the exception and traceback are saved, and if that node's subtree items are requested, the exception is re-raised.",
    "rubric": [
      "Identifies `call_and_report()` in runner.py as a call site during test execution (setup/call/teardown phases)",
      "Identifies `collect_one_node()` in runner.py as a call site during collection errors",
      "Explains that `collect_one_node` retrieves the CallInfo from the report's __dict__ (stashed via rep.call = call, then popped)",
      "Describes `check_interactive_exception()` filtering: returns False if call.excinfo is None, report has wasxfail, or exception is Skipped/BdbQuit/unittest.SkipTest",
      "Identifies `PdbInvoke` in debugging.py as the primary consumer, registered when --pdb is passed",
      "Explains that PdbInvoke's implementation suspends capture, writes captured output, and enters post-mortem debugging via _enter_pdb"
    ],
    "key_files": [
      "src/_pytest/runner.py",
      "src/_pytest/debugging.py",
      "src/_pytest/hookspec.py",
      "src/_pytest/fixtures.py",
      "src/_pytest/nodes.py",
      "src/_pytest/monkeypatch.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.4.0.rst] Changes between 2.3.5 and 2.4\nknown incompatibilities:\n\n- if calling --genscript from python2.7 or above, you only get a\n  standalone script which works on python2.7 or above.  Use Python2.6\n  to also get a python2.5 compatible version.\n\n- all xunit-style teardown methods (nose-style, pytest-style,\n  unittest-style) will not be called if the corresponding setup method failed,\n  see issue322 below.\n\n- the pytest_plugin_unregister hook wasn't ever properly called\n  and there is no known implementation of the hook - so it got removed.\n\n- pytest.fixture-decorated functions cannot be generators (i.e. use\n  yield) anymore.  This change might be reversed in 2.4.1 if it causes\n  unforeseen real-life issues.  However, you can always write and return\n  an inner function/generator and change the fixture consumer to iterate\n  over the returned generator.  This change was done in lieu of the new\n  ``pytest.yield_fixture`` decorator, see below.\n\nnew features:\n\n- experimentally introduce a new ``pytest.yield_fixture`` decorator\n  which accepts exactly the same parameters as pytest.fixture but\n  mandates a ``yield`` statement instead of a ``return statement`` from\n  fixture functions.  This allows direct integration with \"with-style\"\n  context managers in fixture functions and generally avoids registering\n  of finalization callbacks in favour of treating the \"after-yield\" as\n  teardown code.  Thanks Andreas Pelme, Vladimir Keleshev, Floris\n  Bruynooghe, Ronny Pfannschmidt and many others for discussions.\n\n- allow boolean expression directly with skipif/xfail\n  if a \"reason\" is also specified.  Rework skipping documentation\n  to recommend \"condition as booleans\" because it prevents surprises\n  when importing markers between modules.  Specifying conditions\n  as strings will remain fully supported.\n\n- reporting: color the last line red or green depending if\n  failures/errors occurred or everything passed.  thanks Christian\n  Theunert.\n\n- make \"import pdb ; pdb.set_trace()\" work natively wrt capturing (no\n  \"-s\" needed anymore), making ``pytest.set_trace()`` a mere shortcut.\n\n- fix issue181: --pdb now also works on collect errors (and\n  on internal errors) .  This was implemented by a slight internal\n  refactoring and the introduction of a new hook\n  ``pytest_exception_interact`` hook (see next item).\n\n- fix issue341: introduce new experimental hook for IDEs/terminals to\n  intercept debugging: ``pytest_exception_interact(node, call, report)``.\n\n- new monkeypatch.setattr() variant to provide a shorter\n  invocation for patching out classes/functions from modules:\n\n     monkeypatch.setattr(\"requests.get\", myfunc)\n\n  will replace the \"get\" function of the \"requests\" module with ``myfunc``.\n\n- fix issue322: tearDownClass is not run if setUpClass failed. Thanks\n  Mathieu Agopian for the initial fix.  Also make all of pytest/nose\n  finalizer mimic the same generic behaviour: if a setupX exists and\n  fails, don't run teardownX.  This internally introduces a new method\n  \"node.addfinalizer()\" helper which can only be called during the setup\n  phase of a node.\n\n- simplify pytest.mark.parametrize() signature: allow to pass a\n  CSV-separated string to specify argnames.  For example:\n  ``pytest.mark.parametrize(\"input,expected\",  [(1,2), (2,3)])``\n  works as well as the previous:\n  ``pytest.mark.parametrize((\"input\", \"expected\"), ...)``.\n\n- add support for setUpModule/tearDownModule detection, thanks Brian Okken.\n\n- integrate tab-completion on options through use of \"argcomplete\".\n  Thanks Anthon van der Neut for the PR.\n\n- change option names to be hyphen-separated long options but keep the\n  old spelling backward compatible.  py.test -h will only show the\n  hyphenated version, for example \"--collect-only\" but \"--collectonly\"\n  will remain valid as well (for backward-compat reasons).  Many thanks to\n  Anthon van der Neut for the implementation and to Hynek Schlawack for\n  pushing us.\n\n- fix issue 308 - allow to mark/xfail/skip individual parameter sets\n  when parametrizing.  Thanks Brianna Laugher.\n\n- call new experimental pytest_load_initial_conftests hook to allow\n  3rd party plugins to do something before a conftest is loaded.\n\nBug fixes:\n\n- fix issue358 - capturing options are now parsed more properly\n  by using a new parser.parse_known_args method.\n\n- pytest now uses argparse instead of optparse (thanks Anthon) which\n  means that \"argparse\" is added as a dependency if installing into python2.6\n  environments or below.\n\n- fix issue333: fix a case of bad unittest/pytest hook interaction.\n\n- PR27: correctly handle nose.SkipTest during collection.  Thanks\n  Antonio Cuni, Ronny Pfannschmidt.\n\n- fix issue355: junitxml puts name=\"pytest\" attribute to testsuite tag.\n\n- fix issue336: autouse fixture in plugins should work again.\n\n- fix issue279: improve object comparisons on assertion failure\n  for standard datatypes and recognise collections.abc.  Thanks to\n  Brianna Laugher and Mathieu Agopian.\n\n- fix issue317: assertion rewriter support for the is_package method\n\n- fix issue335: document py.code.ExceptionInfo() object returned\n  from pytest.raises(), thanks Mathieu Agopian.\n\n- remove implicit distribute_setup support from setup.py.\n\n- fix issue305: ignore any problems when writing pyc files.\n\n- SO-17664702: call fixture finalizers even if the fixture function\n  partially failed (finalizers would not always be called before)\n\n- fix issue320 - fix class scope for fixtures when mixed with\n  module-level functions.  Thanks Anatoly Bubenkoff.\n\n- you can specify \"-q\" or \"-qq\" to get different levels of \"quieter\"\n  reporting (thanks Katarzyna Jachim)\n\n- fix issue300 - Fix order of conftest loading when starting py.test\n  in a subdirectory.\n\n- fix issue323 - sorting of many module-scoped arg parametrizations\n\n- make sessionfinish hooks execute with the same cwd-context as at\n  session start (helps fix plugin behaviour which write output files\n  with relative path such as pytest-cov)\n\n- fix issue316 - properly reference collection hooks in docs\n\n- fix issue 306 - cleanup of -k/-m options to only match markers/test\n  names/keywords respectively.  Thanks Wouter van Ackooy.\n\n- improved doctest counting for doctests in python modules --\n  files without any doctest items will not show up anymore\n  and doctest examples are counted as separate test items.\n  thanks Danilo Bellini.\n\n- fix issue245 by depending on the released py-1.4.14\n  which fixes py.io.dupfile to work with files with no\n  mode. Thanks Jason R. Coombs.\n\n- fix junitxml generation when test output contains control characters,\n  addressing issue267, thanks Jaap Broekhuizen\n\n- fix issue338: honor --tb style for setup/teardown errors as well.  Thanks Maho.\n\n- fix issue307 - use yaml.safe_load in example, thanks Mark Eichin.\n\n- better parametrize error messages, thanks Brianna Laugher\n\n- pytest_terminal_summary(terminalreporter) hooks can now use\n  \".section(title)\" and \".line(msg)\" methods to print extra\n  information at the end of a test run.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extremely specific implementation details (line numbers, function names, class names, internal mechanisms) that are not present in the documentation and cannot be verified from it. These appear to be extrapolations from source code rather than from the provided documentation.",
      "The documentation only mentions that pytest_exception_interact was introduced as 'new experimental hook for IDEs/terminals to intercept debugging: pytest_exception_interact(node, call, report)' and that it helps --pdb work on collect errors. The detailed implementation description in the answer goes far beyond what the documentation states.",
      "The answer's description of node.addfinalizer() includes specific implementation details (line numbers, SetupState.stack dictionary structure) that cannot be verified from the documentation, which only states it 'can only be called during the setup phase of a node'.",
      "The documentation states the hook signature as 'pytest_exception_interact(node, call, report)' which is consistent with the answer's description of the parameters."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the faulthandler.py reference and specific line numbers are not recoverable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_05",
    "repo": "pytest",
    "question": "What is the class hierarchy and architecture of RaisesGroup and RaisesExc, and how do they handle match/check failures differently?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`RaisesGroup` and `RaisesExc` are both `@final` classes that inherit from `AbstractRaises`, an ABC (Abstract Base Class) that is generic over `BaseExcT_co` and inherits from both `ABC` and `Generic[BaseExcT_co]`. They are defined in `src/_pytest/raises.py`.\n\n**Key architectural details:**\n\n1. **`AbstractRaises`** provides shared functionality including:\n   - `_check_match()`: Uses `stringify_exception(e, include_subexception_msg=False)` from `_pytest._code.code` to get the string to match against. For `BaseExceptionGroup` instances, this uses `exc.message` (just the group message) instead of `str(exc)` (which includes sub-exception counts).\n   - `_check_check()`: Calls the user-provided `check` callable. When `self._nested` is True (meaning this object is used inside a `RaisesGroup`), it suppresses the `repr` of the check callable in the error message.\n   - `rawmatch`: A special attribute that stores the unescaped match string when the pattern is a fully-escaped regex bounded by `^...$`. When set, match failures use `_diff_text` from `_pytest.assertion.util` to show a proper diff instead of a regex mismatch message.\n   - An abstract `matches()` method that returns `TypeGuard`.\n\n2. **`RaisesExc`** handles individual exceptions:\n   - `__exit__` uses a `_just_propagate` flag: when type-check fails, it returns `False` (letting the unexpected exception propagate naturally). When match/check fail, it raises `AssertionError` directly.\n   - Requires at least one of `expected_exception`, `match`, or `check` parameters.\n   - `pytest.raises()` is actually a thin wrapper that constructs and returns a `RaisesExc` instance.\n\n3. **`RaisesGroup`** handles exception groups (inherits from `AbstractRaises[BaseExceptionGroup[BaseExcT_co]]`):\n   - Uses `fail()` (from `_pytest.outcomes`) for reporting failures \u2014 different from `RaisesExc`'s `AssertionError`.\n   - Uses a matching algorithm tracked via a `ResultHolder` matrix.\n   - Has a `flatten_subgroups` parameter visible in its overload signatures.\n   - The `is_baseexception` flag is used in determining the expected type.\n\n4. **`ResultHolder`** and **`NotChecked`** are helper classes for the matching algorithm.\n\n5. **Python 3.10 compatibility**: For Python < 3.11, `BaseExceptionGroup` and `ExceptionGroup` are imported from the `exceptiongroup` backport package.",
    "rubric": [
      "Both RaisesGroup and RaisesExc are @final classes inheriting from AbstractRaises, which is an ABC generic over BaseExcT_co that inherits from ABC and Generic[BaseExcT_co]",
      "AbstractRaises provides _check_match() which uses stringify_exception with include_subexception_msg=False, and _check_check() which suppresses check repr when _nested is True",
      "AbstractRaises has a rawmatch attribute that stores the unescaped match string for fully-escaped ^...$ regex patterns, enabling _diff_text-based diff output instead of regex mismatch messages",
      "RaisesExc uses a _just_propagate flag: on type mismatch it returns False (propagates the exception), on match/check failure it raises AssertionError directly",
      "RaisesExc requires at least one of expected_exception, match, or check parameters",
      "RaisesGroup uses fail() from _pytest.outcomes for reporting failures, different from RaisesExc's AssertionError approach",
      "RaisesGroup uses a ResultHolder matrix for its matching algorithm that pairs expected and actual exceptions greedily",
      "ResultHolder and NotChecked are helper classes for the matching algorithm",
      "For Python < 3.11, BaseExceptionGroup and ExceptionGroup are imported from the exceptiongroup backport package",
      "pytest.raises() is a thin wrapper that constructs and returns a RaisesExc instance"
    ],
    "key_files": [
      "src/_pytest/raises.py",
      "src/_pytest/_code/code.py",
      "src/pytest/__init__.py"
    ],
    "source_doc": "[doc_file: doc/en/how-to/assert.rst] Assertions about expected exception groups\nWhen expecting a :exc:`BaseExceptionGroup` or :exc:`ExceptionGroup` you can use :class:`pytest.RaisesGroup`:\n\n.. code-block:: python\n\n    def test_exception_in_group():\n        with pytest.RaisesGroup(ValueError):\n            raise ExceptionGroup(\"group msg\", [ValueError(\"value msg\")])\n        with pytest.RaisesGroup(ValueError, TypeError):\n            raise ExceptionGroup(\"msg\", [ValueError(\"foo\"), TypeError(\"bar\")])\n\n\nIt accepts a ``match`` parameter, that checks against the group message, and a ``check`` parameter that takes an arbitrary callable which it passes the group to, and only succeeds if the callable returns ``True``.\n\n.. code-block:: python\n\n    def test_raisesgroup_match_and_check():\n        with pytest.RaisesGroup(BaseException, match=\"my group msg\"):\n            raise BaseExceptionGroup(\"my group msg\", [KeyboardInterrupt()])\n        with pytest.RaisesGroup(\n            Exception, check=lambda eg: isinstance(eg.__cause__, ValueError)\n        ):\n            raise ExceptionGroup(\"\", [TypeError()]) from ValueError()\n\nIt is strict about structure and unwrapped exceptions, unlike :ref:`except* <except_star>`, so you might want to set the ``flatten_subgroups`` and/or ``allow_unwrapped`` parameters.\n\n.. code-block:: python\n\n    def test_structure():\n        with pytest.RaisesGroup(pytest.RaisesGroup(ValueError)):\n            raise ExceptionGroup(\"\", (ExceptionGroup(\"\", (ValueError(),)),))\n        with pytest.RaisesGroup(ValueError, flatten_subgroups=True):\n            raise ExceptionGroup(\"1st group\", [ExceptionGroup(\"2nd group\", [ValueError()])])\n        with pytest.RaisesGroup(ValueError, allow_unwrapped=True):\n            raise ValueError\n\nTo specify more details about the contained exception you can use :class:`pytest.RaisesExc`\n\n.. code-block:: python\n\n    def test_raises_exc():\n        with pytest.RaisesGroup(pytest.RaisesExc(ValueError, match=\"foo\")):\n            raise ExceptionGroup(\"\", (ValueError(\"foo\")))\n\nThey both supply a method :meth:`pytest.RaisesGroup.matches` :meth:`pytest.RaisesExc.matches` if you want to do matching outside of using it as a :external+python:std:ref:`context manager <context-managers>`. This can be helpful when checking ``.__context__`` or ``.__cause__``.\n\n.. code-block:: python\n\n    def test_matches():\n        exc = ValueError()\n        exc_group = ExceptionGroup(\"\", [exc])\n        if RaisesGroup(ValueError).matches(exc_group):\n            ...\n        # helpful error is available in `.fail_reason` if it fails to match\n        r = RaisesExc(ValueError)\n        assert r.matches(e), r.fail_reason\n\nCheck the documentation on :class:`pytest.RaisesGroup` and :class:`pytest.RaisesExc` for more details and examples.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims `pytest.raises()` is a thin wrapper that constructs and returns a `RaisesExc` instance - this is not mentioned in the documentation and could be a hallucination about implementation details.",
      "The answer claims `RaisesGroup.__exit__` uses `fail()` from `_pytest.outcomes` which raises `_pytest.outcomes.Failed` while `RaisesExc` raises `AssertionError` - this implementation detail is not in the documentation and cannot be verified from it.",
      "The answer claims these were 'added in version 8.4' - not mentioned in the documentation.",
      "The answer provides extensive implementation details (line numbers, internal class names like `ResultHolder`, `NotChecked`, algorithms, etc.) that are not in the documentation and cannot be verified from it.",
      "The answer claims `_check_match()` uses `stringify_exception(e, include_subexception_msg=False)` and that for `BaseExceptionGroup` instances it uses `exc.message` - not verifiable from the documentation.",
      "The documentation shows `RaisesExc` matches individual exceptions and `RaisesGroup` matches exception groups, which is consistent with the answer's description of their roles.",
      "The documentation mentions `match`, `check`, `flatten_subgroups`, `allow_unwrapped` parameters and `.matches()` method, all of which are correctly referenced in the answer.",
      "The answer's description of `rawmatch` and `_diff_text` for showing diffs is not in the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; version info and some truncated-code details need removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_06",
    "repo": "pytest",
    "question": "Why is `pytest.warns` designed the way it is \u2014 specifically regarding re-emitting unmatched warnings, handling control-flow exceptions, and the `re.escape` hint?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `pytest.warns` function exists to provide a structured, assertion-based mechanism for verifying that specific warnings are emitted during test execution, while also ensuring unmatched warnings are properly re-emitted so they aren't silently swallowed. Here's why the design works this way:\n\n1. **Re-emission of unmatched warnings**: The `WarningsChecker.__exit__` method uses a `finally` block to re-emit all warnings that don't match the expected pattern via `warnings.warn_explicit()`. This is done regardless of whether the assertion passed or failed.\n\n2. **BaseException/Exit propagation guard**: The `__exit__` method has an early-return guard that checks if the exception is not an `Exception` (i.e., it's a `BaseException` like `KeyboardInterrupt`) or if it's specifically a `pytest.Exit` instance. Since `Exit` inherits from `Exception` (as visible in outcomes.py) rather than `BaseException`, it needs a special case. This prevents `pytest.warns` from emitting a 'DID NOT WARN' failure when the real cause is a control-flow exception like `pytest.skip`, `pytest.fail`, `pytest.exit`, or Ctrl-C.\n\n3. **`re.escape()` hint**: The code provides a helpful hint suggesting `re.escape()` when the `match` expression fails as a regex but exactly equals the warning message string. This only triggers when the match_expr is a string and literally equals `str(w.message)` for a warning of the expected type \u2014 avoiding false positives for non-matching warning categories.\n\n4. **`simplefilter('always')` vs `simplefilter('default')`**: The `WarningsRecorder.__enter__` sets `warnings.simplefilter('always')` to ensure every warning is captured, even duplicates. But the `recwarn` fixture overrides this with `warnings.simplefilter('default')` after entering, which filters duplicate warnings. This means `pytest.warns` captures duplicate warnings while `recwarn` does not by default.\n\n5. **Non-string warning guard**: The code at the end of `__exit__` guards against warnings whose message argument is not a string. It checks if any `UserWarning` has a non-string first argument and raises a `TypeError` with a descriptive message if so.\n\n6. **`WarningsChecker` is `@final`**: The class is decorated with `@final` from `typing`, preventing subclassing.\n\n7. **`matches()` uses `re.search` not `re.match`**: The `WarningsChecker.matches()` method uses `re.search(self.match_expr, str(warning.message))` \u2014 this means the pattern can match anywhere in the warning message string, not just at the beginning, consistent with how `pytest.raises` works with its `match` parameter.",
    "rubric": [
      "Explains that unmatched warnings are re-emitted via `warnings.warn_explicit()` in a `finally` block so they aren't silently swallowed, regardless of whether the assertion passed or failed",
      "Explains the early-return guard for BaseException/Exit: prevents 'DID NOT WARN' failures when the real cause is a control-flow exception (skip, fail, exit, KeyboardInterrupt), noting that Exit inherits from Exception rather than BaseException and thus needs a special case",
      "Explains the `re.escape()` hint triggers only when match_expr is a string that literally equals `str(w.message)` for a warning of the expected type, helping users who pass regex-special characters",
      "Explains the difference between `simplefilter('always')` in WarningsRecorder.__enter__ (captures all including duplicates) vs `simplefilter('default')` in the recwarn fixture (filters duplicates)",
      "Mentions that `matches()` uses `re.search` (not `re.match`), allowing pattern matches anywhere in the warning message",
      "Mentions the non-string warning guard that raises TypeError when a UserWarning has a non-string first argument",
      "Notes that WarningsChecker is decorated with @final to prevent subclassing"
    ],
    "key_files": [
      "src/_pytest/recwarn.py",
      "src/_pytest/outcomes.py",
      "testing/test_recwarn.py"
    ],
    "source_doc": "[doc_file: doc/en/how-to/capture-warnings.rst] Asserting warnings with the warns function\nYou can check that code raises a particular warning using :func:`pytest.warns`,\nwhich works in a similar manner to :ref:`raises <assertraises>` (except that\n:ref:`raises <assertraises>` does not capture all exceptions, only the\n``expected_exception``):\n\n.. code-block:: python\n\n    import warnings\n\n    import pytest\n\n\n    def test_warning():\n        with pytest.warns(UserWarning):\n            warnings.warn(\"my warning\", UserWarning)\n\nThe test will fail if the warning in question is not raised. Use the keyword\nargument ``match`` to assert that the warning matches a text or regex.\nTo match a literal string that may contain regular expression metacharacters like ``(`` or ``.``, the pattern can\nfirst be escaped with ``re.escape``.\n\nSome examples:\n\n.. code-block:: pycon\n\n\n    >>> with warns(UserWarning, match=\"must be 0 or None\"):\n    ...     warnings.warn(\"value must be 0 or None\", UserWarning)\n    ...\n\n    >>> with warns(UserWarning, match=r\"must be \\d+$\"):\n    ...     warnings.warn(\"value must be 42\", UserWarning)\n    ...\n\n    >>> with warns(UserWarning, match=r\"must be \\d+$\"):\n    ...     warnings.warn(\"this is not here\", UserWarning)\n    ...\n    Traceback (most recent call last):\n      ...\n    Failed: Regex pattern did not match any of the 1 warnings emitted.\n     Regex: ...\n     Emitted warnings: ...UserWarning...\n\n    >>> with warns(UserWarning, match=re.escape(\"issue with foo() func\")):\n    ...     warnings.warn(\"issue with foo() func\")\n    ...\n\nThe function also returns a list of all raised warnings (as\n``warnings.WarningMessage`` objects), which you can query for\nadditional information:\n\n.. code-block:: python\n\n    with pytest.warns(RuntimeWarning) as record:\n        warnings.warn(\"another warning\", RuntimeWarning)\n\n    # check that only one warning was raised\n    assert len(record) == 1\n    # check that the message matches\n    assert record[0].message.args[0] == \"another warning\"\n\nAlternatively, you can examine raised warnings in detail using the\n:fixture:`recwarn` fixture (see :ref:`below <recwarn>`).\n\n\nThe :fixture:`recwarn` fixture automatically ensures to reset the warnings\nfilter at the end of the test, so no global state is leaked.\n\n.. _`recording warnings`:\n\n.. _recwarn:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation-level details (line numbers, internal method names, class decorators, CPython bug workarounds, etc.) that are not present in the provided documentation. These appear to be derived from source code rather than the documentation chunk provided.",
      "Claim about re-emission of unmatched warnings since pytest 8.0 is not mentioned in the documentation - this is plausible but unsupported by the given doc.",
      "Claim about BaseException/Exit propagation guard (issue #11907) is not mentioned in the documentation - plausible code-level detail but unsupported.",
      "Claim about simplefilter('always') vs simplefilter('default') behavior difference between pytest.warns and recwarn is not in the documentation - the doc only says recwarn 'automatically ensures to reset the warnings filter at the end of the test'.",
      "Claim about non-string warning guard (CPython bug #10865) is not in the documentation.",
      "Claim about WarningsChecker being @final is not in the documentation.",
      "Claim that re.search is used (not re.match) is not in the documentation, though the doc's examples with match=r'must be \\d+$' using $ anchoring are consistent with re.search behavior.",
      "The documentation states pytest.warns returns 'a list of all raised warnings (as warnings.WarningMessage objects)' - the answer doesn't contradict this.",
      "The documentation's description of re.escape usage is correctly reflected in the answer.",
      "The answer correctly describes the match parameter functionality as shown in the documentation."
    ],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "Minor doc leakage in issue references, version notes, and design rationale comments; core behavioral claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_08",
    "repo": "pytest",
    "question": "How does `pytest_runtest_logreport` get invoked for each test phase, and how do different plugins handle setup/teardown vs call reports?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `pytest_runtest_logreport` hook is called unconditionally for all three phases (setup, call, teardown) through the `call_and_report` function in `src/_pytest/runner.py`. Here's how the algorithm works:\n\n1. **`runtestprotocol()`** orchestrates the test execution by calling `call_and_report()` for each phase:\n   - Always calls `call_and_report(item, \"setup\", log)` first\n   - Only calls `call_and_report(item, \"call\", log)` if setup passed AND `setuponly` option is not set\n   - **Always** calls `call_and_report(item, \"teardown\", log, nextitem=nextitem)` regardless of whether setup/call passed or failed\n\n2. **`call_and_report()`** handles the actual hook invocation:\n   - Selects the appropriate runtest hook based on `when` (setup/call/teardown)\n   - Wraps the hook execution in `CallInfo.from_call()` which captures exceptions and timing\n   - Calls `ihook.pytest_runtest_makereport(item=item, call=call)` to generate a `TestReport`\n   - If `log=True` (default), **unconditionally** calls `ihook.pytest_runtest_logreport(report=report)` \u2014 there is no condition checking whether the report is for setup/teardown vs call\n   - Then checks `check_interactive_exception()` for debugger interaction\n\n3. **Handling setup/teardown reports in plugins:**\n   - The `pytest_report_teststatus` hook in runner.py returns `(\"\", \"\", \"\")` for passing setup/teardown phases\n   - The `Session.pytest_runtest_logreport` (main.py) counts ALL failures including setup/teardown \u2014 it increments `self.testsfailed` for any failed report without `wasxfail`, regardless of `report.when`\n   - The `cacheprovider` plugin's `pytest_runtest_logreport` specifically checks `report.when == \"call\" and report.passed` to only clear lastfailed on call-phase passes, but marks nodeid as failed for ANY phase failure\n   - The `stepwise` plugin does NOT filter by `report.when` for failures, so a setup failure will also trigger stepwise stopping\n   - The `junitxml` plugin handles each phase differently: passes are only recorded for `when == \"call\"`, failures during teardown trigger a special double-failure case, and setup/teardown failures are recorded as errors rather than failures\n\nThe key implementation detail is that the `log` parameter in `call_and_report()` is the only gate \u2014 when `log=True`, the hook is called for ALL phases unconditionally. The `TestReport.when` attribute (a `Literal[\"setup\", \"call\", \"teardown\"]`) allows plugin implementors to distinguish phases and filter accordingly.",
    "rubric": [
      "Explains that `runtestprotocol()` calls `call_and_report()` for setup, conditionally for call (only if setup passed and not setuponly), and always for teardown",
      "Explains that `call_and_report()` unconditionally calls `ihook.pytest_runtest_logreport(report=report)` when `log=True`, with no filtering by phase",
      "Mentions that `pytest_report_teststatus` in runner.py returns empty strings ('', '', '') for passing setup/teardown reports but returns 'error'/'E'/'ERROR' for failed ones",
      "Describes that `Session.pytest_runtest_logreport` increments `testsfailed` for ANY failed report (regardless of `report.when`) unless it has `wasxfail`",
      "Describes that `cacheprovider`'s implementation only clears lastfailed when `report.when == 'call' and report.passed`, but marks failures for any phase",
      "Describes that the `stepwise` plugin does not filter by `report.when` for failures, so setup failures also trigger stopping",
      "Describes that the `junitxml` plugin records passes only for `when == 'call'`, records setup/teardown failures as errors, and handles teardown-after-call-failure as a double-failure case",
      "Mentions that the `TestReport.when` attribute allows plugins to distinguish phases"
    ],
    "key_files": [
      "src/_pytest/runner.py",
      "src/_pytest/reports.py",
      "src/_pytest/hookspec.py",
      "src/_pytest/main.py",
      "src/_pytest/terminal.py",
      "src/_pytest/junitxml.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.2.0.rst] notes on incompatibility\nWhile test suites should work unchanged you might need to upgrade plugins:\n\n* You need a new version of the pytest-xdist plugin (1.7) for distributing\n  test runs.\n\n* Other plugins might need an upgrade if they implement\n  the ``pytest_runtest_logreport`` hook which now is called unconditionally\n  for the setup/teardown fixture phases of a test. You may choose to\n  ignore setup/teardown failures by inserting \"if rep.when != 'call': return\"\n  or something similar. Note that most code probably \"just\" works because\n  the hook was already called for failing setup/teardown phases of a test\n  so a plugin should have been ready to grok such reports already.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only one comment-based claim ('Probably passed setup/teardown') was removed, and some line number references were dropped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_07",
    "repo": "pytest",
    "question": "Where is the doctest collection logic implemented and how does pytest decide which files to collect as doctests?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The doctest collection functionality is implemented in `src/_pytest/doctest.py`. The entry point for collecting doctest files is the `pytest_collect_file` hook function (line 126), which decides whether to create a `DoctestTextfile` or `DoctestModule` collector based on file type and configuration.\n\n**Key location details:**\n\n1. **`pytest_collect_file` hook** (line 126-138): For `.py` files, if `--doctest-modules` is enabled, it creates a `DoctestModule` collector (excluding `setup.py` files that contain 'setuptools' or 'distutils', and `__main__.py` files). For non-Python files, it delegates to `_is_doctest()` to decide if a `DoctestTextfile` should be created.\n\n2. **`_is_doctest` function** (line 148-152): This determines whether a non-Python file should be collected as a doctest. It has two paths: (a) files with `.txt` or `.rst` suffix that are explicitly given as initial paths are always collected; (b) otherwise, it checks against the `doctestglob` option patterns (defaulting to `[\"test*.txt\"]` if none provided) using `fnmatch_ex` from `_pytest/pathlib.py`.\n\n3. **`DoctestTextfile`** (line 420-448): Inherits from `Module`, sets `obj = None`, reads the file using the `doctest_encoding` ini option (default: 'utf-8'), parses it with `doctest.DocTestParser`, and yields `DoctestItem` instances only if `test.examples` is non-empty.\n\n4. **`DoctestModule`** (line 500-573): Also inherits from `Module`, uses a nested `MockAwareDocTestFinder` class (extending `doctest.DocTestFinder`).\n\n5. **`DoctestItem`** (line 251): Inherits from `Item`. During `setup()`, it injects `getfixture` (mapped to `self._request.getfixturevalue`) and all entries from the `doctest_namespace` fixture into the doctest's globals.\n\n6. **`PytestDoctestRunner`** (created lazily in `_init_runner_class`, line 178-228): Extends `doctest.DebugRunner` (not `DocTestRunner`). It collects failures into a list when `continue_on_failure=True`, otherwise raises immediately. It also handles `OutcomeException` (re-raising it) and `bdb.BdbQuit` (calling `outcomes.exit`).\n\n7. **Pattern matching via `fnmatch_ex`** in `src/_pytest/pathlib.py` (line 419-452): If the pattern contains no path separator, it matches against only `path.name`; otherwise it matches against the full string path. On Windows, Posix path separators in patterns are replaced with Windows separators.",
    "rubric": [
      "Identifies src/_pytest/doctest.py as the main location for doctest collection",
      "Mentions the pytest_collect_file hook as the entry point that decides between DoctestTextfile and DoctestModule",
      "Explains that .py files use DoctestModule when --doctest-modules is enabled, excluding setup.py with setuptools/distutils and __main__.py",
      "Describes the _is_doctest function for non-Python files, including .txt/.rst suffix handling and doctestglob pattern matching",
      "Mentions DoctestTextfile reads files using doctest_encoding and parses with DocTestParser",
      "Mentions DoctestModule uses MockAwareDocTestFinder extending doctest.DocTestFinder",
      "Describes DoctestItem's setup injecting getfixture and doctest_namespace into test globals",
      "Mentions PytestDoctestRunner extends doctest.DebugRunner with continue_on_failure support"
    ],
    "key_files": [
      "src/_pytest/doctest.py",
      "src/_pytest/pathlib.py",
      "src/_pytest/python.py"
    ],
    "source_doc": "[doc_file: doc/en/how-to/doctest.rst] How to run doctests\nBy default, all files matching the ``test*.txt`` pattern will\nbe run through the python standard :mod:`doctest` module.  You\ncan change the pattern by issuing:\n\n.. code-block:: bash\n\n    pytest --doctest-glob=\"*.rst\"\n\non the command line. :option:`--doctest-glob` can be given multiple times in the command-line.\n\nIf you then have a text file like this:\n\n.. code-block:: text\n\n    # content of test_example.txt\n\n    hello this is a doctest\n    >>> x = 3\n    >>> x\n    3\n\nthen you can just invoke ``pytest`` directly:\n\n.. code-block:: pytest\n\n    $ pytest\n    =========================== test session starts ============================\n    platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y\n    rootdir: /home/sweet/project\n    collected 1 item\n\n    test_example.txt .                                                   [100%]\n\n    ============================ 1 passed in 0.12s =============================\n\nBy default, pytest will collect ``test*.txt`` files looking for doctest directives, but you\ncan pass additional globs using the :option:`--doctest-glob` option (multi-allowed).\n\nIn addition to text files, you can also execute doctests directly from docstrings of your classes\nand functions, including from test modules:\n\n.. code-block:: python\n\n    # content of mymodule.py\n    def something():\n        \"\"\"a doctest in a docstring\n        >>> something()\n        42\n        \"\"\"\n        return 42\n\n.. code-block:: bash\n\n    $ pytest --doctest-modules\n    =========================== test session starts ============================\n    platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y\n    rootdir: /home/sweet/project\n    collected 2 items\n\n    mymodule.py .                                                        [ 50%]\n    test_example.txt .                                                   [100%]\n\n    ============================ 2 passed in 0.12s =============================\n\nYou can make these changes permanent in your project by\nputting them into a configuration file like this:\n\n.. code-block:: toml\n\n    # content of pytest.toml\n    [pytest]\n    addopts = [\"--doctest-modules\"]",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped source code; minor rewrites needed for truncated sections and Module's full inheritance chain.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_09",
    "repo": "pytest",
    "question": "What components and steps are involved when pytest parametrizes fixtures, from registration through test ID generation, caching, and item reordering?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "When pytest parametrizes fixtures, the process involves several interconnected components:\n\n1. **Fixture Registration**: The `@pytest.fixture(params=...)` decorator creates a `FixtureFunctionMarker` (a frozen dataclass in `src/_pytest/fixtures.py`) which stores params as a tuple (converted from the iterable via `tuple(params)`). The marker then creates a `FixtureFunctionDefinition` which wraps the original function and prevents direct calling (raising an error with a specific message about fixtures not being meant to be called directly).\n\n2. **Test Generation via `pytest_generate_tests`**: The `FixtureManager.pytest_generate_tests` hook iterates over `metafunc.fixturenames` and for each fixture with non-None `params`, calls `metafunc.parametrize()` with `indirect=True` and `scope=fixturedef.scope`. This means fixture parametrization is internally implemented as indirect parametrization.\n\n3. **ID Generation**: The `IdMaker` class (in `src/_pytest/python.py`) generates test IDs. The `_idval` method tries four strategies in order: (a) `_idval_from_function` - calls user-provided `idfn` callable, (b) `_idval_from_hook` - calls `pytest_make_parametrize_id` hook, (c) `_idval_from_value` - handles known types, (d) `_idval_from_argname` - fallback generating `argname + idx` string.\n\n4. **Value-based ID generation** (`_idval_from_value`): Supports `str|bytes` (ASCII-escaped), `None|float|int|bool|complex` (via `str()`), `re.Pattern` (escaped pattern), `enum.Enum` (via `str()`), and objects with a `__name__` attribute. NOTSET is explicitly skipped even though it's an enum. For unsupported types, returns None causing fallback to `argname + index`.\n\n5. **Duplicate ID handling**: `make_unique_parameterset_ids` detects duplicate IDs and appends a numeric suffix. If the ID already ends with a digit, an underscore separator is added before the suffix (e.g., `id_0`, `id_1`). If `strict_parametrization_ids` config is enabled, duplicates raise a `CollectError` instead.\n\n6. **Cache Key Mechanism**: `FixtureDef.cache_key` uses `getattr(request, \"param\", None)` - the `param` attribute is only set on `SubRequest` when the value is not `NOTSET`.\n\n7. **SubRequest param assignment**: In `SubRequest.__init__`, `self.param = param` is only set if `param is not NOTSET`. The `NOTSET` sentinel is defined as `NotSetType.token` where `NotSetType` is an `enum.Enum` with a single member `token = 0` (in `src/_pytest/compat.py`).\n\n8. **Fixture Teardown for Generators**: `call_fixture_func` detects generator functions (yield fixtures) and registers a `_teardown_yield_fixture` finalizer. This finalizer advances the generator past the yield point and fails if there's more than one yield.\n\n9. **Item Reordering**: After collection, `pytest_collection_modifyitems` calls `reorder_items()` which groups tests sharing the same parametrized fixture value together, operating on scopes higher than function (session, package, module, class) as defined by `HIGH_SCOPES`.",
    "rubric": [
      "Explains that `FixtureFunctionMarker` stores params as a tuple and creates a `FixtureFunctionDefinition` that prevents direct calling of the fixture function",
      "Describes that `FixtureManager.pytest_generate_tests` iterates over `metafunc.fixturenames` and calls `metafunc.parametrize()` with `indirect=True` and `scope=fixturedef.scope` for fixtures with non-None params",
      "Explains the `IdMaker._idval` method's four-strategy fallback chain: `_idval_from_function` (user idfn), `_idval_from_hook` (pytest_make_parametrize_id hook), `_idval_from_value` (known types), and `_idval_from_argname` (argname+index fallback)",
      "Lists the types handled by `_idval_from_value`: str/bytes (ASCII-escaped), None/float/int/bool/complex (str()), re.Pattern (escaped pattern), enum.Enum (str()), and objects with `__name__`, noting NOTSET is explicitly skipped despite being an enum",
      "Describes duplicate ID handling in `make_unique_parameterset_ids`: appends numeric suffix, with underscore separator if ID already ends in a digit, and optionally raises CollectError if `strict_parametrization_ids` is enabled",
      "Explains that `FixtureDef.cache_key` uses `getattr(request, 'param', None)` and that `SubRequest.__init__` only sets `self.param` when param is not NOTSET",
      "Mentions the NOTSET sentinel is defined as `NotSetType.token` where `NotSetType` is an `enum.Enum` with value 0",
      "Describes how `call_fixture_func` handles generator/yield fixtures by calling next() for the value and registering `_teardown_yield_fixture` as a finalizer that fails on multiple yields",
      "Explains that `reorder_items` groups tests sharing parametrized fixture values together, operating on HIGH_SCOPES (all scopes except Function)"
    ],
    "key_files": [
      "src/_pytest/fixtures.py",
      "src/_pytest/python.py",
      "src/_pytest/compat.py"
    ],
    "source_doc": "[doc_file: doc/en/how-to/fixtures.rst] Parametrizing fixtures\nFixture functions can be parametrized in which case they will be called\nmultiple times, each time executing the set of dependent tests, i.e. the\ntests that depend on this fixture.  Test functions usually do not need\nto be aware of their re-running.  Fixture parametrization helps to\nwrite exhaustive functional tests for components which themselves can be\nconfigured in multiple ways.\n\nExtending the previous example, we can flag the fixture to create two\n``smtp_connection`` fixture instances which will cause all tests using the fixture\nto run twice.  The fixture function gets access to each parameter\nthrough the special :py:class:`request <pytest.FixtureRequest>` object:\n\n.. code-block:: python\n\n    # content of conftest.py\n    import smtplib\n\n    import pytest\n\n\n    @pytest.fixture(scope=\"module\", params=[\"smtp.gmail.com\", \"mail.python.org\"])\n    def smtp_connection(request):\n        smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)\n        yield smtp_connection\n        print(f\"finalizing {smtp_connection}\")\n        smtp_connection.close()\n\nThe main change is the declaration of ``params`` with\n:py:func:`@pytest.fixture <pytest.fixture>`, a list of values\nfor each of which the fixture function will execute and can access\na value via ``request.param``.  No test function code needs to change.\nSo let's just do another run:\n\n.. code-block:: pytest\n\n    $ pytest -q test_module.py\n    FFFF                                                                 [100%]\n    ================================= FAILURES =================================\n    ________________________ test_ehlo[smtp.gmail.com] _________________________\n\n    smtp_connection = <smtplib.SMTP object at 0xdeadbeef0004>\n\n        def test_ehlo(smtp_connection):\n            response, msg = smtp_connection.ehlo()\n            assert response == 250\n            assert b\"smtp.gmail.com\" in msg\n    >       assert 0  # for demo purposes\n            ^^^^^^^^\n    E       assert 0\n\n    test_module.py:7: AssertionError\n    ________________________ test_noop[smtp.gmail.com] _________________________\n\n    smtp_connection = <smtplib.SMTP object at 0xdeadbeef0004>\n\n        def test_noop(smtp_connection):\n            response, msg = smtp_connection.noop()\n            assert response == 250\n    >       assert 0  # for demo purposes\n            ^^^^^^^^\n    E       assert 0\n\n    test_module.py:13: AssertionError\n    ________________________ test_ehlo[mail.python.org] ________________________\n\n    smtp_connection = <smtplib.SMTP object at 0xdeadbeef0005>\n\n        def test_ehlo(smtp_connection):\n            response, msg = smtp_connection.ehlo()\n            assert response == 250\n    >       assert b\"smtp.gmail.com\" in msg\n    E       AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\\nPIPELINING\\nSIZE 51200000\\nETRN\\nSTARTTLS\\nAUTH DIGEST-MD5 NTLM CRAM-MD5\\nENHANCEDSTATUSCODES\\n8BITMIME\\nDSN\\nSMTPUTF8\\nCHUNKING'\n\n    test_module.py:6: AssertionError\n    -------------------------- Captured stdout setup ---------------------------\n    finalizing <smtplib.SMTP object at 0xdeadbeef0004>\n    ________________________ test_noop[mail.python.org] ________________________\n\n    smtp_connection = <smtplib.SMTP object at 0xdeadbeef0005>\n\n        def test_noop(smtp_connection):\n            response, msg = smtp_connection.noop()\n            assert response == 250\n    >       assert 0  # for demo purposes\n            ^^^^^^^^\n    E       assert 0\n\n    test_module.py:13: AssertionError\n    ------------------------- Captured stdout teardown -------------------------\n    finalizing <smtplib.SMTP object at 0xdeadbeef0005>\n    ========================= short test summary info ==========================\n    FAILED test_module.py::test_ehlo[smtp.gmail.com] - assert 0\n    FAILED test_module.py::test_noop[smtp.gmail.com] - assert 0\n    FAILED test_module.py::test_ehlo[mail.python.org] - AssertionError: asser...\n    FAILED test_module.py::test_noop[mail.python.org] - assert 0\n    4 failed in 0.12s\n\nWe see that our two test functions each ran twice, against the different\n``smtp_connection`` instances.  Note also, that with the ``mail.python.org``\nconnection the second test fails in ``test_ehlo`` because a\ndifferent server string is expected than what arrived.\n\npytest will build a string that is the test ID for each fixture value\nin a parametrized fixture, e.g. ``test_ehlo[smtp.gmail.com]`` and\n``test_ehlo[mail.python.org]`` in the above examples.  These IDs can\nbe used with :option:`-k` to select specific cases to run, and they will\nalso identify the specific case when one is failing.  Running pytest\nwith :option:`--collect-only` will show the generated IDs.\n\nNumbers, strings, booleans and ``None`` will have their usual string\nrepresentation used in the test ID. For other objects, pytest will\nmake a string based on the argument name.  It is possible to customise\nthe string used in a test ID for a certain fixture value by using the\n``ids`` keyword argument:\n\n.. code-block:: python\n\n   # content of test_ids.py\n   import pytest\n\n\n   @pytest.fixture(params=[0, 1], ids=[\"spam\", \"ham\"])\n   def a(request):\n       return request.param\n\n\n   def test_a(a):\n       pass\n\n\n   def idfn(fixture_value):\n       if fixture_value == 0:\n           return \"eggs\"\n       else:\n           return None\n\n\n   @pytest.fixture(params=[0, 1], ids=idfn)\n   def b(request):\n       return request.param\n\n\n   def test_b(b):\n       pass\n\nThe above shows how ``ids`` can be either a list of strings to use or\na function which will be called with the fixture value and then\nhas to return a string to use.  In the latter case if the function\nreturns ``None`` then pytest's auto-generated ID will be used.\n\nRunning the above tests results in the following test IDs being used:\n\n.. code-block:: pytest\n\n   $ pytest --collect-only\n   =========================== test session starts ============================\n   platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y\n   rootdir: /home/sweet/project\n   collected 12 items\n\n   <Dir fixtures.rst-234>\n     <Module test_anothersmtp.py>\n       <Function test_showhelo[smtp.gmail.com]>\n       <Function test_showhelo[mail.python.org]>\n     <Module test_emaillib.py>\n       <Function test_email_received>\n     <Module test_finalizers.py>\n       <Function test_bar>\n     <Module test_ids.py>\n       <Function test_a[spam]>\n       <Function test_a[ham]>\n       <Function test_b[eggs]>\n       <Function test_b[1]>\n     <Module test_module.py>\n       <Function test_ehlo[smtp.gmail.com]>\n       <Function test_noop[smtp.gmail.com]>\n       <Function test_ehlo[mail.python.org]>\n       <Function test_noop[mail.python.org]>\n\n   ======================= 12 tests collected in 0.12s ========================\n\n.. _`fixture-parametrize-marks`:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, class names, internal mechanisms) that are not present in the provided documentation. These appear to be derived from source code knowledge rather than the documentation chunk provided.",
      "The documentation states that for objects other than numbers, strings, booleans, and None, 'pytest will make a string based on the argument name' - the answer's point 4 mentions additional types like re.Pattern, enum.Enum, complex, bytes, and objects with __name__ attribute, which goes beyond what the documentation states but doesn't directly contradict it.",
      "The answer's description of ID generation strategies (point 3) is consistent with the documented behavior showing ids as either a list of strings or a callable function, though the internal implementation details (IdMaker class, line numbers) cannot be verified from the documentation alone.",
      "The answer's claim about duplicate ID handling with strict_parametrization_ids config (point 5) is not mentioned in the documentation and cannot be verified.",
      "The answer's claim about item reordering (point 9) to minimize fixture setup/teardown is not explicitly stated in the documentation, though the output order in the example (both gmail tests first, then both python.org tests) is consistent with this behavior.",
      "The documentation shows the idfn returning None causes 'pytest's auto-generated ID will be used', and the answer's point 3d about fallback to 'argname + idx' is consistent with the test output showing test_b[1] (where idfn returned None for value 1)."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with line number references and one design rationale claim that was documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_10",
    "repo": "pytest",
    "question": "Why does the pytester plugin disable garbage collection and default to in-process execution for inline test runs?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The pytester plugin uses several performance optimizations in its design to make inline test execution fast and efficient:\n\n1. **GC collection suppression for inline runs**: The most explicit performance optimization is in `inline_run()`, where a `PytesterHelperPlugin` sets `config.stash[gc_collect_iterations_key] = 0` during `pytest_configure`. Without this override, the default behavior in `_pytest/unraisableexception.py` would call `gc.collect()` up to 5 times (the default value in `config.stash.get(gc_collect_iterations_key, 5)`) at the end of each test session's cleanup phase. Setting it to 0 means `gc_collect_harder()` loops zero times, completely skipping garbage collection in inner runs.\n\n2. **In-process execution as default mode**: The `--runpytest` option defaults to `\"inprocess\"` rather than `\"subprocess\"`, avoiding the overhead of spawning a new Python interpreter process for each test. The `runpytest()` method checks `self._method` and dispatches accordingly.\n\n3. **`importlib.invalidate_caches()` call**: In `inline_run()`, `importlib.invalidate_caches()` is called to ensure the importlib cache is properly updated between file creation and inline_run.\n\n4. **`perf_counter` for precise duration measurement**: The `RunResult.duration` is measured using `timing.Instant` which internally uses `time.perf_counter()` (via `Duration.seconds` property) for monotonic, high-resolution timing rather than wall clock time. This ensures timing measurements are not affected by system clock adjustments.\n\n5. **SysModulesSnapshot with selective preservation**: The `__take_sys_modules_snapshot` method preserves modules starting with \"zope\" and \"readline\" to avoid the overhead and issues of reloading them.",
    "rubric": [
      "Explains that gc_collect_iterations_key is set to 0 in PytesterHelperPlugin.pytest_configure to skip gc.collect() calls during session cleanup",
      "Mentions that without this override, gc_collect_harder would call gc.collect() up to 5 times (the default iteration count)",
      "Notes that --runpytest defaults to 'inprocess' to avoid the overhead of spawning a new subprocess",
      "Mentions importlib.invalidate_caches() is called to ensure proper cache state between file creation and inline_run",
      "Mentions sys.modules snapshot with selective preservation of 'zope' and 'readline' modules to avoid reloading issues"
    ],
    "key_files": [
      "src/_pytest/pytester.py",
      "src/_pytest/unraisableexception.py",
      "src/_pytest/timing.py",
      "src/_pytest/assertion/rewrite.py"
    ],
    "source_doc": "[doc_file: doc/en/how-to/writing_plugins.rst] Testing plugins\npytest comes with a plugin named ``pytester`` that helps you write tests for\nyour plugin code. The plugin is disabled by default, so you will have to enable\nit before you can use it.\n\nYou can do so by adding the following line to a ``conftest.py`` file in your\ntesting directory:\n\n.. code-block:: python\n\n    # content of conftest.py\n\n    pytest_plugins = [\"pytester\"]\n\nAlternatively you can invoke pytest with the ``-p pytester`` command line\noption.\n\nThis will allow you to use the :py:class:`pytester <pytest.Pytester>`\nfixture for testing your plugin code.\n\nLet's demonstrate what you can do with the plugin with an example. Imagine we\ndeveloped a plugin that provides a fixture ``hello`` which yields a function\nand we can invoke this function with one optional parameter. It will return a\nstring value of ``Hello World!`` if we do not supply a value or ``Hello\n{value}!`` if we do supply a string value.\n\n.. code-block:: python\n\n    import pytest\n\n\n    def pytest_addoption(parser):\n        group = parser.getgroup(\"helloworld\")\n        group.addoption(\n            \"--name\",\n            action=\"store\",\n            dest=\"name\",\n            default=\"World\",\n            help='Default \"name\" for hello().',\n        )\n\n\n    @pytest.fixture\n    def hello(request):\n        name = request.config.getoption(\"name\")\n\n        def _hello(name=None):\n            if not name:\n                name = request.config.getoption(\"name\")\n            return f\"Hello {name}!\"\n\n        return _hello\n\n\nNow the ``pytester`` fixture provides a convenient API for creating temporary\n``conftest.py`` files and test files. It also allows us to run the tests and\nreturn a result object, with which we can assert the tests' outcomes.\n\n.. code-block:: python\n\n    def test_hello(pytester):\n        \"\"\"Make sure that our plugin works.\"\"\"\n\n        # create a temporary conftest.py file\n        pytester.makeconftest(\n            \"\"\"\n            import pytest\n\n            @pytest.fixture(params=[\n                \"Brianna\",\n                \"Andreas\",\n                \"Floris\",\n            ])\n            def name(request):\n                return request.param\n        \"\"\"\n        )\n\n        # create a temporary pytest test file\n        pytester.makepyfile(\n            \"\"\"\n            def test_hello_default(hello):\n                assert hello() == \"Hello World!\"\n\n            def test_hello_name(hello, name):\n                assert hello(name) == \"Hello {0}!\".format(name)\n        \"\"\"\n        )\n\n        # run all tests with pytest\n        result = pytester.runpytest()\n\n        # check that all 4 tests passed\n        result.assert_outcomes(passed=4)\n\n\nAdditionally it is possible to copy examples to the ``pytester``'s isolated environment\nbefore running pytest on it. This way we can abstract the tested logic to separate files,\nwhich is especially useful for longer tests and/or longer ``conftest.py`` files.\n\nNote that for ``pytester.copy_example`` to work we need to set `pytester_example_dir`\nin our configuration file to tell pytest where to look for example files.\n\n.. code-block:: toml\n\n    # content of pytest.toml\n    [pytest]\n    pytester_example_dir = \".\"\n\n\n.. code-block:: python\n\n    # content of test_example.py\n\n\n    def test_plugin(pytester):\n        pytester.copy_example(\"test_example.py\")\n        pytester.runpytest(\"-k\", \"test_example\")\n\n\n    def test_example():\n        pass\n\n.. code-block:: pytest\n\n    $ pytest\n    =========================== test session starts ============================\n    platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y\n    rootdir: /home/sweet/project\n    configfile: pytest.toml\n    collected 2 items\n\n    test_example.py ..                                                   [100%]\n\n    ============================ 2 passed in 0.12s =============================\n\nFor more information about the result object that ``runpytest()`` returns, and\nthe methods that it provides please check out the :py:class:`RunResult\n<_pytest.pytester.RunResult>` documentation.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses internal implementation details (GC suppression, assertion rewriting, SysModulesSnapshot, etc.) that are not covered in the provided documentation chunk. The documentation only describes how to use pytester for testing plugins, not its internal performance optimizations.",
      "None of the 6 claims in the answer are directly supported by the provided documentation. However, they appear to be plausible code-level implementation details that could be verified from the source code, rather than contradictions of the documentation.",
      "The answer does not address the actual content of the documentation (how to enable pytester, how to use it for writing plugin tests, the hello fixture example, copy_example functionality, etc.).",
      "The answer appears to be responding to a different question ('what performance optimizations does pytester use?') than what the documentation covers ('how to use pytester for testing plugins')."
    ],
    "strip_verify_leakage": 0.33,
    "strip_verify_summary": "Several claims rely on comments/docstrings (PYTEST_DONT_REWRITE in docstring, comment about cpython bug, Trio project reference, Python bug #41033), while core code logic claims are verifiable from stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_11",
    "repo": "pytest",
    "question": "Where is dynamic fixture resolution implemented in pytest, and where are the proposed `fixture_request` and `define_combined_fixture` APIs actually defined?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `pytest.fixture_request()` and `pytest.define_combined_fixture()` functions mentioned in the documentation are **proposals only** \u2014 they do not exist anywhere in the pytest source code. Neither function is implemented, exported, or defined. The `src/pytest/__init__.py` file (the public API module) exports neither `fixture_request` nor `define_combined_fixture` in its `__all__` list. The entire proposal document at `doc/en/proposals/parametrize_with_fixtures.rst` is marked as `:orphan:` (line 1) and carries a warning that it only \"outlines a proposal.\"\n\nThe current mechanism for dynamic fixture resolution is `request.getfixturevalue(argname)`, defined in `FixtureRequest.getfixturevalue()` in `src/_pytest/fixtures.py`. However, it **cannot** be combined with parametrized fixtures \u2014 the method `_check_fixturedef_without_param()` is called when `getfixturevalue` is used on a fixture that has `params` but the current test didn't go through parametrize collection for it, raising an error.\n\nThe old `request.getfuncargvalue()` function mentioned in the proposal's first code example has been **completely removed** from the codebase \u2014 no references to `getfuncargvalue` exist anywhere in the source.\n\nFor existing parametrization, pytest uses `Metafunc.parametrize()` (in `src/_pytest/python.py`) which creates `CallSpec2` objects. The `_resolve_args_directness()` function exists in `src/_pytest/fixtures.py` (imported in `python.py`) and handles the directness resolution of parametrized arguments.\n\nThe `FixtureRequest` class hierarchy consists of the abstract `FixtureRequest` (using `abc.ABC`), and subclasses including `SubRequest`. `SubRequest` stores a `_parent_request` reference forming a chain (as seen in `_iter_chain`), and param handling uses `NOTSET` to determine whether a param value is set.\n\nCalling `getfixturevalue` during teardown now triggers `FIXTURE_GETFIXTUREVALUE_DURING_TEARDOWN` (a `PytestRemovedIn10Warning`, defined in `src/_pytest/deprecated.py`).",
    "rubric": [
      "States that `fixture_request()` and `define_combined_fixture()` do not exist anywhere in the source code \u2014 they are unimplemented proposals only",
      "Identifies `request.getfixturevalue(argname)` as the actual dynamic fixture resolution method, defined in `FixtureRequest.getfixturevalue()` in `src/_pytest/fixtures.py`",
      "Explains that `_check_fixturedef_without_param()` prevents `getfixturevalue` from being used on parametrized fixtures that the test wasn't collected for",
      "Notes that `getfuncargvalue` (the old name) has been completely removed from the codebase",
      "Mentions the `FixtureRequest` class hierarchy: abstract `FixtureRequest(abc.ABC)`, with `TopRequest` and `SubRequest` subclasses",
      "Notes that `SubRequest` forms a chain via `_parent_request`, traversed by `_iter_chain`",
      "Identifies that calling `getfixturevalue` during teardown raises `FIXTURE_GETFIXTUREVALUE_DURING_TEARDOWN`, a `PytestRemovedIn10Warning`"
    ],
    "key_files": [
      "doc/en/proposals/parametrize_with_fixtures.rst",
      "src/pytest/__init__.py",
      "src/_pytest/fixtures.py",
      "src/_pytest/python.py",
      "src/_pytest/deprecated.py"
    ],
    "source_doc": "[doc_file: doc/en/proposals/parametrize_with_fixtures.rst] Alternative approach\nA new helper function named ``fixture_request`` would tell pytest to yield\nall parameters marked as a fixture.\n\n.. note::\n\n    The :pypi:`pytest-lazy-fixture` plugin implements a very\n    similar solution to the proposal below, make sure to check it out.\n\n.. code-block:: python\n\n    @pytest.fixture(\n        params=[\n            pytest.fixture_request(\"default_context\"),\n            pytest.fixture_request(\"extra_context\"),\n        ]\n    )\n    def context(request):\n        \"\"\"Returns all values for ``default_context``, one-by-one before it\n        does the same for ``extra_context``.\n\n        request.param:\n            - {}\n            - {'author': 'alice'}\n            - {'project_slug': 'helloworld'}\n            - {'author': 'bob', 'project_slug': 'foobar'}\n        \"\"\"\n        return request.param\n\nThe same helper can be used in combination with ``pytest.mark.parametrize``.\n\n.. code-block:: python\n\n\n    @pytest.mark.parametrize(\n        \"context, expected_response_code\",\n        [\n            (pytest.fixture_request(\"default_context\"), 0),\n            (pytest.fixture_request(\"extra_context\"), 0),\n        ],\n    )\n    def test_generate_project(cookies, context, exit_code):\n        \"\"\"Call the cookiecutter API to generate a new project from a\n        template.\n        \"\"\"\n        result = cookies.bake(extra_context=context)\n\n        assert result.exit_code == exit_code",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; one interpretive claim about design rationale was removed, and several were rewritten to remove unverifiable line numbers and implementation details from truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_12",
    "repo": "pytest",
    "question": "How does pytest's teardown mechanism decide which fixtures and collectors to tear down between consecutive test items?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The teardown mechanism ensures that test teardown happens immediately at the appropriate scope level rather than being delayed until the end of the session. This is implemented through the `SetupState` class in `src/_pytest/runner.py` and the `nextitem` parameter passed through the test protocol.\n\n**How it works:**\n\n1. **The `pytest_runtestloop` function** in `src/_pytest/main.py` iterates through all collected items and computes the `nextitem` for each: `nextitem = session.items[i + 1] if i + 1 < len(session.items) else None`. For the last item, `nextitem` is `None`.\n\n2. **The `pytest_runtest_protocol` hook** receives `nextitem` and passes it to `runtestprotocol()`, which in turn passes it to `call_and_report(item, \"teardown\", log, nextitem=nextitem)`.\n\n3. **The `pytest_runtest_teardown` hook** calls `item.session._setupstate.teardown_exact(nextitem)` - the core algorithm.\n\n4. **The `SetupState.teardown_exact` method** implements the scoped teardown algorithm:\n   - It computes `needed_collectors = (nextitem and nextitem.listchain()) or []` \u2014 this is the chain of nodes the next item descends from.\n   - It pops nodes from the stack (LIFO order via `dict.popitem()`) until the remaining stack matches the common prefix with the next item's collector chain.\n   - The comparison `list(self.stack.keys()) == needed_collectors[:len(self.stack)]` determines shared ancestors that should NOT be torn down.\n   - When `nextitem is None` (last item), `needed_collectors` is `[]`, so the entire stack is torn down, enforced by `assert not self.stack`.\n\n5. **Finalizer execution**: Each node on the stack stores a list of finalizers. The node's own `teardown()` method is the first finalizer added (`self.stack[col] = ([col.teardown], None)`). Additional finalizers (from fixtures) are appended via `addfinalizer()`. Finalizers are popped and called in reverse order (LIFO).\n\n6. **Exception handling**: All finalizer exceptions are caught (as `TEST_OUTCOME = (OutcomeException, Exception)`) and collected. If multiple exceptions occur for one node, they're wrapped in a `BaseExceptionGroup`. If multiple nodes have errors, another `BaseExceptionGroup` wraps them all.\n\n7. **Session stop/fail override**: In `runtestprotocol`, if `item.session.shouldfail or item.session.shouldstop`, `nextitem` is forced to `None`, ensuring complete teardown even when stopping early.\n\n8. **Session finish safety net**: `pytest_sessionfinish` also calls `teardown_exact(None)` to guarantee everything is torn down at session end.",
    "rubric": [
      "Explains that the `nextitem` parameter is computed in `pytest_runtestloop` and passed through the protocol to `teardown_exact`",
      "Describes that `teardown_exact` computes `needed_collectors` from `nextitem.listchain()` (or empty list if None)",
      "Explains the stack comparison logic: nodes are popped until the remaining stack matches the common prefix with the next item's collector chain",
      "Mentions that when `nextitem` is None (last item), the entire stack is torn down",
      "Describes that finalizers are stored per-node and executed in LIFO (reverse) order via `pop()`",
      "Mentions that `node.teardown()` is the first finalizer added to the stack entry, with additional finalizers appended via `addfinalizer`",
      "Notes the exception handling: exceptions are caught as TEST_OUTCOME and potentially grouped into BaseExceptionGroup",
      "Mentions the early-stop override: if `shouldfail` or `shouldstop`, `nextitem` is forced to None for complete teardown"
    ],
    "key_files": [
      "src/_pytest/runner.py",
      "src/_pytest/main.py",
      "src/_pytest/nodes.py",
      "src/_pytest/fixtures.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.2.1.rst] pytest-2.2.1: bug fixes, perfect teardowns\npytest-2.2.1 is a minor backward-compatible release of the py.test\ntesting tool.   It contains bug fixes and little improvements, including\ndocumentation fixes.  If you are using the distributed testing\npluginmake sure to upgrade it to pytest-xdist-1.8.\n\nFor general information see here:\n\n     http://pytest.org/\n\nTo install or upgrade pytest:\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nSpecial thanks for helping on this release to Ronny Pfannschmidt, Jurko\nGospodnetic and Ralf Schmitt.\n\nbest,\nholger krekel",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only mentions 'perfect teardowns' in the title and describes pytest 2.2.1 as containing 'bug fixes and little improvements.' It does not describe any mechanism, issue number (#93), or implementation details about perfect teardowns.",
      "All specific implementation details (SetupState class, nextitem parameter, line numbers, exception handling with BaseExceptionGroup, issue #11706, etc.) are not supported by the provided documentation - they appear to be sourced from actual code rather than from this doc chunk.",
      "The documentation does not mention issue #93 or any specific feature descriptions beyond 'bug fixes and little improvements, including documentation fixes.'",
      "The answer references specific source files (src/_pytest/runner.py, src/_pytest/main.py) and line numbers that are not present in the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from version/issue references and the 'perfect teardowns' feature name; core algorithmic claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_13",
    "repo": "pytest",
    "question": "What is the architecture of the Frame class and how does it interact with Code, TracebackEntry, and the rest of the traceback system?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `Frame` class in `src/_pytest/_code/code.py` is a wrapper around Python's `FrameType` object. Here's its architecture:\n\n**Class Structure:**\n- Uses `__slots__ = (\"raw\",)`, storing only the underlying `FrameType` object.\n- Has a `code` property that creates a new `Code` instance wrapping `self.raw.f_code` on every access (no caching).\n- The `lineno` property returns a 0-based line number by subtracting 1 from the raw frame's `f_lineno`.\n\n**Key Relationships:**\n1. `Frame` is created by `TracebackEntry.frame` which wraps `self._rawentry.tb_frame`.\n2. `Frame.getargs()` delegates to `Code.getargs()` to get argument names, then looks up values from `self.f_locals`. It silently skips `KeyError`.\n3. `Frame.repr()` delegates directly to `saferepr()` from `_pytest._io.saferepr`, which uses a `DEFAULT_REPR_MAX_SIZE` of 240 characters and a `SafeRepr` class inheriting from `reprlib.Repr`.\n4. `Frame.eval()` copies `f_locals`, updates with provided `**vars`, and calls Python's built-in `eval()` with both `f_globals` and the merged locals.\n5. `Frame.statement` returns an empty `Source(\"\")` if `self.code.fullsource` is None, otherwise delegates to `Source.getstatement()` which uses `getstatementrange_ast()` for AST-based statement boundary detection.\n\n**Usage in the System:**\n- `Traceback.recursionindex()` uses `entry.frame.f_locals`, `entry.frame.code.path`, and `id(entry.frame.code.raw)` to detect recursion by comparing frame locals at the same code location.\n- `FormattedExcinfo.repr_args()` calls `entry.frame.getargs(var=True)` to display function arguments in traceback output.\n- `filter_traceback()` accesses `entry.frame.code.raw.co_filename` to determine if a traceback entry should be filtered.\n- `TracebackEntry.ishidden()` checks both `self.frame.f_locals` and `self.frame.f_globals` for a `__tracebackhide__` variable.\n\n**Code class (companion):**\n- `Code.getargs(var=False)` uses `inspect.CO_VARARGS` and `inspect.CO_VARKEYWORDS` flag constants, adding them to `co_argcount` when `var=True` to include *args and **kwargs names from `co_varnames`.\n- `Code.fullsource` uses `findsource()` from `source.py` which wraps `inspect.findsource()`, returning `None` on any exception.\n- `Code.firstlineno` returns `self.raw.co_firstlineno - 1` (0-based).",
    "rubric": [
      "Mentions that Frame uses __slots__ = ('raw',) storing only the underlying FrameType object",
      "Explains that the code property creates a new Code instance wrapping self.raw.f_code on every access (no caching)",
      "Notes that lineno returns a 0-based line number by subtracting 1 from f_lineno",
      "Describes that Frame.getargs() delegates to Code.getargs() for argument names, then looks up values from f_locals, silently skipping KeyError",
      "Explains that Frame.eval() copies f_locals, updates with provided **vars, and calls built-in eval() with f_globals and merged locals",
      "Describes that Frame.statement returns Source('') if fullsource is None, otherwise delegates to Source.getstatement()",
      "Mentions that Frame.repr() delegates to saferepr()",
      "Explains that TracebackEntry.frame wraps self._rawentry.tb_frame in a Frame",
      "Describes how Traceback.recursionindex() uses frame.f_locals, frame.code.path, and id(frame.code.raw) to detect recursion",
      "Notes that FormattedExcinfo.repr_args() calls entry.frame.getargs(var=True) for traceback argument display",
      "Mentions that filter_traceback() accesses entry.frame.code.raw.co_filename to check for generated code",
      "Explains that TracebackEntry.ishidden() checks both frame.f_locals and frame.f_globals for __tracebackhide__",
      "Describes Code.getargs() using CO_VARARGS and CO_VARKEYWORDS flags to optionally include *args/**kwargs names"
    ],
    "key_files": [
      "src/_pytest/_code/code.py",
      "src/_pytest/_code/source.py",
      "src/_pytest/_io/saferepr.py"
    ],
    "source_doc": "[docstring: src/_pytest/_code/code.py] src._pytest._code.code.Frame\nsrc._pytest._code.code.Frame:\n  Wrapper around a Python frame holding f_locals and f_globals\n    in which expressions can be evaluated.\n\nsrc._pytest._code.code.Frame.statement:\n  Statement this frame is at.\n\nsrc._pytest._code.code.Frame.eval:\n  Evaluate 'code' in the frame.\n\n        'vars' are optional additional local variables.\n\n        Returns the result of the evaluation.\n\nsrc._pytest._code.code.Frame.repr:\n  Return a 'safe' (non-recursive, one-line) string repr for 'object'.\n\nsrc._pytest._code.code.Frame.getargs:\n  Return a list of tuples (name, value) for all arguments.\n\n        If 'var' is set True, also include the variable and keyword arguments\n        when present.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (e.g., __slots__ = ('raw',), lineno being 0-based, SafeRepr class with DEFAULT_REPR_MAX_SIZE of 240, getstatementrange_ast(), inspect.CO_VARARGS flags, etc.) that are not present in the provided documentation. These could be accurate based on the actual source code, but they cannot be verified from the given documentation alone.",
      "The documentation says Frame.getargs 'If var is set True, also include the variable and keyword arguments when present.' The answer says Code.getargs(var=False) uses inspect.CO_VARARGS and inspect.CO_VARKEYWORDS - this is plausible but unverifiable from the doc.",
      "The answer's claim about Frame.eval() copying f_locals and updating with **vars is consistent with the doc stating 'vars are optional additional local variables' and 'Returns the result of the evaluation'.",
      "The answer's claim about Frame.repr() delegating to saferepr() is consistent with the doc saying it returns 'a safe (non-recursive, one-line) string repr for object'.",
      "No claims directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale comments (memory efficiency, Psyco compatibility) and interpretive descriptions, but most claims are directly verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_14",
    "repo": "pytest",
    "question": "Why do several pytest internals use special escaping or guarding tricks \u2014 like doubled percents, XML character filtering, initstate tracking, and bracket-aware splitting?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The design rationale behind these fixes reveals several defensive programming patterns to handle edge cases in assertion rewriting, XML output generation, and session lifecycle management:\n\n**Issue: % operator in assertion rewriting** \u2014 The `BINOP_MAP` in `src/_pytest/assertion/rewrite.py` maps `ast.Mod` to `\"%%\"` (doubled percent) rather than `\"%\"`. This is because the assertion explanation strings are later formatted using Python's `%` string formatting via `ast.BinOp(expl_expr, ast.Mod(), format_dict)`. Without the escaped `%%`, a literal modulo operator `%` in the user's assertion would be misinterpreted as a format specifier. Similarly, `_format_assertmsg()` replaces `\"%\"` with `\"%%\"` in its `replaces` list to prevent assertion messages containing `%` from being corrupted.\n\n**Issue: invalid XML characters** \u2014 The `bin_xml_escape()` function in `src/_pytest/junitxml.py` uses a negated character class regex `[^\\u0009\\u000a\\u000d\\u0020-\\u007e\\u0080-\\ud7ff\\ue000-\\ufffd\\u10000-\\u10ffff]` that defines all valid XML characters. Anything not matching is escaped to a visual representation like `#x1B`. The regex additionally disallows 0x7F (DEL) by excluding it from the valid range (valid goes up to 0x7E, then continues at 0x80).\n\n**Issue: configure/sessionfinish lifecycle** \u2014 The `wrap_session()` function in `src/_pytest/main.py` uses an `initstate` integer variable (0, 1, or 2) to track lifecycle progression: 0=initial, 1=after `_do_configure()`, 2=after `pytest_sessionstart()`. In the `finally` block, `pytest_sessionfinish` is only called if `initstate >= 2` (i.e., if sessionstart was successfully called). `config._ensure_unconfigure()` is always called in the `finally` block regardless of initstate.\n\n**Issue: mangle test IDs** \u2014 The `mangle_test_address()` function first partitions the address at `[` to isolate parametrize parameters, then splits by `::` only on the path part. This prevents `::` within parametrize parameters (like `[double::colon]`) from being incorrectly split into separate classname components. The file path portion uses `nodes.SEP` (which is `\"/\"`) replaced with `.` to create dotted classnames, and `.py` suffix is stripped via regex.\n\n**Issue: bound classmethods** \u2014 The `getimfunc()` function in `src/_pytest/compat.py` extracts the real function from bound methods via `func.__func__`, falling back to returning `func` directly if `AttributeError` is raised.",
    "rubric": [
      "Explains that BINOP_MAP maps ast.Mod to '%%' (doubled percent) because assertion explanation strings are later %-formatted, and a single '%' would be misinterpreted as a format specifier",
      "Mentions that _format_assertmsg also replaces '%' with '%%' in its replaces list for the same reason \u2014 to prevent assertion messages with '%' from being corrupted by string formatting",
      "Explains that bin_xml_escape uses a negated regex of valid XML character ranges to visually escape invalid characters (e.g., to '#x1B'), and that 0x7F (DEL) is additionally excluded from the valid range",
      "Explains that wrap_session's initstate variable (0, 1, 2) tracks lifecycle stages so that pytest_sessionfinish is only called (initstate >= 2) if pytest_sessionstart was successfully called, while _ensure_unconfigure always runs",
      "Explains that mangle_test_address partitions on '[' before splitting on '::' to prevent '::' inside parametrize parameters (like '[double::colon]') from being incorrectly split into separate components",
      "Mentions that getimfunc extracts the underlying function from bound methods via __func__, falling back to returning the function directly if AttributeError is raised"
    ],
    "key_files": [
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/junitxml.py",
      "src/_pytest/main.py",
      "src/_pytest/compat.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.2.4.rst] Changes between 2.2.3 and 2.2.4\n- fix error message for rewritten assertions involving the % operator\n- fix issue 126: correctly match all invalid xml characters for junitxml\n  binary escape\n- fix issue with unittest: now @unittest.expectedFailure markers should\n  be processed correctly (you can also use @pytest.mark markers)\n- document integration with the extended distribute/setuptools test commands\n- fix issue 140: properly get the real functions\n  of bound classmethods for setup/teardown_class\n- fix issue #141: switch from the deceased paste.pocoo.org to bpaste.net\n- fix issue #143: call unconfigure/sessionfinish always when\n  configure/sessionstart where called\n- fix issue #144: better mangle test ids to junitxml classnames\n- upgrade distribute_setup.py to 0.6.27",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive code-level implementation details (specific function names, line numbers, regex patterns, variable names) that are not present in the documentation and cannot be verified from it alone. These appear to be plausible extrapolations from actual source code, but the documentation only provides high-level descriptions of the fixes.",
      "The documentation says 'fix issue 140: properly get the real functions of bound classmethods for setup/teardown_class' - the answer's description of getimfunc() and _register_setup_class_fixture() is plausible but unverifiable from the doc alone.",
      "The documentation says 'fix issue #144: better mangle test ids to junitxml classnames' - the answer's detailed description of mangle_test_address() behavior is plausible but goes far beyond what the doc states.",
      "The answer does not contradict any claims made in the documentation - all high-level descriptions align with the changelog entries."
    ],
    "strip_verify_leakage": 0.16,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed XML spec reference, test case comment reference, and unshown _register_setup_class_fixture usage claim.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_15",
    "repo": "pytest",
    "question": "Where is `_safe_key` used in the pretty-printing code, and how does it flow from the public API to actual sorting?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `_safe_key` class in `src/_pytest/_io/pprint.py` is used exclusively in the `_pprint_set` method as the `key=` argument to `sorted()` when pretty-printing set and frozenset objects. The data flows as follows:\n\n1. **Entry point**: `PrettyPrinter.pformat()` is called from `src/_pytest/assertion/util.py` in `_compare_eq_iterable()` to generate diff output for assertion failures in pytest.\n\n2. **Dispatch mechanism**: `_format()` checks `self._dispatch` dict keyed by `type(object).__repr__` to route to type-specific handlers. Both `set.__repr__` and `frozenset.__repr__` map to `_pprint_set`.\n\n3. **_safe_key usage**: In `_pprint_set`, `sorted(object, key=_safe_key)` is called. Since `_safe_key` is a class, passing it as `key=` means each element is wrapped in a `_safe_key` instance, and Python's sort uses `__lt__` for comparisons.\n\n4. **Fallback logic**: `_safe_key.__lt__` first tries direct comparison (`self.obj < other.obj`). If that raises `TypeError` (unorderable types), it falls back to comparing `(str(type(self.obj)), id(self.obj))` tuples - comparing type names first, then object ids.\n\n5. **_safe_tuple is defined but not called in this file**: The function `_safe_tuple(t)` (which applies `_safe_key` to both elements of a 2-tuple) is defined but does not appear to be used within the file.\n\n6. **Module customization**: The `PrettyPrinter.__init__` uses a default indent of 4 and does not include `stream`, `compact`, or `sort_dicts` parameters.\n\n7. The `_safe_key` class uses `__slots__ = [\"obj\"]`.",
    "rubric": [
      "_safe_key is used in _pprint_set as the key= argument to sorted() when pretty-printing sets and frozensets",
      "PrettyPrinter.pformat() is called from assertion/util.py in _compare_eq_iterable() for generating diff output",
      "_format() dispatches to type-specific handlers via self._dispatch dict keyed by type(object).__repr__",
      "Both set.__repr__ and frozenset.__repr__ map to _pprint_set in the dispatch dict",
      "_safe_key.__lt__ first tries direct comparison (self.obj < other.obj), then falls back to comparing (str(type(self.obj)), id(self.obj)) tuples on TypeError",
      "_safe_key uses __slots__ = ['obj']",
      "_safe_tuple is defined but not used/called within the file"
    ],
    "key_files": [
      "src/_pytest/_io/pprint.py",
      "src/_pytest/assertion/util.py",
      "testing/io/test_pprint.py"
    ],
    "source_doc": "[docstring: src/_pytest/_io/pprint.py] src._pytest._io.pprint._safe_key\nsrc._pytest._io.pprint._safe_key:\n  Helper function for key functions when sorting unorderable objects.\n\n    The wrapped-object will fallback to a Py2.x style comparison for\n    unorderable types (sorting first comparing the type name and then by\n    the obj ids).  Does not work recursively, so dict.items() must have\n    _safe_key applied to both the key and the value.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states _safe_key is a class, but the documentation describes it as a 'Helper function for key functions'. While it could be implemented as a class (callable), this is an extrapolation not directly supported by the doc.",
      "The answer claims _safe_tuple is 'defined but never called' and is 'dead code' - this is not verifiable from the documentation provided.",
      "The answer provides many specific implementation details (line numbers, commit hashes, __slots__, indent defaults) that cannot be verified from the documentation chunk provided - these are plausible but unverifiable.",
      "The documentation says it 'Does not work recursively, so dict.items() must have _safe_key applied to both the key and the value' - the answer's claim that _safe_tuple is dead code potentially contradicts this implication that dict items handling exists and is needed."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage: CPython commit origin, 'dead code carried over' rationale, and memory efficiency rationale are not recoverable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_16",
    "repo": "pytest",
    "question": "How does pytest's custom PrettyPrinter differ from the standard library's pprint in its formatting behavior, and where is it used?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `PrettyPrinter` class in `src/_pytest/_io/pprint.py` is a customized version of a pretty-printer that has key design differences from the standard library's `pprint` module:\n\n**Always-expand design**: Unlike CPython's standard `pprint.PrettyPrinter` which first checks if an object's repr fits within the configured width before expanding to multi-line, pytest's version ALWAYS expands recognized container types (list, dict, tuple, set, etc.) to one-item-per-line format. The `_format_items` method unconditionally writes each item on its own line with no width-fitting check.\n\n**Simplified constructor**: The `__init__` takes only 3 parameters (`indent=4`, `width=80`, `depth=None`). The default indent is 4.\n\n**Dispatch mechanism**: The `_dispatch` class-level dictionary maps `type.__repr__` functions to specialized formatters. It handles: dict, OrderedDict, list, tuple, set, frozenset, str, bytes, bytearray, MappingProxyType, SimpleNamespace, defaultdict, Counter, ChainMap, deque, UserDict, UserList, and UserString.\n\n**Dataclass support**: The `_format` method has special handling for dataclasses that checks `object.__dataclass_params__.repr` is True AND that the repr method has `__wrapped__` with `\"__create_fn__\"` in its `__qualname__` \u2014 ensuring only auto-generated dataclass reprs get pretty-printed.\n\n**Usage in pytest**: It's used in `src/_pytest/assertion/util.py`'s `_compare_eq_iterable` function, where `PrettyPrinter().pformat(left/right).splitlines()` is called to generate line-based representations for `difflib.ndiff` comparison. This is only triggered when `verbose > 0` OR when running on CI.\n\n**Recursion handling**: Uses a `context` set of `id()` values to detect recursive structures. For namespace items (dataclasses/SimpleNamespace), recursion is represented as `\"...\"`, while other recursion uses a different format (via the `_recursion` function).\n\n**The `_safe_repr` method** is used as a fallback for inline/compact representation of nested objects (e.g., dict keys) and is called via the `_repr` method with depth and level parameters for depth limiting.",
    "rubric": [
      "Explains that pytest's PrettyPrinter always expands containers to one-item-per-line format without checking if the repr fits within the width (unlike CPython's pprint which tries compact first)",
      "Mentions the simplified constructor with indent=4 (default), width=80, and depth=None parameters",
      "Describes the _dispatch dictionary mechanism that maps type.__repr__ to specialized formatter methods",
      "Lists at least 5 of the supported types (dict, list, tuple, set, frozenset, str, bytes, OrderedDict, defaultdict, Counter, ChainMap, deque, etc.)",
      "Explains dataclass support: checks __dataclass_params__.repr is True AND __repr__ has __wrapped__ with '__create_fn__' in __qualname__",
      "Identifies its usage in assertion comparison (_compare_eq_iterable) where pformat().splitlines() feeds into difflib.ndiff, triggered when verbose > 0 or running on CI",
      "Mentions recursion handling using a context set of id() values, with '...' for namespace items and _recursion() format for other types"
    ],
    "key_files": [
      "src/_pytest/_io/pprint.py",
      "src/_pytest/assertion/util.py",
      "testing/io/test_pprint.py"
    ],
    "source_doc": "[docstring: src/_pytest/_io/pprint.py] src._pytest._io.pprint.PrettyPrinter\nsrc._pytest._io.pprint.PrettyPrinter.__init__:\n  Handle pretty printing operations onto a stream using a set of\n        configured parameters.\n\n        indent\n            Number of spaces to indent for each level of nesting.\n\n        width\n            Attempted maximum number of columns in the output.\n\n        depth\n            The maximum depth to print out nested structures.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a brief docstring about the PrettyPrinter class and its __init__ parameters (indent, width, depth). The generated answer makes extensive claims about implementation details (always-expand design, dispatch mechanism, dataclass support, specific line numbers, usage in pytest, recursion handling, _safe_repr method) that cannot be verified from the provided documentation alone.",
      "The documentation does not mention default values for indent (4), width (80), or depth (None), though these are plausible.",
      "The claim about being a fork of CPython's pprint from a specific commit cannot be verified from the documentation.",
      "The claim about _format_items unconditionally expanding containers cannot be verified from the documentation.",
      "The claim about usage in src/_pytest/assertion/util.py cannot be verified from the documentation.",
      "No claims directly contradict the documentation - the documentation simply doesn't provide enough information to verify most of the answer's claims."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Some claims about CPython origins (commit hash, version), design rationale ('makes diffs stable'), and line numbers are documentation-dependent; core behavioral claims are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_17",
    "repo": "pytest",
    "question": "What is the architecture of the SafeRepr class and how does it integrate with the rest of pytest's safe representation utilities?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `SafeRepr` class in `src/_pytest/_io/saferepr.py` is a subclass of `reprlib.Repr` from Python's standard library that provides safe representation of objects with size limits and exception handling.\n\n**Key architectural details:**\n\n1. **Class hierarchy**: `SafeRepr` extends `reprlib.Repr`. It overrides three methods: `repr()`, `repr_instance()`, and `repr_dict()`.\n\n2. **`maxstring` fallback**: When `maxsize` is `None` (no truncation desired), `self.maxstring` is set to `1_000_000_000` (one billion). This effectively disables truncation at the superclass level.\n\n3. **`_ellipsize` function**: The truncation logic uses a helper function `_ellipsize(s, maxsize)` that inserts `\"...\"` in the middle of the string. It calculates `i = max(0, (maxsize - 3) // 2)` for the first half and `j = max(0, maxsize - 3 - i)` for the second half, then returns `s[:i] + \"...\" + s[len(s) - j:]`.\n\n4. **Exception safety**: Both `repr()` and `repr_instance()` catch all `BaseException` except `KeyboardInterrupt` and `SystemExit`, formatting caught exceptions via `_format_repr_exception()` which produces strings like `<[ExcInfo raised in repr()] ClassName object at 0xADDRESS>`.\n\n5. **`use_ascii` mode**: When `use_ascii=True`, the `repr()` method bypasses `super().repr(x)` entirely and calls Python's built-in `ascii(x)` directly.\n\n6. **`repr_dict` preserves insertion order**: `SafeRepr.repr_dict` iterates directly over the dict using `islice(x, self.maxdict)`. It uses the inherited `self.repr1` method (from `reprlib.Repr`) to recursively represent keys and values with decremented level.\n\n7. **`DEFAULT_REPR_MAX_SIZE = 240`**: This constant is used as the default for assertion error messages. At verbosity >= 1, the size is multiplied by 10 (2400), and at verbosity >= 2, truncation is disabled (`None`).\n\n8. **Two separate unlimited repr paths**: `saferepr_unlimited()` is a standalone function that does NOT use `SafeRepr` at all \u2014 it directly calls `ascii()` or `repr()` and catches only `Exception` (not `BaseException`). Its `use_ascii` defaults to `True`, unlike `saferepr()` where it defaults to `False`.\n\n9. **Usage in assertion rewriting**: The `_saferepr()` function in `assertion/rewrite.py` calls `saferepr()` and replaces newlines with `\\\\n`. For bound methods (`types.MethodType`), it returns just `obj.__name__` instead of the full repr.\n\n10. **`safeformat()` function**: Uses `pprint.pformat(obj)` for pretty-printing, falling back to `_format_repr_exception` on failure \u2014 this is separate from the `SafeRepr` class approach.",
    "rubric": [
      "Identifies SafeRepr as a subclass of reprlib.Repr that overrides repr(), repr_instance(), and repr_dict()",
      "Explains that maxstring is set to 1_000_000_000 when maxsize is None to effectively disable truncation",
      "Describes the _ellipsize helper function's truncation logic (inserting '...' in the middle with i = max(0, (maxsize-3)//2) and j = max(0, maxsize-3-i))",
      "Notes that exception handling catches BaseException except KeyboardInterrupt and SystemExit, formatting via _format_repr_exception",
      "Explains use_ascii mode bypasses super().repr() and calls ascii() directly",
      "Mentions repr_dict uses islice to preserve insertion order and calls self.repr1 for recursive representation",
      "States DEFAULT_REPR_MAX_SIZE = 240 with verbosity-based scaling (\u00d710 at v>=1, None/unlimited at v>=2)",
      "Distinguishes saferepr_unlimited() as a standalone function not using SafeRepr, catching only Exception (not BaseException), with use_ascii defaulting to True",
      "Describes how _saferepr in assertion/rewrite.py replaces newlines and handles bound methods specially",
      "Mentions safeformat() uses pprint.pformat with fallback to _format_repr_exception as a separate approach"
    ],
    "key_files": [
      "src/_pytest/_io/saferepr.py",
      "src/_pytest/assertion/rewrite.py",
      "testing/io/test_saferepr.py"
    ],
    "source_doc": "[docstring: src/_pytest/_io/saferepr.py] src._pytest._io.saferepr.SafeRepr\nsrc._pytest._io.saferepr.SafeRepr:\n  repr.Repr that limits the resulting size of repr() and includes\n    information on exceptions raised during the call.\n\nsrc._pytest._io.saferepr.SafeRepr.__init__:\n  :param maxsize:\n            If not None, will truncate the resulting repr to that specific size, using ellipsis\n            somewhere in the middle to hide the extra text.\n            If None, will not impose any size limits on the returning repr.\n\nsrc._pytest._io.saferepr.SafeRepr.repr_dict:\n  Represent a dict while preserving its insertion order.\n\n        Differs from ``reprlib.Repr.repr_dict`` by iterating directly over ``x``\n        rather than using the stdlib's sorting helper.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer (points 2, 3, 4, 5, 6's islice detail, 7, 8, 9, 10) go far beyond what the documentation provides. These are plausible implementation details but cannot be verified from the given documentation.",
      "Point 1 claims SafeRepr overrides three methods: repr(), repr_instance(), and repr_dict(). The documentation only mentions repr_dict explicitly. The other overrides are plausible but unverifiable from the doc.",
      "Point 6 states repr_dict uses 'islice(x, self.maxdict)' - the documentation only says it 'iterates directly over x rather than using the stdlib's sorting helper'. The islice detail is plausible but unverifiable.",
      "The documentation does not contradict any claims, but the vast majority of the answer's content is extrapolated beyond what the documentation explicitly states."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rationale about reprlib.Repr's internal requirements was documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_18",
    "repo": "pytest",
    "question": "Why does `saferepr_unlimited` exist as a separate function instead of just calling `saferepr` with `maxsize=None`?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "`saferepr_unlimited` exists as a separate function (rather than simply calling `saferepr(obj, maxsize=None)`) because `saferepr` delegates to `SafeRepr(reprlib.Repr)`, which inherits `reprlib.Repr`'s internal per-type truncation limits for collections even when the overall `maxsize` is set to `None`. When `maxsize=None`, `SafeRepr` sets `self.maxstring = 1_000_000_000` as a workaround and skips the `_ellipsize` step, but `reprlib.Repr` still truncates collections at its built-in limits. By contrast, `saferepr_unlimited` bypasses `reprlib.Repr` entirely and calls Python's built-in `repr()` or `ascii()` directly, producing truly unlimited output with no per-collection-type truncation. This is confirmed by the test `test_saferepr_unlimited` which creates a 1000-element dict and asserts `'...' not in r`.\n\nThe function exists specifically to support high-verbosity assertion output: in `_saferepr()` (rewrite.py), when `_get_maxsize_for_saferepr` returns `None` (verbosity >= 2), `saferepr_unlimited` is called instead of `saferepr`. Similarly, `assertion/util.py` uses `saferepr_unlimited` when `verbose > 1` to show full left/right operands in assertion comparison explanations.\n\nAdditional design differences: (1) `saferepr_unlimited` defaults to `use_ascii=True`, while `SafeRepr` defaults to `use_ascii=False`; (2) `saferepr_unlimited` catches `Exception` in its error handler, while `SafeRepr.repr` catches the broader `BaseException` (though both re-raise `KeyboardInterrupt`/`SystemExit` \u2014 `SafeRepr.repr` does so explicitly, while `saferepr_unlimited` relies on `Exception` not catching them).",
    "rubric": [
      "Explains that saferepr delegates to SafeRepr which inherits from reprlib.Repr, and reprlib.Repr has built-in per-type collection truncation limits (e.g., maxdict, maxlist) that apply even when the overall maxsize is None",
      "Notes that SafeRepr uses a workaround of setting maxstring to 1_000_000_000 and skipping _ellipsize when maxsize=None, but this doesn't remove reprlib's per-collection-type truncation",
      "Explains that saferepr_unlimited bypasses reprlib.Repr entirely by calling Python's built-in repr() or ascii() directly, producing truly unlimited output",
      "Mentions the practical use case: saferepr_unlimited is used for high-verbosity assertion output (verbose >= 2 in rewrite.py and util.py)",
      "Notes at least one additional design difference: different default for use_ascii (True vs False), or different exception handling scope (Exception vs BaseException)"
    ],
    "key_files": [
      "src/_pytest/_io/saferepr.py",
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/assertion/util.py",
      "testing/io/test_saferepr.py"
    ],
    "source_doc": "[docstring: src/_pytest/_io/saferepr.py] src._pytest._io.saferepr.saferepr_unlimited\nReturn an unlimited-size safe repr-string for the given object.\n\n    As with saferepr, failing __repr__ functions of user instances\n    will be represented with a short exception info.\n\n    This function is a wrapper around simple repr.\n\n    Note: a cleaner solution would be to alter ``saferepr``this way\n    when maxsize=None, but that might affect some other code.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the specific reprlib default values (maxdict=4, maxlist=6) required minor rewording.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_19",
    "repo": "pytest",
    "question": "Where is `TerminalWriter` defined, and how do its `width_of_current_line`, `_write_source`, and `_highlight` methods flow into other parts of the codebase?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `TerminalWriter` class and its key methods (`width_of_current_line`, `_write_source`, `_highlight`) are located in `src/_pytest/_io/terminalwriter.py`. Here is where each method is used and how data flows through the system:\n\n**`width_of_current_line`** is a `@property` that delegates to `wcswidth(self._current_line)` from `src/_pytest/_io/wcwidth.py`. The `_current_line` string is tracked by the `write()` method: on each write, if the message contains a newline, `_current_line` resets to the text after the last newline; otherwise it appends to `_current_line`. The `wcswidth` function normalizes the string with `unicodedata.normalize('NFC', s)` and sums per-character widths, where east Asian full/wide characters count as 2 and combining characters (categories 'Me', 'Mn') count as 0. This property is consumed in `src/_pytest/terminal.py` by `TerminalReporter` in `wrap_write()` for wrapping messages with a margin, and in progress information methods to determine terminal width usage.\n\n**`_write_source`** is defined in `src/_pytest/_io/terminalwriter.py`. The method joins lines into a single source string, passes it through `_highlight()`, then splits the result back and writes each line with its corresponding indent. It validates that `indents` and `lines` have the same length if indents is non-empty, raising a `ValueError` otherwise.\n\n**`_highlight`** supports two lexers: `\"python\"` (using pygments `PythonLexer`) and `\"diff\"` (using pygments `DiffLexer`), selected via `_get_pygments_lexer()`. It returns the source unchanged if any of three conditions is true: empty source, `hasmarkup` is False, or `code_highlight` is False. The pygments formatter uses the `PYTEST_THEME` and `PYTEST_THEME_MODE` environment variables (defaulting to `\"dark\"` mode), and raises `UsageError` (imported from `_pytest.config.exceptions`) on invalid values. The highlighted output is prefixed with `\"\\x1b[0m\"` to reset previous colors, and trailing newlines added by pygments are stripped. `_highlight` is also directly used in `src/_pytest/assertion/util.py`: `config.get_terminal_writer()._highlight` is passed as a `_HighlightFunc` callback to assertion comparison functions like `_diff_text`, `_compare_eq_any`, and set comparison functions. In `_diff_text`, it is called with `lexer=\"diff\"` to highlight ndiff output.\n\nThe `TerminalWriter` class is decorated with `@final` (from `typing`). The `create_terminal_writer` function in `src/_pytest/config/__init__.py` creates `TerminalWriter` instances (as referenced by `TerminalReporter` which calls `_pytest.config.create_terminal_writer(config, file)`).",
    "rubric": [
      "States that TerminalWriter is defined in src/_pytest/_io/terminalwriter.py and is decorated with @final",
      "Explains that width_of_current_line is a property that delegates to wcswidth(), which normalizes with NFC and handles east Asian wide/full chars as width 2 and combining chars (Me, Mn) as width 0",
      "Explains that _current_line is tracked by the write() method: resets after newlines, appends otherwise",
      "Identifies that width_of_current_line is consumed by TerminalReporter in terminal.py for wrap_write() and progress display width calculations",
      "Explains that _write_source joins lines, calls _highlight, splits the result, and writes each line with its indent; raises ValueError if indents/lines length mismatch",
      "Explains that _highlight supports 'python' (PythonLexer) and 'diff' (DiffLexer) lexers and returns source unchanged if source is empty, hasmarkup is False, or code_highlight is False",
      "Mentions that _highlight uses PYTEST_THEME and PYTEST_THEME_MODE env vars (defaulting to 'dark'), and raises UsageError on invalid values",
      "Mentions the \\x1b[0m prefix for reset and trailing newline stripping in _highlight output",
      "Identifies that _highlight is passed as a _HighlightFunc callback to assertion comparison functions in src/_pytest/assertion/util.py, including use with lexer='diff'",
      "Mentions that create_terminal_writer in src/_pytest/config/__init__.py is how TerminalWriter instances are created (used by TerminalReporter and others)"
    ],
    "key_files": [
      "src/_pytest/_io/terminalwriter.py",
      "src/_pytest/_io/wcwidth.py",
      "src/_pytest/_code/code.py",
      "src/_pytest/assertion/util.py",
      "src/_pytest/terminal.py",
      "src/_pytest/config/__init__.py"
    ],
    "source_doc": "[docstring: src/_pytest/_io/terminalwriter.py] src._pytest._io.terminalwriter.TerminalWriter\nsrc._pytest._io.terminalwriter.TerminalWriter.width_of_current_line:\n  Return an estimate of the width so far in the current line.\n\nsrc._pytest._io.terminalwriter.TerminalWriter._write_source:\n  Write lines of source code possibly highlighted.\n\n        Keeping this private for now because the API is clunky. We should discuss how\n        to evolve the terminal writer so we can have more precise color support, for example\n        being able to write part of a line in one color and the rest in another, and so on.\n\nsrc._pytest._io.terminalwriter.TerminalWriter._highlight:\n  Highlight the given source if we have markup support.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few about truncated files and specific line numbers needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_20",
    "repo": "pytest",
    "question": "How does `ErrorMaker.checked_call` convert OS-level exceptions into typed errno-specific error classes?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `ErrorMaker.checked_call` method implements a pattern for wrapping OS/IO function calls and converting raw `OSError` exceptions into typed errno-specific exception classes. Here's how the algorithm works:\n\n1. **Function invocation**: It calls `func(*args, **kwargs)` in a try block, setting `__tracebackhide__ = True` to hide the frame from pytest tracebacks.\n\n2. **Early re-raise for existing Error instances**: If the function raises an `Error` (the module's base class that subclasses `EnvironmentError`), it's re-raised immediately without modification.\n\n3. **OSError handling with platform-specific logic**:\n   - If the caught `OSError` doesn't have an `errno` attribute, it's re-raised as-is.\n   - On Windows (`sys.platform == \"win32\"`): It looks up the Windows error number in `_winerrnomap` (a dict mapping Windows error codes to POSIX errno values, e.g., 2\u2192ENOENT, 3\u2192ENOENT, 17\u2192EEXIST, 18\u2192EXDEV, 13\u2192EBUSY, 22\u2192ENOTDIR, 20\u2192ENOTDIR, 267\u2192ENOTDIR, 5\u2192EACCES). If the Windows errno is not in the map, the original exception is re-raised.\n   - On non-Windows: If `value.errno` is `None`, a one-off `UnknownErrnoNone` class is dynamically created (not cached). If `value.errno` is an integer, it calls `_geterrnoclass` to get or create the appropriate error class.\n\n4. **Error class creation** (`_geterrnoclass`): Uses a class-level `_errno2class` dict as a cache. On cache miss, it dynamically creates a new class using `type()` with the name from `errno.errorcode` (or `\"UnknownErrno{eno}\"` if not found), with `(Error,)` as the base class tuple, and sets `__module__` to `\"py.error\"` and `__doc__` to `os.strerror(eno)`.\n\n5. **Exception message format**: The raised exception message is formatted as `f\"{func.__name__}{args!r}\"` \u2014 e.g., for `os.stat('/nonexistent')`, the message would be `\"stat('/nonexistent',)\"`. This includes the function name and a repr of the positional args tuple.\n\n6. **Lazy attribute access** (`__getattr__`): When client code accesses `error.ENOENT`, the `__getattr__` method looks up the errno value using `getattr(errno, name)`, creates the class via `_geterrnoclass`, then caches it on the instance with `setattr(self, name, cls)` so subsequent accesses don't trigger `__getattr__` again. Names starting with `_` raise `AttributeError` immediately.\n\n7. **Module-level API**: A singleton `_error_maker = ErrorMaker()` is created, `checked_call` is exposed as a module-level function bound to this instance, and a module-level `__getattr__` delegates to the same instance \u2014 enabling usage like `error.ENOENT` (returns the exception class) and `error.checked_call(os.stat, path)` directly.\n\nThe module is exposed as `sys.modules[\"py.error\"]` via `src/py.py`.",
    "rubric": [
      "Explains that the function calls func(*args, **kwargs) in a try block and catches OSError",
      "Mentions that existing Error instances (module's base class subclassing EnvironmentError) are re-raised immediately without modification",
      "Describes Windows-specific handling: uses _winerrnomap dict to translate Windows error codes to POSIX errno values, re-raises if not found in map",
      "Describes non-Windows handling: creates a one-off UnknownErrnoNone class dynamically if value.errno is None, otherwise uses _geterrnoclass",
      "Explains _geterrnoclass: uses _errno2class dict as cache, on miss dynamically creates class with type() using errno.errorcode name (or UnknownErrno{eno}), base class (Error,), __module__='py.error', and __doc__=os.strerror(eno)",
      "Mentions the exception message format uses func.__name__ and repr of args tuple (e.g., func.__name__ + repr(args))",
      "Describes __getattr__ on ErrorMaker: looks up errno value by name, creates class via _geterrnoclass, caches with setattr for future access",
      "Mentions module-level exposure: singleton _error_maker, module-level checked_call bound to instance, module-level __getattr__ delegating to instance"
    ],
    "key_files": [
      "src/_pytest/_py/error.py",
      "src/_pytest/_py/path.py",
      "src/py.py"
    ],
    "source_doc": "[docstring: src/_pytest/_py/error.py] src._pytest._py.error.ErrorMaker\nsrc._pytest._py.error.ErrorMaker:\n  lazily provides Exception classes for each possible POSIX errno\n    (as defined per the 'errno' module).  All such instances\n    subclass EnvironmentError.\n\nsrc._pytest._py.error.ErrorMaker.checked_call:\n  Call a function and raise an errno-exception if applicable.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of the answer's claims go far beyond what the provided documentation states. The documentation only says ErrorMaker 'lazily provides Exception classes for each possible POSIX errno' that 'subclass EnvironmentError', and that checked_call 'Call a function and raise an errno-exception if applicable.'",
      "Details about _winerrnomap, UnknownErrnoNone, _geterrnoclass, __tracebackhide__, exception message format, __getattr__ caching, _errno2class dict, module-level singleton, sys.modules registration, etc. are all extrapolations presumably from reading the actual source code \u2014 they are plausible but not verifiable from the provided documentation alone.",
      "The claim that Error subclasses EnvironmentError is consistent with the doc stating 'All such instances subclass EnvironmentError'.",
      "The claim about checked_call calling a function and raising errno-exceptions is consistent with the doc's description."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; minor documentation-only rationale was trimmed from two claims.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_21",
    "repo": "pytest",
    "question": "What are the key components and runtime mechanisms of pytest's assertion rewriting system?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `AssertionRewriter` class in `src/_pytest/assertion/rewrite.py` extends `ast.NodeVisitor`. Here are key details discovered from reading the actual source code:\n\n**Class hierarchy and instantiation:**\n- `AssertionRewriter` inherits from `ast.NodeVisitor` (line 615).\n- Its `__init__` takes `module_path: str | None`, `config: Config | None`, and `source: bytes`. It reads the `enable_assertion_pass_hook` ini setting from config (defaulting to `False` if config is None).\n- It is instantiated only once per module, in the `rewrite_asserts()` function (line 419): `AssertionRewriter(module_path, config, source).run(mod)`.\n\n**Import hook integration:**\n- `AssertionRewritingHook` (line 76) inherits from both `importlib.abc.MetaPathFinder` and `importlib.abc.Loader`.\n- In `exec_module()`, it calls `_rewrite_test(fn, self.config)` which parses the source with `ast.parse()`, calls `rewrite_asserts()`, and then compiles the rewritten AST with `compile(tree, strfn, \"exec\", dont_inherit=True)`.\n- The hook is installed in `sys.meta_path` at position 0 by `install_importhook()` in `src/_pytest/assertion/__init__.py` (line 120).\n\n**Special imports injected by `run()`:**\n- The `run()` method inserts two import statements at the top of every rewritten module (after docstrings and `__future__` imports): `import builtins as @py_builtins` and `import _pytest.assertion.rewrite as @pytest_ar` (lines 725-737). These use the `@` prefix which is invalid in Python identifiers.\n\n**Module-level helper functions called at runtime:**\n- `_saferepr(obj)` (line 422): For bound methods (`types.MethodType`), it returns just `obj.__name__` instead of the full repr. It uses `_get_maxsize_for_saferepr()` which returns `None` (unlimited) at verbosity >= 2, `DEFAULT_REPR_MAX_SIZE * 10` (2400) at verbosity >= 1, and `DEFAULT_REPR_MAX_SIZE` (240) at default verbosity. Newlines are replaced with `\\\\n` in the output.\n- `_should_repr_global_name(obj)` (line 477): Returns `True` for callables only if they are instances of `FixtureFunctionDefinition` (from `_pytest.fixtures`); for non-callables, returns `True` if the object has no `__name__` attribute.\n- `_call_reprcompare()` (line 493): Delegates to `util._reprcompare` if available; this is set during test execution by `pytest_runtest_protocol` in `__init__.py` to invoke the `pytest_assertrepr_compare` hook.\n- `_format_boolop()` (line 488): Joins explanations with ' or ' or ' and ' and wraps in parentheses, replacing `%` with `%%`.\n- `_check_if_assertion_pass_impl()` (line 518): Returns `True` if `util._assertion_pass` is set, `False` otherwise.\n- `_call_assertion_pass()` (line 513): Delegates to `util._assertion_pass` if not None.\n\n**Visitor methods (10 total):**\n- `visit_Assert`, `visit_NamedExpr`, `visit_Name`, `visit_BoolOp`, `visit_UnaryOp`, `visit_BinOp`, `visit_Call`, `visit_Starred`, `visit_Attribute`, `visit_Compare`, plus `generic_visit`.\n\n**BINOP_MAP** includes 22 entries mapping AST operator classes to symbol strings, including `ast.MatMult: \"@\"` and `ast.Mod: \"%%\"` (escaped for %-formatting). **UNARY_MAP** maps 4 unary operators: `ast.Not`, `ast.Invert`, `ast.USub`, `ast.UAdd`.\n\n**Pyc caching:** Rewritten bytecode is cached using the tag format `{sys.implementation.cache_tag}-pytest-{version}` (line 68). The `_write_pyc` function writes to a temporary file `{pyc}.{os.getpid()}` and then does an atomic `os.replace()`.\n\n**Scope tracking for walrus operator:** The `run()` method tracks scope changes by pushing `ast.FunctionDef`, `ast.AsyncFunctionDef`, and `ast.ClassDef` nodes onto a scope tuple, using a `_SCOPE_END_MARKER` sentinel (an instance of a simple `Sentinel` class) to know when to pop. The `variables_overwrite` dict is keyed by scope tuples.\n\n**Rewrite disabling:** The static method `is_rewrite_disabled()` checks for the string `\"PYTEST_DONT_REWRITE\"` anywhere in the module docstring.\n\n**`_get_assertion_exprs`** is decorated with `@functools.lru_cache(maxsize=1)` and uses `tokenize.tokenize()` to extract assertion expression text from source bytes, keyed by line number.",
    "rubric": [
      "AssertionRewriter inherits from ast.NodeVisitor and is instantiated per module via rewrite_asserts()",
      "AssertionRewritingHook implements both importlib.abc.MetaPathFinder and importlib.abc.Loader and is installed at sys.meta_path[0]",
      "Two special imports are injected: 'import builtins as @py_builtins' and 'import _pytest.assertion.rewrite as @pytest_ar'",
      "_saferepr handles MethodType specially and adjusts max size based on verbosity level (240 default, 2400 at v>=1, unlimited at v>=2)",
      "_should_repr_global_name returns True for callables only if they are FixtureFunctionDefinition instances, and for non-callables if they lack __name__",
      "_call_reprcompare delegates to util._reprcompare which is set during test execution to invoke pytest_assertrepr_compare hook",
      "Visitor methods include visit_Assert, visit_BoolOp, visit_Compare, visit_BinOp, visit_UnaryOp, visit_Call, visit_Attribute, visit_Name, visit_NamedExpr, and visit_Starred",
      "Pyc caching uses tag format '{sys.implementation.cache_tag}-pytest-{version}' with atomic os.replace() for writing",
      "Walrus operator (NamedExpr) scope tracking uses a scope tuple stack with _SCOPE_END_MARKER sentinel and variables_overwrite dict keyed by scope",
      "is_rewrite_disabled checks for 'PYTEST_DONT_REWRITE' in the module docstring",
      "_get_assertion_exprs is decorated with @functools.lru_cache(maxsize=1) and uses tokenize to extract assertion expressions"
    ],
    "key_files": [
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/assertion/__init__.py",
      "src/_pytest/assertion/util.py",
      "src/_pytest/fixtures.py",
      "src/_pytest/_io/saferepr.py"
    ],
    "source_doc": "[docstring: src/_pytest/assertion/rewrite.py] src._pytest.assertion.rewrite.AssertionRewriter\nsrc._pytest.assertion.rewrite.AssertionRewriter:\n  Assertion rewriting implementation.\n\n    The main entrypoint is to call .run() with an ast.Module instance,\n    this will then find all the assert statements and rewrite them to\n    provide intermediate values and a detailed assertion error.  See\n    http://pybites.blogspot.be/2011/07/behind-scenes-of-pytests-new-assertion.html\n    for an overview of how this works.\n\n    The entry point here is .run() which will iterate over all the\n    statements in an ast.Module and for each ast.Assert statement it\n    finds call .visit() with it.  Then .visit_Assert() takes over and\n    is responsible for creating new ast statements to replace the\n    original assert statement: it rewrites the test of an assertion\n    to provide intermediate values and replace it with an if statement\n    which raises an assertion error with a detailed explanation in\n    case the expression is false and calls pytest_assertion_pass hook\n    if expression is true.\n\n    For this .visit_Assert() uses the visitor pattern to visit all the\n    AST nodes of the ast.Assert.test field, each visit call returning\n    an AST node and the corresponding explanation string.  During this\n    state is kept in several instance attributes:\n\n    :statements: All the AST statements which will replace the assert\n       statement.\n\n    :variables: This is populated by .variable() with each variable\n       used by the statements so that they can all be set to None at\n       the end of the statements.\n\n    :variable_counter: Counter to create new unique variables needed\n       by statements.  Variables are created using .variable() and\n       have the form of \"@py_assert0\".\n\n    :expl_stmts: The AST statements which will be executed to get\n       data from the assertion.  This is the code which will construct\n       the detailed assertion message that is used in the AssertionError\n       or for the pytest_assertion_pass hook.\n\n    :explanation_specifiers: A dict filled by .explanation_param()\n       with %-formatting placeholders and their corresponding\n       expressions to use in the building of an assertion message.\n       This is used by .pop_format_context() to build a message.\n\n    :stack: A stack of the explanation_specifiers dicts maintained by\n       .push_format_context() and .pop_format_context() which allows\n       to build another %-formatted string while already building one.\n\n    :scope: A tuple containing the current scope used for variables_overwrite.\n\n    :variables_overwrite: A dict filled with references to variables\n       that change value within an assert. This happens when a variable is\n       reassigned with the walrus operator\n\n    This state, except the variables_overwrite,  is reset on every new assert\n    statement visited and used by the other visitors.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.run:\n  Find all assert statements in *mod* and rewrite them.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.variable:\n  Get a new variable.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.assign:\n  Give *expr* a name.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.display:\n  Call saferepr on the expression.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.helper:\n  Call a helper in this module.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.builtin:\n  Return the builtin called *name*.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.explanation_param:\n  Return a new named %-formatting placeholder for expr.\n\n        This creates a %-formatting placeholder for expr in the\n        current formatting context, e.g. ``%(py0)s``.  The placeholder\n        and expr are placed in the current format context so that it\n        can be used on the next call to .pop_format_context().\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.push_format_context:\n  Create a new formatting context.\n\n        The format context is used for when an explanation wants to\n        have a variable value formatted in the assertion message.  In\n        this case the value required can be added using\n        .explanation_param().  Finally .pop_format_context() is used\n        to format a string of %-formatted values as added by\n        .explanation_param().\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.pop_format_context:\n  Format the %-formatted string with current format context.\n\n        The expl_expr should be an str ast.expr instance constructed from\n        the %-placeholders created by .explanation_param().  This will\n        add the required code to format said string to .expl_stmts and\n        return the ast.Name instance of the formatted string.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.generic_visit:\n  Handle expressions we don't have custom code for.\n\nsrc._pytest.assertion.rewrite.AssertionRewriter.visit_Assert:\n  Return the AST statements to replace the ast.Assert instance.\n\n        This rewrites the test of an assertion to provide\n        intermediate values and replace it with an if statement which\n        raises an assertion error with a detailed explanation in case\n        the expression is false.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor doc leakage in design rationale phrases ('to avoid name clashes', 'to avoid race conditions', 'to handle walrus operator reassignments').",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_22",
    "repo": "pytest",
    "question": "Why does the `main()` function short-circuit `--version` handling before calling `_prepareconfig()`, and how does double `--version` behave differently?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `main()` function in `src/_pytest/config/__init__.py` short-circuits the `--version` flag handling *before* any pytest infrastructure is initialized. If the args contain exactly one `--version` string (checked via `new_args.count(\"--version\") == 1`), the function immediately writes `pytest {__version__}` to stdout and returns `ExitCode.OK` without ever calling `_prepareconfig()`.\n\nThis matters because the normal initialization path involves significant work:\n\n1. `_prepareconfig()` calls `get_config()`, which instantiates a `PytestPluginManager` and then imports all 28+ default plugins (mark, main, runner, fixtures, helpconfig, python, terminal, debugging, unittest, capture, skipping, legacypath, tmpdir, monkeypatch, recwarn, pastebin, assertion, junitxml, doctest, cacheprovider, setuponly, setupplan, stepwise, unraisableexception, threadexception, warnings, logging, reports, faulthandler, subtests).\n\n2. The initialization path also loads setuptools entrypoints for third-party pytest plugins.\n\n3. The initialization path walks the directory tree loading conftest.py files via `_set_initial_conftests`.\n\n4. `determine_setup()` is called to find ini files and rootdir.\n\nThe test `test_version_less_verbose` verifies this by creating a conftest.py that prints text and confirming it is NOT printed when using `--version`, proving that conftest loading is skipped entirely.\n\nNotably, this early exit only catches the long-form `--version` flag, not the `-V` short form. When `--version` appears twice (`--version --version`), the count equals 2, bypassing the early exit and falling through to the full initialization which enables verbose version output including plugin information (handled by `helpconfig.py`'s `pytest_cmdline_main` where `config.option.version > 1` triggers `show_version_verbose()`).\n\nAnother notable detail in the same initialization path is the `lru_cache(256)` wrapping of `_get_directory` in `PytestPluginManager.__init__`, which caches directory lookups to avoid redundant filesystem calls when `_getconftestmodules()` is called repeatedly with the same path.",
    "rubric": [
      "Explains that single --version avoids initializing the entire pytest infrastructure (instantiating PytestPluginManager, importing default plugins, loading conftest files, etc.)",
      "Mentions the check uses `new_args.count(\"--version\") == 1` to detect a single --version argument",
      "States that when --version appears once, it immediately writes `pytest {version}` to stdout and returns ExitCode.OK",
      "Explains that double --version (count == 2) bypasses the early exit and falls through to full initialization",
      "Mentions that the full initialization path in _prepareconfig/get_config imports 28+ default plugins (mark, main, runner, fixtures, helpconfig, python, terminal, etc.)",
      "Notes that double --version triggers verbose version output with plugin information via helpconfig.py's pytest_cmdline_main (config.option.version > 1)",
      "Mentions that conftest.py loading is skipped in the early-exit path (test_version_less_verbose confirms this)"
    ],
    "key_files": [
      "src/_pytest/config/__init__.py",
      "src/_pytest/helpconfig.py",
      "testing/test_helpconfig.py"
    ],
    "source_doc": "[docstring: src/_pytest/config/__init__.py] src._pytest.config.main\nPerform an in-process test run.\n\n    :param args:\n        List of command line arguments. If `None` or not given, defaults to reading\n        arguments directly from the process command line (:data:`sys.argv`).\n    :param plugins: List of plugin objects to be auto-registered during initialization.\n\n    :returns: An exit code.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the `main()` function's signature (args, plugins parameters) and its return type (exit code). The generated answer makes extensive claims about internal implementation details (line numbers, specific optimizations, issue references, plugin lists, caching strategies) that cannot be verified from the provided documentation alone.",
      "The claims about `--version` short-circuiting, specific line numbers (185-189), issue #13574, the exact list of 28+ default plugins, `lru_cache(256)`, issue #9478, and test names like `test_version_less_verbose` are all implementation details that go far beyond what the documentation states but are plausible code-level details.",
      "Nothing in the answer contradicts the documentation \u2014 it correctly identifies that `main()` performs an in-process test run and returns an exit code, which aligns with the docstring."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only issue references and explicit design rationale text are documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_23",
    "repo": "pytest",
    "question": "Where is the Parser class defined and how is it initialized, including its argument parser setup and key internal data structures?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Parser` class is defined in `src/_pytest/config/argparsing.py` and is marked with `@final`. It is instantiated in `Config.__init__` (in `src/_pytest/config/__init__.py`) with a `processopt` callback parameter.\n\nKey implementation details:\n\n1. **Parser.__init__** creates a `PytestArgumentParser` (a subclass of `argparse.ArgumentParser`) stored in `self.optparser`. The first argument group created is labeled `\"Custom options\"` and assigned to `self._anonymous` as an `OptionGroup`. The `_anonymous` group is the first element in `self._groups` list.\n\n2. **PytestArgumentParser** is configured with `add_help=False`, `formatter_class=DropShorterLongHelpFormatter`, `allow_abbrev=False`, and `fromfile_prefix_chars=\"@\"`. Its `error()` method raises `UsageError` (from `_pytest.config.exceptions`) instead of calling `sys.exit`.\n\n3. **Parser._opt2dest** is a `dict[str, str]` mapping option strings (like `\"-V\"`, `\"--version\"`) to their dest names. It is populated in `OptionGroup._addoption_inner`.\n\n4. **Parser._ini_aliases** is a `dict[str, str]` mapping alias names to canonical ini option names, populated in `addini`.\n\n5. **addini** uses `NOTSET` (imported from `_pytest.compat`) as the sentinel for the `default` parameter. When default is NOTSET, it calls `get_ini_default_for_type()` which returns: `[]` for paths/pathlist/args/linelist, `False` for bool, `0` for int, `0.0` for float, and `\"\"` for string.\n\n6. **parse()** uses `parse_intermixed_args` (standard argparse), and sets `namespace._raise_print_help = True` before parsing. This enables the `HelpAction` class (in `src/_pytest/helpconfig.py`) to raise `PrintHelp` exception when `--help` is encountered (checking `getattr(namespace, \"_raise_print_help\", False)`).\n\n7. **parse_known_and_unknown_args** has a Python version check: for versions before 3.12.8 (or 3.13.0 to 3.13.1), it falls back to `parse_known_args` instead of `parse_known_intermixed_args`.\n\n8. **OptionGroup.addoption** enforces that lowercase short options (e.g., `-x`) are reserved \u2014 the check is `opt[1].islower()` which raises `ValueError(\"lowercase short options are reserved\")`. The internal `_addoption` method bypasses this restriction with `allow_reserved=True`.\n\n9. **OverrideIniAction** (at bottom of argparsing.py) is a custom argparse action. When invoked, it sets the dest to `True` and appends to `namespace.override_ini` list.\n\n10. **check_ispytest** (from `_pytest.deprecated`) guards both `Parser.__init__` and `OptionGroup.__init__`.\n\n11. The `FILE_OR_DIR` constant is `\"file_or_dir\"` and is used as the positional argument name in the parser, with `nargs=\"*\"` and a `filescompleter` from `_pytest._argcomplete` attached to it.",
    "rubric": [
      "Parser is defined in src/_pytest/config/argparsing.py and marked with @final",
      "Parser is instantiated in Config.__init__ with a processopt callback parameter",
      "Parser.__init__ creates a PytestArgumentParser stored in self.optparser, configured with add_help=False, formatter_class=DropShorterLongHelpFormatter, allow_abbrev=False, and fromfile_prefix_chars='@'",
      "PytestArgumentParser.error() raises UsageError instead of calling sys.exit",
      "The first argument group is labeled 'Custom options' and assigned to self._anonymous as an OptionGroup",
      "Parser._opt2dest is a dict mapping option strings to dest names, populated in OptionGroup._addoption_inner",
      "Parser._ini_aliases maps alias names to canonical ini option names, populated in addini",
      "addini uses NOTSET as the default sentinel and get_ini_default_for_type returns type-appropriate defaults ([] for paths/pathlist/args/linelist, False for bool, 0 for int, 0.0 for float, '' for string)",
      "parse() uses parse_intermixed_args and sets namespace._raise_print_help = True before parsing",
      "parse_known_and_unknown_args has a Python version check falling back to parse_known_args for versions before 3.12.8 or 3.13.0-3.13.1",
      "OptionGroup.addoption enforces that lowercase short options are reserved (raises ValueError)",
      "check_ispytest guards both Parser.__init__ and OptionGroup.__init__"
    ],
    "key_files": [
      "src/_pytest/config/argparsing.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/config/exceptions.py",
      "src/_pytest/helpconfig.py",
      "src/_pytest/config/findpaths.py",
      "src/_pytest/_argcomplete.py"
    ],
    "source_doc": "[docstring: src/_pytest/config/argparsing.py] src._pytest.config.argparsing.Parser\nsrc._pytest.config.argparsing.Parser:\n  Parser for command line arguments and config-file values.\n\n    :ivar extra_info: Dict of generic param -> value to display in case\n        there's an error processing the command line arguments.\n\nsrc._pytest.config.argparsing.Parser.getgroup:\n  Get (or create) a named option Group.\n\n        :param name: Name of the option group.\n        :param description: Long description for --help output.\n        :param after: Name of another group, used for ordering --help output.\n        :returns: The option group.\n\n        The returned group object has an ``addoption`` method with the same\n        signature as :func:`parser.addoption <pytest.Parser.addoption>` but\n        will be shown in the respective group in the output of\n        ``pytest --help``.\n\nsrc._pytest.config.argparsing.Parser.addoption:\n  Register a command line option.\n\n        :param opts:\n            Option names, can be short or long options.\n        :param attrs:\n            Same attributes as the argparse library's :meth:`add_argument()\n            <argparse.ArgumentParser.add_argument>` function accepts.\n\n        After command line parsing, options are available on the pytest config\n        object via ``config.option.NAME`` where ``NAME`` is usually set\n        by passing a ``dest`` attribute, for example\n        ``addoption(\"--long\", dest=\"NAME\", ...)``.\n\nsrc._pytest.config.argparsing.Parser.parse:\n  Parse the arguments.\n\n        Unlike ``parse_known_args`` and ``parse_known_and_unknown_args``,\n        raises PrintHelp on `--help` and UsageError on unknown flags\n\n        :meta private:\n\nsrc._pytest.config.argparsing.Parser.parse_known_args:\n  Parse the known arguments at this point.\n\n        :returns: An argparse namespace object.\n\nsrc._pytest.config.argparsing.Parser.parse_known_and_unknown_args:\n  Parse the known arguments at this point, and also return the\n        remaining unknown flag arguments.\n\n        :returns:\n            A tuple containing an argparse namespace object for the known\n            arguments, and a list of unknown flag arguments.\n\nsrc._pytest.config.argparsing.Parser.addini:\n  Register a configuration file option.\n\n        :param name:\n            Name of the configuration.\n        :param type:\n            Type of the configuration. Can be:\n\n                * ``string``: a string\n                * ``bool``: a boolean\n                * ``args``: a list of strings, separated as in a shell\n                * ``linelist``: a list of strings, separated by line breaks\n                * ``paths``: a list of :class:`pathlib.Path`, separated as in a shell\n                * ``pathlist``: a list of ``py.path``, separated as in a shell\n                * ``int``: an integer\n                * ``float``: a floating-point number\n\n                .. versionadded:: 8.4\n\n                    The ``float`` and ``int`` types.\n\n            For ``paths`` and ``pathlist`` types, they are considered relative to the config-file.\n            In case the execution is happening without a config-file defined,\n            they will be considered relative to the current working directory (for example with ``--override-ini``).\n\n            .. versionadded:: 7.0\n                The ``paths`` variable type.\n\n            .. versionadded:: 8.1\n                Use the current working directory to resolve ``paths`` and ``pathlist`` in the absence of a config-file.\n\n            Defaults to ``string`` if ``None`` or not passed.\n        :param default:\n            Default value if no config-file option exists but is queried.\n        :param aliases:\n            Additional names by which this option can be referenced.\n            Aliases resolve to the canonical name.\n\n            .. versionadded:: 9.0\n                The ``aliases`` parameter.\n\n        The value of configuration keys can be retrieved via a call to\n        :py:func:`config.getini(name) <pytest.Config.getini>`.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few claims about specific line numbers, behavior in truncated code sections, and rationale from comments needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_24",
    "repo": "pytest",
    "question": "How does `OptionGroup.addoption` validate, register, and connect a new command-line option through pytest's argument parsing pipeline?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `OptionGroup.addoption` method works through a layered architecture that validates options, registers them with argparse, and connects them to pytest's configuration system. Here's how it works in detail:\n\n1. **Duplicate detection**: `addoption()` first checks for conflicts by intersecting the provided option names (`opts`) with all existing option names already in `self.options`. If any conflict is found, it raises a `ValueError` with the conflicting option names.\n\n2. **Reserved option validation**: `addoption()` calls `_addoption_inner(opts, attrs, allow_reserved=False)`. The `_addoption_inner` method checks if `allow_reserved` is False, and if so, iterates through all option strings. For any option where `len(opt) >= 2 and opt[0] == \"-\" and opt[1].islower()`, it raises `ValueError(\"lowercase short options are reserved\")`. This means single-dash lowercase options like `-x`, `-p`, `-h` cannot be registered via `addoption()` \u2014 only via `_addoption()` which passes `allow_reserved=True`.\n\n3. **argparse integration**: The actual option is registered by calling `self._arggroup.add_argument(*opts, **attrs)`, where `_arggroup` is an `argparse._ArgumentGroup` instance. This returns an `argparse.Action` object.\n\n4. **Argument wrapping**: The returned action is wrapped in a pytest `Argument` object (which delegates `dest`, `default`, `type`, and `names()` to the underlying `argparse.Action`'s properties like `action.dest`, `action.default`, `action.type`, and `action.option_strings`).\n\n5. **Parser registration**: If `self.parser` is not None, for each option name in `option.names()`, it populates `self.parser._opt2dest[name] = option.dest` (mapping option strings like \"-V\" and \"--version\" to their dest like \"version\"). Then it calls `self.parser.processoption(option)`.\n\n6. **Config default setting**: `processoption()` calls the `_processopt` callback if one was set (checked via `if self._processopt:` and `if option.dest:`).\n\n7. **Help formatting**: The `PytestArgumentParser` uses `DropShorterLongHelpFormatter` which suppresses shorter long options that differ only by hyphens. It works by stripping all hyphens from long options and keeping only the longest version (with the most hyphens). For example, `--traceconfig, --trace-config` shows only `--trace-config` in help. If exactly two options exist and one is a short option (2 chars), it uses a shortcut path and shows both (e.g., `-h, --help`).\n\n8. **The `_addoption()` private variant**: Bypasses the lowercase short option reservation check by passing `allow_reserved=True` but also skips the duplicate detection check that `addoption()` performs.\n\n9. **Terminal width**: `DropShorterLongHelpFormatter` uses `_pytest._io.get_terminal_width()` which calls `shutil.get_terminal_size(fallback=(80, 24))` and clamps any width below 40 to 80.\n\n10. **Error handling**: `PytestArgumentParser.error()` overrides argparse's default `error()` method to instead raise a `UsageError` with the formatted usage string and any extra context info.",
    "rubric": [
      "Explains duplicate detection: intersects provided option names with existing option names and raises ValueError if conflicts found",
      "Explains reserved option validation: lowercase short options (single-dash, lowercase second char) are rejected via _addoption_inner with allow_reserved=False",
      "Mentions that _addoption() bypasses the reserved check by passing allow_reserved=True",
      "Describes argparse integration: calls self._arggroup.add_argument(*opts, **attrs) to register with argparse",
      "Describes wrapping the returned argparse Action in an Argument object that delegates dest, default, type, and names()",
      "Explains parser registration: populates parser._opt2dest mapping from option names to dest, then calls parser.processoption()",
      "Explains processoption calls _processopt callback if set and option.dest is truthy",
      "Describes DropShorterLongHelpFormatter: suppresses shorter long options differing only by hyphens, keeping the longest version",
      "Mentions PytestArgumentParser.error() raises UsageError instead of default argparse behavior"
    ],
    "key_files": [
      "src/_pytest/config/argparsing.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/deprecated.py",
      "src/_pytest/_io/terminalwriter.py"
    ],
    "source_doc": "[docstring: src/_pytest/config/argparsing.py] src._pytest.config.argparsing.OptionGroup\nsrc._pytest.config.argparsing.OptionGroup:\n  A group of options shown in its own section.\n\nsrc._pytest.config.argparsing.OptionGroup.addoption:\n  Add an option to this group.\n\n        If a shortened version of a long option is specified, it will\n        be suppressed in the help. ``addoption('--twowords', '--two-words')``\n        results in help showing ``--two-words`` only, but ``--twowords`` gets\n        accepted **and** the automatic destination is in ``args.twowords``.\n\n        :param opts:\n            Option names, can be short or long options.\n            Note that lower-case short options (e.g. `-x`) are reserved.\n        :param attrs:\n            Same attributes as the argparse library's :meth:`add_argument()\n            <argparse.ArgumentParser.add_argument>` function accepts.\n\nsrc._pytest.config.argparsing.OptionGroup._addoption:\n  Like addoption(), but also allows registering short lower case options (e.g. -x),\n        which are reserved for pytest core.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most of the detailed implementation claims (points 1-10) go far beyond what the documentation states and cannot be verified from the provided doc chunk alone. These appear to be based on source code knowledge rather than the documentation provided.",
      "The core claims that are verifiable against the documentation are correct: addoption adds options to the group, short lowercase options are reserved, shortened long options are suppressed in help, --twowords/--two-words behavior matches the doc, opts/attrs parameter descriptions align with the documentation.",
      "Point 8 claims _addoption() 'skips the duplicate detection check' - this is not stated in the documentation; the doc only says it 'also allows registering short lower case options'. This could be accurate code-level detail or could be a hallucination.",
      "The documentation says _addoption is 'Like addoption()' which implies it shares most behavior, so claiming it skips duplicate detection is an unverifiable extrapolation that could contradict the 'like addoption()' framing."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; minor rewrites needed for inferred plugin/core distinction and argparse default behavior.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_25",
    "repo": "pytest",
    "question": "What is the `ConfigValue` dataclass and how does its `origin`/`mode` system integrate with config resolution, alias precedence, and INI vs TOML processing?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `ConfigValue` class is a frozen dataclass defined in `src/_pytest/config/findpaths.py` with three fields: `value: object`, `origin: Literal[\"file\", \"override\"]`, and `mode: Literal[\"ini\", \"toml\"]`. The `origin` and `mode` fields are keyword-only (enforced via `_: KW_ONLY` sentinel). The `ConfigDict` type alias is defined as `dict[str, ConfigValue]`.\n\n**Architecture and Integration:**\n\n1. **Creation points**: `ConfigValue` instances are created in four contexts:\n   - `load_config_dict_from_file()` \u2014 creates values with `origin=\"file\"` and either `mode=\"ini\"` (for .ini, .cfg, and [tool.pytest.ini_options] in .toml) or `mode=\"toml\"` (for [tool.pytest] in pyproject.toml and [pytest] in pytest.toml/.pytest.toml).\n   - `parse_override_ini()` \u2014 creates values with `origin=\"override\"` and `mode=\"ini\"` for -o/--override-ini arguments.\n   - `_DeprecatedInicfgProxy.__setitem__()` \u2014 creates values with `origin=\"override\"` and `mode=\"toml\"` when setting values programmatically through the deprecated `config.inicfg` interface.\n\n2. **Consumption in `Config._getini()`**: The method collects candidates from both the canonical name and aliases in `self._inicfg`. It picks the best candidate using `max(candidates, key=lambda x: (x[0].origin == \"override\", x[1]))`, meaning CLI overrides (`origin=\"override\"`) take precedence over file values, and canonical names take precedence over aliases (second tuple element `is_canonical` is `True` for canonical). After selecting, it dispatches to `_getini_ini()` or `_getini_toml()` based on the `mode`.\n\n3. **INI mode processing (`_getini_ini`)**: Values are `str | list[str]`. String values are coerced based on registered type (e.g., `shlex.split` for \"args\", `int()` for \"int\", splitting on newlines for \"linelist\").\n\n4. **TOML mode processing (`_getini_toml`)**: Values preserve native TOML types with strict type validation \u2014 no coercion is performed. For example, `bool` must be a `bool` (not an int), `int` must be an `int` (not a bool, despite bool being a subclass of int), and `float` accepts both `float` and `int` but not `bool`.\n\n5. **Merge mechanism**: In `determine_setup()`, overrides from `parse_override_ini()` are merged into the config dict via `inicfg.update(ini_overrides)`, so overrides for the same key replace file values in the dict. However, when aliases are involved, the precedence logic in `_getini()` handles the case where both a canonical name and alias exist in `_inicfg` with different origins.\n\n6. **Alias system**: The `_ini_aliases` dict on `Parser` maps alias names to canonical names. The `addini()` method's `aliases` parameter validates that aliases don't conflict with existing options or other aliases. When `getini()` is called with an alias name, it first resolves to the canonical name for cache lookup.",
    "rubric": [
      "Describes ConfigValue as a frozen dataclass with fields value (object), origin (Literal['file', 'override']), and mode (Literal['ini', 'toml']), noting KW_ONLY for origin and mode",
      "Mentions ConfigDict is a type alias for dict[str, ConfigValue]",
      "Identifies creation points: load_config_dict_from_file (origin='file', mode varies), parse_override_ini (origin='override', mode='ini'), and _DeprecatedInicfgProxy.__setitem__ (origin='override', mode='toml')",
      "Explains the candidate selection logic in _getini using max() with key=(origin=='override', is_canonical) for precedence",
      "Explains that INI mode (_getini_ini) coerces string values based on registered type (shlex.split for args, int() for int, etc.)",
      "Explains that TOML mode (_getini_toml) performs strict type validation with no coercion, including special handling like rejecting bool for int despite subclass relationship",
      "Describes the alias system: _ini_aliases maps alias->canonical, addini validates no conflicts, getini resolves alias to canonical for cache lookup",
      "Mentions the merge mechanism in determine_setup where ini_overrides update the config dict via inicfg.update(ini_overrides)"
    ],
    "key_files": [
      "src/_pytest/config/findpaths.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/config/argparsing.py"
    ],
    "source_doc": "[docstring: src/_pytest/config/findpaths.py] src._pytest.config.findpaths.ConfigValue\nsrc._pytest.config.findpaths.ConfigValue:\n  Represents a configuration value with its origin and parsing mode.\n\n    This allows tracking whether a value came from a configuration file\n    or from a CLI override (--override-ini), which is important for\n    determining precedence when dealing with ini option aliases.\n\n    The mode tracks the parsing mode/data model used for the value:\n    - \"ini\": from INI files or [tool.pytest.ini_options], where the only\n      supported value types are `str` or `list[str]`.\n    - \"toml\": from TOML files (not in INI mode), where native TOML types\n       are preserved.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Only one minor version-number claim ('added in version 9.0') was documentation-only; all other claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_26",
    "repo": "pytest",
    "question": "Why does the FixtureManager use negative indexing into ordered fixture definition lists, and what other architectural choices support fixture overriding and scoping?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `FixtureManager` class in `src/_pytest/fixtures.py` is designed with several deliberate architectural choices that balance performance, correctness, and the complexity of fixture scoping/overriding:\n\n**1. Why fixtures are stored in an ordered list per name (`_arg2fixturedefs`)**\nThe `_arg2fixturedefs` dict maps fixture names to a *list* of `FixtureDef` objects. This ordering is critical for the fixture override mechanism. When `_get_active_fixturedef` uses negative indexing (`index = -1`) to get the most specific fixture, the override chain works correctly \u2014 a conftest fixture overrides a plugin fixture, and a module-level fixture overrides both. The chain walk is visible in `_get_active_fixturedef` where `index` starts at `-1` and decrements by 1 for each matching fixture in the request chain (`for request in self._iter_chain(): if request.fixturename == argname: index -= 1`).\n\n**2. Why `getfixtureclosure` returns `arg2fixturedefs` alongside the closure**\n`getfixtureclosure` returns an `arg2fixturedefs` mapping alongside the closure as a performance optimization, avoiding the need to re-discover fixturedefs for each fixture name.\n\n**3. Why `getfixtureclosure` tracks indices with negative indexing**\nThe closure computation mirrors the override chain walk that `_get_active_fixturedef` does at runtime. In `_get_active_fixturedef`, when a fixture overrides another fixture of the same name (requesting itself), the code walks up the chain by decrementing the index to -2, -3, etc., to discover dependencies of overridden fixtures.\n\n**4. Why `_nodeid_autousenames` uses an empty string key**\nThe empty string `\"\"` serves as the root nodeid (matching all nodes), since the Session node's nodeid is `\"\"`. This allows config-level `usefixtures` to be effectively applied globally.\n\n**5. Why `getfixturedefs` returns `None` vs empty tuple**\nThe code distinguishes between \"no fixtures defined with this name at all\" and \"fixtures exist but none applicable to this node.\" This distinction is visible in `_get_active_fixturedef` where the code first checks `self._arg2fixturedefs.get(argname, None)`, then falls back to `self._fixturemanager.getfixturedefs(argname, self._pyfuncitem)`, and then checks both `if fixturedefs is None` and `if not fixturedefs`, raising `FixtureLookupError` in both cases.\n\n**6. Why `parsefactories` uses `safe_getattr` on the type rather than the instance**\nWhen the holder object is not a class or module, `parsefactories` looks up attributes on `type(holderobj)` rather than the instance to avoid triggering `@property` descriptors. Only after confirming a `FixtureFunctionDefinition` is found on the type does it access the instance to get the properly bound method.\n\n**7. Why the closure is sorted by scope (highest first)**\nThe closure is sorted so that higher-scoped fixtures (Session > Package > Module > Class > Function, as defined by the `Scope` enum ordering in `scope.py` and its `__lt__` method) are set up first.\n\n**8. Why `prune_dependency_tree` exists separately from initial closure computation**\nCalled in `src/_pytest/python.py` in `_genfunctions` after `pytest_generate_tests`, `prune_dependency_tree` re-computes the closure because programmatic parametrizations in `metafunc.parametrize()` calls (invoked via `pytest_generate_tests` hooks) may have shadowed fixtures that were included in the original dependency tree. The `prune_dependency_tree` method in `FuncFixtureInfo` starts from `initialnames`, walks the dependency graph using `name2fixturedefs`, and rebuilds `names_closure` to only include reachable fixtures.\n\n**9. Why `reorder_items_atscope` has a PyPy workaround**\nIn `reorder_items_atscope`, there is a check for `sys.pypy_version_info`. On PyPy, instead of using `OrderedDict.move_to_end()`, the code falls back to clearing and rebuilding the dict (`bkp = argkey_dict.copy(); argkey_dict.clear(); argkey_dict[i] = None; argkey_dict.update(bkp)`) to achieve the same reordering effect, working around a PyPy bug with `move_to_end()`.",
    "rubric": [
      "Explains that _arg2fixturedefs maps fixture names to ordered lists of FixtureDefs, where the ordering enables the override mechanism (furthest to closest/most specific)",
      "Explains that _get_active_fixturedef uses negative indexing starting at -1, decrementing for each matching fixture in the request chain, to walk override chains when a fixture requests itself",
      "Explains that getfixtureclosure returns arg2fixturedefs alongside the closure as a performance optimization to avoid re-discovering fixturedefs",
      "Explains that getfixtureclosure tracks current_indices with negative indexing to mirror the override chain walk done at runtime in _get_active_fixturedef",
      "Explains that _nodeid_autousenames uses empty string key because Session's nodeid is '' and this allows config-level usefixtures to apply globally",
      "Explains that getfixturedefs returns None (no fixtures defined at all) vs empty tuple (fixtures exist but none applicable), and both cases are checked separately in _get_active_fixturedef",
      "Explains that parsefactories uses safe_getattr on the type (not instance) to avoid triggering @property descriptors, only accessing the instance after confirming a FixtureFunctionDefinition",
      "Explains that the closure is sorted by scope (highest first) so higher-scoped fixtures are set up first",
      "Explains that prune_dependency_tree exists because pytest_generate_tests/parametrize may shadow fixtures after the initial closure computation",
      "Explains the PyPy workaround in reorder_items_atscope where move_to_end is replaced with clear/rebuild due to a PyPy bug"
    ],
    "key_files": [
      "src/_pytest/fixtures.py",
      "src/_pytest/scope.py",
      "src/_pytest/python.py",
      "src/_pytest/compat.py"
    ],
    "source_doc": "[docstring: src/_pytest/fixtures.py] src._pytest.fixtures.FixtureManager\nsrc._pytest.fixtures.FixtureManager:\n  pytest fixture definitions and information is stored and managed\n    from this class.\n\n    During collection fm.parsefactories() is called multiple times to parse\n    fixture function definitions into FixtureDef objects and internal\n    data structures.\n\n    During collection of test functions, metafunc-mechanics instantiate\n    a FuncFixtureInfo object which is cached per node/func-name.\n    This FuncFixtureInfo object is later retrieved by Function nodes\n    which themselves offer a fixturenames attribute.\n\n    The FuncFixtureInfo object holds information about fixtures and FixtureDefs\n    relevant for a particular function. An initial list of fixtures is\n    assembled like this:\n\n    - config-defined usefixtures\n    - autouse-marked fixtures along the collection chain up from the function\n    - usefixtures markers at module/class/function level\n    - test function funcargs\n\n    Subsequently the funcfixtureinfo.fixturenames attribute is computed\n    as the closure of the fixtures needed to setup the initial fixtures,\n    i.e. fixtures needed by fixture functions themselves are appended\n    to the fixturenames list.\n\n    Upon the test-setup phases all fixturenames are instantiated, retrieved\n    by a lookup of their FuncFixtureInfo.\n\nsrc._pytest.fixtures.FixtureManager.getfixtureinfo:\n  Calculate the :class:`FuncFixtureInfo` for an item.\n\n        If ``func`` is None, or if the item sets an attribute\n        ``nofuncargs = True``, then ``func`` is not examined at all.\n\n        :param node:\n            The item requesting the fixtures.\n        :param func:\n            The item's function.\n        :param cls:\n            If the function is a method, the method's class.\n\nsrc._pytest.fixtures.FixtureManager._getautousenames:\n  Return the names of autouse fixtures applicable to node.\n\nsrc._pytest.fixtures.FixtureManager._getusefixturesnames:\n  Return the names of usefixtures fixtures applicable to node.\n\nsrc._pytest.fixtures.FixtureManager.pytest_generate_tests:\n  Generate new tests based on parametrized fixtures used by the given metafunc\n\nsrc._pytest.fixtures.FixtureManager._register_fixture:\n  Register a fixture\n\n        :param name:\n            The fixture's name.\n        :param func:\n            The fixture's implementation function.\n        :param nodeid:\n            The visibility of the fixture. The fixture will be available to the\n            node with this nodeid and its children in the collection tree.\n            None means that the fixture is visible to the entire collection tree,\n            e.g. a fixture defined for general use in a plugin.\n        :param scope:\n            The fixture's scope.\n        :param params:\n            The fixture's parametrization params.\n        :param ids:\n            The fixture's IDs.\n        :param autouse:\n            Whether this is an autouse fixture.\n\nsrc._pytest.fixtures.FixtureManager.parsefactories:\n  Collect fixtures from a collection node or object.\n\n        Found fixtures are parsed into `FixtureDef`s and saved.\n\n        If `node_or_object` is a collection node (with an underlying Python\n        object), the node's object is traversed and the node's nodeid is used to\n        determine the fixtures' visibility. `nodeid` must not be specified in\n        this case.\n\n        If `node_or_object` is an object (e.g. a plugin), the object is\n        traversed and the given `nodeid` is used to determine the fixtures'\n        visibility. `nodeid` must be specified in this case; None and \"\" mean\n        total visibility.\n\nsrc._pytest.fixtures.FixtureManager.getfixturedefs:\n  Get FixtureDefs for a fixture name which are applicable\n        to a given node.\n\n        Returns None if there are no fixtures at all defined with the given\n        name. (This is different from the case in which there are fixtures\n        with the given name, but none applicable to the node. In this case,\n        an empty result is returned).\n\n        :param argname: Name of the fixture to search for.\n        :param node: The requesting Node.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation-level details (specific line numbers, variable names, internal data structures like `_arg2fixturedefs`, `_nodeid_autousenames`, negative indexing strategies, PyPy workarounds, etc.) that go far beyond what the provided documentation describes. These details cannot be verified from the documentation alone.",
      "The documentation does describe the general flow (parsefactories during collection, FuncFixtureInfo caching, fixture closure computation, the assembly of initial fixtures from config usefixtures/autouse/markers/funcargs), and the answer's high-level claims about these are consistent with the documentation.",
      "The answer's claim about `getfixturedefs` returning `None` vs empty result is consistent with the documentation which states: 'Returns None if there are no fixtures at all defined with the given name. (This is different from the case in which there are fixtures with the given name, but none applicable to the node. In this case, an empty result is returned).'",
      "The answer's description of `parsefactories` behavior regarding collection nodes vs objects is consistent with the documentation's description of the two modes of operation.",
      "The answer's description of `_register_fixture` parameters is consistent with the documented parameters (name, func, nodeid, scope, params, ids, autouse).",
      "Most of the specific implementation details (line numbers, internal variable names, sorting strategies, PyPy bugs) are plausible code-level details but are unverifiable from the provided documentation and could potentially be hallucinated or inaccurate."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims rely on comments/docs (performance rationale, bug references, design explanations) or truncated code sections not visible in the stripped source; these were removed or rewritten to reflect only what's verifiable from the code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_27",
    "repo": "pytest",
    "question": "Where does captured log data flow between the LogCaptureFixture, LoggingPlugin, and the test item's stash during a test run?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `LogCaptureFixture` class's data flows through the `LoggingPlugin` and `catching_logs` context manager into test items via pytest's `Stash` mechanism. Here's the detailed data flow:\n\n1. **Initialization**: The `caplog` fixture function creates a `LogCaptureFixture` with `request.node` (the test item) and calls `_finalize()` after the test yields.\n\n2. **Handler Setup via LoggingPlugin._runtest_for**: The `LoggingPlugin` (registered during `pytest_configure`) uses `catching_logs` to add a shared `self.caplog_handler` (a `LogCaptureHandler` instance) to the root logger. At the start of each phase (setup/call/teardown), it calls `caplog_handler.reset()` which creates a **new** `records` list, then stores that list reference in `item.stash[caplog_records_key][when]` and the handler itself in `item.stash[caplog_handler_key]`.\n\n3. **Data access**: `LogCaptureFixture.handler` retrieves the handler from `self._item.stash[caplog_handler_key]`. The `.records` property returns `self.handler.records`, which is the **same list object** stored in the stash for the current phase.\n\n4. **clear() vs reset() distinction**: `handler.clear()` uses `self.records.clear()` (in-place clearing), so it empties the current phase's records in the stash while preserving previous phases' records (which are separate list objects created by earlier `reset()` calls). `handler.reset()` creates a completely new list (`self.records = []`), severing the link to the old list.\n\n5. **catching_logs data flow**: The `catching_logs` class is NOT a `@contextmanager` \u2014 it is a class implementing `__enter__` and `__exit__` directly. It adds the handler to the root logger in `__enter__` and sets the root logger level to `min(orig_level, level)`. It removes the handler and restores the level in `__exit__`.\n\n6. **_force_enable_logging flow**: When `set_level` or `at_level` is called, `_force_enable_logging` accesses `logger_obj.manager.disable` to get the global disabled level. If `logger_obj.isEnabledFor(level)` returns False, it calculates `disable_level = max(level - 10, logging.NOTSET)` and calls `logging.disable(disable_level)` to enable that level.\n\n7. **LogCaptureHandler.handleError**: If `logging.raiseExceptions` is True, it re-raises the current exception (bare `raise`), making logging formatting errors visible as test failures.\n\n8. **text property flow**: `LogCaptureFixture.text` gets `self.handler.stream.getvalue()` (the handler's `StringIO` stream) and passes it through `_remove_ansi_escape_sequences` which strips ANSI escape codes using regex `r\"\\x1b\\[[\\d;]+m\"`.",
    "rubric": [
      "Explains that the caplog fixture creates a LogCaptureFixture with the test item's node and calls _finalize() after the test yields",
      "Describes that LoggingPlugin._runtest_for uses catching_logs to add caplog_handler to the root logger, calls reset() to create a new records list, and stores the list and handler in item.stash via caplog_records_key and caplog_handler_key",
      "Explains that LogCaptureFixture.handler retrieves the handler from self._item.stash[caplog_handler_key], and .records returns self.handler.records (the same list object stored in the stash)",
      "Distinguishes between handler.clear() (in-place clearing via self.records.clear()) and handler.reset() (creates a new list with self.records = []), explaining how reset() severs the link to the old list while clear() preserves per-phase records",
      "Describes that catching_logs is a class (not a @contextmanager) that adds the handler to the root logger in __enter__ and removes it in __exit__, setting root logger level to min(orig_level, level)",
      "Explains _force_enable_logging: checks logger_obj.manager.disable, and if isEnabledFor returns False, calls logging.disable(max(level - 10, NOTSET)) to enable the desired level",
      "Notes that LogCaptureHandler.handleError re-raises the current exception (bare raise) when logging.raiseExceptions is True",
      "Explains the text property flow: reads handler.stream.getvalue() and strips ANSI escape sequences via _remove_ansi_escape_sequences using regex r'\\x1b\\[[\\d;]+m'"
    ],
    "key_files": [
      "src/_pytest/logging.py",
      "src/_pytest/stash.py",
      "testing/logging/test_fixture.py"
    ],
    "source_doc": "[docstring: src/_pytest/logging.py] src._pytest.logging.LogCaptureFixture\nsrc._pytest.logging.LogCaptureFixture:\n  Provides access and control of log capturing.\n\nsrc._pytest.logging.LogCaptureFixture._finalize:\n  Finalize the fixture.\n\n        This restores the log levels and the disabled logging levels changed by :meth:`set_level`.\n\nsrc._pytest.logging.LogCaptureFixture.handler:\n  Get the logging handler used by the fixture.\n\nsrc._pytest.logging.LogCaptureFixture.get_records:\n  Get the logging records for one of the possible test phases.\n\n        :param when:\n            Which test phase to obtain the records from.\n            Valid values are: \"setup\", \"call\" and \"teardown\".\n\n        :returns: The list of captured records at the given stage.\n\n        .. versionadded:: 3.4\n\nsrc._pytest.logging.LogCaptureFixture.text:\n  The formatted log text.\n\nsrc._pytest.logging.LogCaptureFixture.records:\n  The list of log records.\n\nsrc._pytest.logging.LogCaptureFixture.record_tuples:\n  A list of a stripped down version of log records intended\n        for use in assertion comparison.\n\n        The format of the tuple is:\n\n            (logger_name, log_level, message)\n\nsrc._pytest.logging.LogCaptureFixture.messages:\n  A list of format-interpolated log messages.\n\n        Unlike 'records', which contains the format string and parameters for\n        interpolation, log messages in this list are all interpolated.\n\n        Unlike 'text', which contains the output from the handler, log\n        messages in this list are unadorned with levels, timestamps, etc,\n        making exact comparisons more reliable.\n\n        Note that traceback or stack info (from :func:`logging.exception` or\n        the `exc_info` or `stack_info` arguments to the logging functions) is\n        not included, as this is added by the formatter in the handler.\n\n        .. versionadded:: 3.7\n\nsrc._pytest.logging.LogCaptureFixture.clear:\n  Reset the list of log records and the captured log text.\n\nsrc._pytest.logging.LogCaptureFixture._force_enable_logging:\n  Enable the desired logging level if the global level was disabled via ``logging.disabled``.\n\n        Only enables logging levels greater than or equal to the requested ``level``.\n\n        Does nothing if the desired ``level`` wasn't disabled.\n\n        :param level:\n            The logger level caplog should capture.\n            All logging is enabled if a non-standard logging level string is supplied.\n            Valid level strings are in :data:`logging._nameToLevel`.\n        :param logger_obj: The logger object to check.\n\n        :return: The original disabled logging level.\n\nsrc._pytest.logging.LogCaptureFixture.set_level:\n  Set the threshold level of a logger for the duration of a test.\n\n        Logging messages which are less severe than this level will not be captured.\n\n        .. versionchanged:: 3.4\n            The levels of the loggers changed by this function will be\n            restored to their initial values at the end of the test.\n\n        Will enable the requested logging level if it was disabled via :func:`logging.disable`.\n\n        :param level: The level.\n        :param logger: The logger to update. If not given, the root logger.\n\nsrc._pytest.logging.LogCaptureFixture.at_level:\n  Context manager that sets the level for capturing of logs. After\n        the end of the 'with' statement the level is restored to its original\n        value.\n\n        Will enable the requested logging level if it was disabled via :func:`logging.disable`.\n\n        :param level: The level.\n        :param logger: The logger to update. If not given, the root logger.\n\nsrc._pytest.logging.LogCaptureFixture.filtering:\n  Context manager that temporarily adds the given filter to the caplog's\n        :meth:`handler` for the 'with' statement block, and removes that filter at the\n        end of the block.\n\n        :param filter_: A custom :class:`logging.Filter` object.\n\n        .. versionadded:: 7.5",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, internal method implementations, stash mechanisms, catching_logs internals, handleError behavior, ANSI escape regex patterns, etc.) that go far beyond what the documentation describes. These cannot be verified from the provided documentation alone.",
      "The documentation does not mention LoggingPlugin, catching_logs, caplog_handler, caplog_records_key, caplog_handler_key, Stash, handleError, _remove_ansi_escape_sequences, or any of the internal implementation details described in the answer.",
      "Claims about clear() vs reset() distinction, the catching_logs class not being a @contextmanager, _force_enable_logging calculating 'max(level - 10, logging.NOTSET)', and LogCaptureHandler.handleError behavior are all unverifiable from the documentation provided.",
      "The claims that are verifiable against the documentation appear consistent: set_level restores levels at end of test (matches _finalize and set_level docs), _force_enable_logging enables logging disabled via logging.disable (matches doc), at_level is a context manager (matches doc), get_records takes 'setup'/'call'/'teardown' phases (matches doc), messages are format-interpolated (matches doc).",
      "The description of _force_enable_logging checking logger_obj.manager.disable and calling logging.disable() is plausible but not confirmed by the documentation, which only says it 'enables the desired logging level if the global level was disabled via logging.disabled' and 'only enables logging levels greater than or equal to the requested level'."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: line number references and a comment-sourced rationale ('for performance reasons') are not recoverable from stripped code, but nearly all claims are verifiable from code structure and logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_28",
    "repo": "pytest",
    "question": "How does `pytest.HIDDEN_PARAM` work to hide parameters from test names, and what prevents multiple hidden params in one parametrize call?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `pytest.param()` function is implemented as a thin wrapper that delegates to `ParameterSet.param()` classmethod (in `src/_pytest/mark/structures.py`). Here's how the system works:\n\n1. **`ParameterSet` is a `NamedTuple`** with three fields: `values` (Sequence), `marks` (Collection of MarkDecorator/Mark), and `id` (str | _HiddenParam | None).\n\n2. **`ParameterSet.param()` validation logic:**\n   - If `marks` is a single `MarkDecorator`, it wraps it in a tuple. Otherwise it asserts it's a `collections.abc.Collection`.\n   - It explicitly checks `if any(i.name == \"usefixtures\" for i in marks)` and raises a `ValueError` if found.\n   - For `id`, it validates the type: if not None, it must be either a `str` or the exact `HIDDEN_PARAM` sentinel (identity check with `is`), otherwise raises `TypeError`.\n\n3. **`HIDDEN_PARAM` is implemented as an enum singleton:** `_HiddenParam` is an `enum.Enum` with a single member `token = 0`, and `HIDDEN_PARAM = _HiddenParam.token`.\n\n4. **How HIDDEN_PARAM hides the parameter from the test name:**\n   - In `CallSpec2.setmulti()`: `_idlist=self._idlist if id is HIDDEN_PARAM else [*self._idlist, id]` \u2014 when HIDDEN_PARAM is the id, the `_idlist` is not extended at all.\n   - In `PyCollector._genfunctions()`: `subname = f\"{name}[{callspec.id}]\" if callspec._idlist else name` \u2014 if `_idlist` is empty (all params are hidden), the test name is just the bare function name without any `[...]` suffix.\n   - `CallSpec2.id` property returns `\"-\".join(self._idlist)`, so hidden params don't contribute to the bracketed portion.\n\n5. **Multiple HIDDEN_PARAM prevention:** In `IdMaker.make_unique_parameterset_ids()`, during duplicate detection, if a duplicate ID is `HIDDEN_PARAM` (identity check), it calls `_complain_multiple_hidden_parameter_sets()` which calls `fail()` with a message explaining that multiple HIDDEN_PARAM cannot be used in the same parametrize call.\n\n6. **In `_resolve_ids()`**, HIDDEN_PARAM can come from two sources:\n   - `parameterset.id is HIDDEN_PARAM` \u2014 from `pytest.param(..., id=pytest.HIDDEN_PARAM)`\n   - `self.ids[idx] is HIDDEN_PARAM` \u2014 from `parametrize(..., ids=[..., pytest.HIDDEN_PARAM, ...])`\n\n7. **With multiple `@pytest.mark.parametrize` decorators**, a HIDDEN_PARAM in one parametrize only omits that parameter's ID token from the final name while other parametrize IDs still appear. For example, `parametrize(\"foo\", [\"a\", \"b\"], ids=[\"a\", pytest.HIDDEN_PARAM])` combined with `parametrize(\"bar\", [\"x\", \"y\"])` produces test names: `test_func[a-x]`, `test_func[a-y]`, `test_func[x]`, `test_func[y]`.\n\n8. **With `strict_parametrization_ids` enabled**, HIDDEN_PARAM is displayed as `\"<hidden>\"` in duplicate ID error messages.",
    "rubric": [
      "Explains that HIDDEN_PARAM is implemented as an enum singleton (_HiddenParam enum with a single 'token' member)",
      "Explains that in CallSpec2.setmulti(), when id is HIDDEN_PARAM, the _idlist is not extended (identity check with 'is')",
      "Explains that test name generation in _genfunctions checks if _idlist is empty to decide whether to add the [id] suffix",
      "Explains that CallSpec2.id property joins _idlist with '-', so hidden params contribute nothing to the bracketed portion",
      "Explains that in IdMaker.make_unique_parameterset_ids(), duplicate HIDDEN_PARAM entries are detected and _complain_multiple_hidden_parameter_sets() is called which raises a failure",
      "Explains that _resolve_ids() recognizes HIDDEN_PARAM from two sources: parameterset.id and the ids list",
      "Explains that ParameterSet.param() validates the id parameter accepts str, HIDDEN_PARAM sentinel (via identity check), or None"
    ],
    "key_files": [
      "src/_pytest/mark/structures.py",
      "src/_pytest/python.py",
      "src/_pytest/mark/__init__.py"
    ],
    "source_doc": "[docstring: src/_pytest/mark/__init__.py] src._pytest.mark.param\nSpecify a parameter in `pytest.mark.parametrize`_ calls or\n    :ref:`parametrized fixtures <fixture-parametrize-marks>`.\n\n    .. code-block:: python\n\n        @pytest.mark.parametrize(\n            \"test_input,expected\",\n            [\n                (\"3+5\", 8),\n                pytest.param(\"6*9\", 42, marks=pytest.mark.xfail),\n            ],\n        )\n        def test_eval(test_input, expected):\n            assert eval(test_input) == expected\n\n    :param values: Variable args of the values of the parameter set, in order.\n\n    :param marks:\n        A single mark or a list of marks to be applied to this parameter set.\n\n        :ref:`pytest.mark.usefixtures <pytest.mark.usefixtures ref>` cannot be added via this parameter.\n\n    :type id: str | Literal[pytest.HIDDEN_PARAM] | None\n    :param id:\n        The id to attribute to this parameter set.\n\n        .. versionadded:: 8.4\n            :ref:`hidden-param` means to hide the parameter set\n            from the test name. Can only be used at most 1 time, as\n            test names need to be unique.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Only the PEP 484 singleton pattern reference was removed as documentation-only; one claim was rewritten to correct the method location.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_29",
    "repo": "pytest",
    "question": "What is ExpressionMatcher, how does it get used during expression evaluation, and what concrete implementations satisfy it?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`ExpressionMatcher` is a `Protocol` class (from `typing`) defined in `src/_pytest/mark/expression.py` that specifies the interface for matcher callables used in pytest's `-k` and `-m` expression evaluation system. Its `__call__` method signature is `def __call__(self, name: str, /, **kwargs: str | int | bool | None) -> bool`. It is not a concrete class \u2014 it defines a structural typing contract that any callable matching the signature can satisfy.\n\nThe protocol is used by `Expression.evaluate()`, which compiles the match expression string into a Python AST, then evaluates it using `eval()` with `{'__builtins__': {}}` as globals and a `MatcherAdapter` instance as locals. The `MatcherAdapter` is a `Mapping[str, MatcherNameAdapter]` that intercepts name lookups during `eval()`. When `eval()` looks up a name, `MatcherAdapter.__getitem__` strips the `IDENT_PREFIX` (which is `'$'`) from the key and returns a `MatcherNameAdapter` wrapping the matcher and the stripped name.\n\n`MatcherNameAdapter` is a `@dataclasses.dataclass` with two fields (`matcher` and `name`). It has a `__bool__` method (for simple identifier expressions like `foo`) that calls `self.matcher(self.name)` with no kwargs, and a `__call__` method (for expressions like `foo(bar=True)`) that calls `self.matcher(self.name, **kwargs)`.\n\nThere are two concrete implementations that satisfy the `ExpressionMatcher` protocol:\n1. `KeywordMatcher` (in `src/_pytest/mark/__init__.py`) \u2014 used for `-k` expressions. It performs case-insensitive substring matching against collected item names. Its `__call__` raises `UsageError` if kwargs are provided, explicitly rejecting the kwargs syntax.\n2. `MarkMatcher` (in `src/_pytest/mark/__init__.py`) \u2014 used for `-m` expressions. It supports kwargs by checking if any marker with the given name has kwargs matching all provided key-value pairs, using `NOTSET` (imported from `_pytest.compat`) as the default for missing keys.\n\nThe expression grammar is parsed via a recursive descent parser (functions `expression`, `expr`, `and_expr`, `not_expr`) that builds a Python AST. The `not_expr` function handles identifier parsing and, if followed by `(`, parses kwargs via `all_kwargs`/`single_kwarg`. Kwarg values can be string literals (quotes stripped), integers (including negative), or the builtins `True`, `False`, `None` (from the `BUILTIN_MATCHERS` dict). Kwarg names must be valid Python identifiers and not reserved keywords (checked via `keyword.iskeyword`).",
    "rubric": [
      "Identifies ExpressionMatcher as a Protocol class (structural typing contract) with __call__(self, name: str, /, **kwargs: str | int | bool | None) -> bool",
      "Explains that Expression.evaluate() compiles the expression into a Python AST and uses eval() with empty __builtins__ and a MatcherAdapter as locals",
      "Describes MatcherAdapter as a Mapping that intercepts name lookups, strips the IDENT_PREFIX ('$'), and returns MatcherNameAdapter instances",
      "Explains MatcherNameAdapter's __bool__ method (calls matcher with just the name) and __call__ method (calls matcher with name and kwargs)",
      "Identifies KeywordMatcher as a concrete implementation for -k expressions that does case-insensitive substring matching and raises UsageError if kwargs are provided",
      "Identifies MarkMatcher as a concrete implementation for -m expressions that supports kwargs by checking marker kwargs using NOTSET as default for missing keys"
    ],
    "key_files": [
      "src/_pytest/mark/expression.py",
      "src/_pytest/mark/__init__.py",
      "testing/test_mark_expression.py"
    ],
    "source_doc": "[docstring: src/_pytest/mark/expression.py] src._pytest.mark.expression.ExpressionMatcher\nsrc._pytest.mark.expression.ExpressionMatcher:\n  A callable which, given an identifier and optional kwargs, should return\n    whether it matches in an :class:`Expression` evaluation.\n\n    Should be prepared to handle arbitrary strings as input.\n\n    If no kwargs are provided, the expression of the form `foo`.\n    If kwargs are provided, the expression is of the form `foo(1, b=True, \"s\")`.\n\n    If the expression is not supported (e.g. don't want to accept the kwargs\n    syntax variant), should raise :class:`~pytest.UsageError`.\n\n    Example::\n\n        def matcher(name: str, /, **kwargs: str | int | bool | None) -> bool:\n            # Match `cat`.\n            if name == \"cat\" and not kwargs:\n                return True\n            # Match `dog(barks=True)`.\n            if name == \"dog\" and kwargs == {\"barks\": False}:\n                return True\n            return False",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation example shows `kwargs == {\"barks\": False}` for matching `dog(barks=True)` in the comment, which is contradictory within the doc itself. The generated answer doesn't address this inconsistency.",
      "Many implementation details (MatcherAdapter, MatcherNameAdapter, IDENT_PREFIX='$', recursive descent parser functions, KeywordMatcher, MarkMatcher, NOTSET sentinel, keyword.iskeyword check, BUILTIN_MATCHERS dict) are not present in the provided documentation and cannot be verified from it alone \u2014 they are plausible code-level extrapolations but cannot be confirmed from this doc chunk.",
      "The claim that ExpressionMatcher is a Protocol class is not stated in the documentation snippet provided, though it is a plausible implementation detail.",
      "The claim about eval() with {'__builtins__': {}} is not in the documentation and cannot be verified from it."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only a minor detail about NOTSET's type needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_30",
    "repo": "pytest",
    "question": "Why does the `Expression` class prefix identifiers with `$` and pass `{\"__builtins__\": {}}` to `eval()` in its evaluation mechanism?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `Expression` class in `src/_pytest/mark/expression.py` uses Python's built-in `eval()` to evaluate compiled match expressions, and the design employs several deliberate patterns:\n\n**`IDENT_PREFIX = \"$\"` and identifier prefixing:** All identifiers in the generated AST are prefixed with `\"$\"` (e.g., `$True` instead of `True`), as seen in `ast.Name(IDENT_PREFIX + ident.value, ast.Load())`. This prevents collisions with Python reserved names during `eval()`, since `True`, `False`, and `None` can appear as identifiers in match expressions (as seen in the `BUILTIN_MATCHERS` dict). The `MatcherAdapter.__getitem__` strips this prefix back off (`key[len(IDENT_PREFIX):]`) before passing the name to the actual matcher.\n\n**`{\"__builtins__\": {}}` passed to eval:** In `Expression.evaluate()`, the globals dict explicitly sets `__builtins__` to an empty dict. This prevents the evaluated code from accessing any Python built-in functions or names, ensuring the expression can only resolve identifiers through the `MatcherAdapter` locals mapping.\n\n**`MatcherNameAdapter` has both `__bool__` and `__call__`:** The `MatcherNameAdapter` class serves as a dual-purpose proxy. When an identifier appears alone (e.g., `foo`), Python's `eval()` evaluates it in a boolean context, triggering `__bool__()` which calls `self.matcher(self.name)` with no kwargs. When an identifier is followed by parenthesized kwargs (e.g., `foo(a=1)`), the AST contains an `ast.Call` node (created in `not_expr` as `ast.Call(func=name, args=[], keywords=all_kwargs(s))`), so `eval()` calls `__call__(**kwargs)` which passes kwargs through to the matcher. This allows the same evaluation mechanism to handle both the simple `-k`/`-m` syntax and the extended kwargs syntax.\n\n**`KeywordMatcher` raises `UsageError` on kwargs:** The `KeywordMatcher.__call__` explicitly raises `UsageError(\"Keyword expressions do not support call parameters.\")` when kwargs are provided. `KeywordMatcher` performs substring matching on test names (`any(subname in name.lower() for name in self._names)`). In contrast, `MarkMatcher` supports kwargs by iterating through matching marks and checking if `mark.kwargs.get(k, NOTSET) == v` for each kwarg \u2014 using the `NOTSET` sentinel (an enum value from `_pytest.compat`) to distinguish between a kwarg being absent vs. being set to any value.\n\n**Empty expressions evaluate to False:** The `expression()` function checks if the scanner immediately hits EOF, and if so returns `ast.Constant(False)`. This means an empty expression evaluates to `False`.",
    "rubric": [
      "Explains that the `$` prefix (IDENT_PREFIX) prevents collisions with Python reserved names like True, False, and None which are valid match expression identifiers",
      "Mentions that `{\"__builtins__\": {}}` prevents access to Python built-in functions/names, restricting identifier resolution to the MatcherAdapter locals mapping",
      "Explains that MatcherAdapter.__getitem__ strips the prefix back off before passing the name to the actual matcher",
      "Describes how MatcherNameAdapter enables dual-purpose evaluation: __bool__ for bare identifiers and __call__ for identifiers with kwargs",
      "Notes that empty expressions evaluate to False via ast.Constant(False) when EOF is immediately encountered"
    ],
    "key_files": [
      "src/_pytest/mark/expression.py",
      "src/_pytest/mark/__init__.py",
      "src/_pytest/compat.py"
    ],
    "source_doc": "[docstring: src/_pytest/mark/expression.py] src._pytest.mark.expression.Expression\nsrc._pytest.mark.expression.Expression:\n  A compiled match expression as used by -k and -m.\n\n    The expression can be evaluated against different matchers.\n\nsrc._pytest.mark.expression.Expression.compile:\n  Compile a match expression.\n\n        :param input: The input expression - one line.\n\n        :raises SyntaxError: If the expression is malformed.\n\nsrc._pytest.mark.expression.Expression.evaluate:\n  Evaluate the match expression.\n\n        :param matcher:\n            A callback which determines whether an identifier matches or not.\n            See the :class:`ExpressionMatcher` protocol for details and example.\n\n        :returns: Whether the expression matches or not.\n\n        :raises UsageError:\n            If the matcher doesn't support the expression. Cannot happen if the\n            matcher supports all expressions.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific variable names like IDENT_PREFIX, MatcherAdapter, MatcherNameAdapter, KeywordMatcher, MarkMatcher, NOTSET sentinel, etc.) that are not present in the provided documentation chunk. These details cannot be verified against the given documentation.",
      "The documentation only mentions that Expression.evaluate raises UsageError 'If the matcher doesn't support the expression', while the answer claims specific details about KeywordMatcher raising UsageError on kwargs - this is a plausible extrapolation but cannot be verified from the doc.",
      "The documentation mentions ExpressionMatcher protocol but the answer discusses MatcherAdapter and MatcherNameAdapter which are not mentioned in the documentation.",
      "Claims about eval(), __builtins__, AST manipulation, $-prefixing, __bool__/__call__ methods, and empty expression behavior are all implementation details not verifiable from the provided documentation chunk."
    ],
    "strip_verify_leakage": 0.13,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale statements rely on comments that were stripped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_31",
    "repo": "pytest",
    "question": "Where is the `MarkDecorator` class defined, and how does it connect to the rest of the pytest marking system end-to-end?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `MarkDecorator` class is located in `src/_pytest/mark/structures.py`. Here is where it connects to the broader pytest system:\n\n**Creation flow:** `MarkDecorator` instances are created by `MarkGenerator.__getattr__`, which is exposed as the `pytest.mark` singleton via `MARK_GEN = MarkGenerator(_ispytest=True)`, and imported in `src/pytest/__init__.py` as `from _pytest.mark import MARK_GEN as mark`. When you access `pytest.mark.NAME`, `MarkGenerator.__getattr__` creates a new `MarkDecorator` wrapping a `Mark(name, (), {}, _ispytest=True)` with empty args and kwargs.\n\n**Application flow (store_mark):** When `MarkDecorator.__call__` determines it's being applied to a function/class, it calls `store_mark(unwrapped_func, self.mark)`. `store_mark` first checks if the target is a fixture by calling `getfixturemarker()` from `src/_pytest/fixtures.py`. If it is a fixture, `store_mark` raises an error via `fail()`. Otherwise, it appends the mark to the object's `pytestmark` attribute list.\n\n**Staticmethod/classmethod unwrapping:** In `__call__`, if the argument is a `staticmethod` or `classmethod`, it unwraps via `func.__func__` and stores the mark on the underlying function, but returns the original descriptor object.\n\n**Mark retrieval during collection:** `get_unpacked_marks()` retrieves marks from objects. For classes with `consider_mro=True`, it walks the entire MRO in reverse order collecting `pytestmark` attributes. This function is called in `src/_pytest/python.py` in `PyobjMixin.obj` property getter to populate `own_markers` on test items.\n\n**Node integration:** Marks are also added dynamically to nodes via `Node.add_marker()` in `src/_pytest/nodes.py`, which can accept either a `MarkDecorator` instance or a string (which gets resolved through `MARK_GEN`).\n\n**Key structural details:**\n- `MarkDecorator` is a `@dataclasses.dataclass` (non-frozen, non-final)\n- `Mark` is `@final @dataclasses.dataclass(frozen=True)` and uses `object.__setattr__` in its `__init__` to bypass frozen=True\n- Both `Mark.__init__` and `MarkDecorator.__init__` require `_ispytest=True` to avoid a `PytestDeprecationWarning` (via `check_ispytest` in `src/_pytest/deprecated.py`)\n- `MarkGenerator.__getattr__` rejects names starting with underscore and checks three common misspellings of 'parametrize': 'parameterize', 'parametrise', 'parameterise'\n- `istestfunc` checks that the argument is callable and its `__name__` is not `<lambda>`\n- `MarkGenerator._config` is set during `pytest_configure` and restored during `pytest_unconfigure` in `src/_pytest/mark/__init__.py`\n- `Markable` is a TypeVar bound to `Callable[..., object] | type`",
    "rubric": [
      "Identifies that MarkDecorator is defined in src/_pytest/mark/structures.py as a @dataclasses.dataclass",
      "Explains that MarkDecorator instances are created via MarkGenerator.__getattr__, which wraps a Mark(name, (), {}) with empty args/kwargs",
      "Explains that MARK_GEN = MarkGenerator(_ispytest=True) is the singleton, imported in src/pytest/__init__.py as 'from _pytest.mark import MARK_GEN as mark'",
      "Describes that MarkDecorator.__call__ unwraps staticmethod/classmethod via __func__ and calls store_mark on the underlying function",
      "Explains that store_mark checks for fixtures via getfixturemarker() and raises an error if the target is a fixture, otherwise appends the mark to the object's pytestmark list",
      "Describes that get_unpacked_marks retrieves marks from objects, walking the MRO in reverse for classes when consider_mro=True",
      "Mentions that get_unpacked_marks is called in src/_pytest/python.py (e.g., in PyobjMixin or similar) to populate own_markers on test items",
      "Explains that Node.add_marker in src/_pytest/nodes.py can accept a string or MarkDecorator and adds marks dynamically to nodes",
      "Notes that Mark is @final @dataclasses.dataclass(frozen=True) and uses object.__setattr__ to bypass frozen=True in __init__",
      "Mentions that both Mark.__init__ and MarkDecorator.__init__ require _ispytest=True to avoid deprecation warnings (via check_ispytest)",
      "Notes that MarkGenerator.__getattr__ rejects names starting with underscore and checks common misspellings of 'parametrize'",
      "Mentions that MarkGenerator._config is set during pytest_configure and restored during pytest_unconfigure"
    ],
    "key_files": [
      "src/_pytest/mark/structures.py",
      "src/_pytest/nodes.py",
      "src/_pytest/python.py",
      "src/_pytest/deprecated.py",
      "src/_pytest/mark/__init__.py"
    ],
    "source_doc": "[docstring: src/_pytest/mark/structures.py] src._pytest.mark.structures.MarkDecorator\nsrc._pytest.mark.structures.MarkDecorator:\n  A decorator for applying a mark on test functions and classes.\n\n    ``MarkDecorators`` are created with ``pytest.mark``::\n\n        mark1 = pytest.mark.NAME  # Simple MarkDecorator\n        mark2 = pytest.mark.NAME(name1=value)  # Parametrized MarkDecorator\n\n    and can then be applied as decorators to test functions::\n\n        @mark2\n        def test_function():\n            pass\n\n    When a ``MarkDecorator`` is called, it does the following:\n\n    1. If called with a single class as its only positional argument and no\n       additional keyword arguments, it attaches the mark to the class so it\n       gets applied automatically to all test cases found in that class.\n\n    2. If called with a single function as its only positional argument and\n       no additional keyword arguments, it attaches the mark to the function,\n       containing all the arguments already stored internally in the\n       ``MarkDecorator``.\n\n    3. When called in any other case, it returns a new ``MarkDecorator``\n       instance with the original ``MarkDecorator``'s content updated with\n       the arguments passed to this call.\n\n    Note: The rules above prevent a ``MarkDecorator`` from storing only a\n    single function or class reference as its positional argument with no\n    additional keyword or positional arguments. You can work around this by\n    using `with_args()`.\n\nsrc._pytest.mark.structures.MarkDecorator.__init__:\n  :meta private:\n\nsrc._pytest.mark.structures.MarkDecorator.name:\n  Alias for mark.name.\n\nsrc._pytest.mark.structures.MarkDecorator.args:\n  Alias for mark.args.\n\nsrc._pytest.mark.structures.MarkDecorator.kwargs:\n  Alias for mark.kwargs.\n\nsrc._pytest.mark.structures.MarkDecorator.markname:\n  :meta private:\n\nsrc._pytest.mark.structures.MarkDecorator.with_args:\n  Return a MarkDecorator with extra arguments added.\n\n        Unlike calling the MarkDecorator, with_args() can be used even\n        if the sole argument is a callable/class.\n\nsrc._pytest.mark.structures.MarkDecorator.__call__:\n  Call the MarkDecorator.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many specific claims about line numbers, implementation details (e.g., line 598-640, line 643, line 470, line 412-413, etc.), internal function names (store_mark, get_unpacked_marks, istestfunc), dataclass decorators, misspelling checks, and other code-level details are not present in the provided documentation and cannot be verified from it. These are plausible extrapolations from actual source code but are not supported by the given doc chunk.",
      "The core behavioral claims about MarkDecorator (creation via pytest.mark, application to functions/classes, returning new MarkDecorator in other cases, with_args() workaround) are consistent with the documentation.",
      "The claim about `with_args()` being usable 'even if the sole argument is a callable/class' is supported by the documentation.",
      "The claim about `name`, `args`, `kwargs` being aliases for `mark.name`, `mark.args`, `mark.kwargs` is supported by the documentation.",
      "No claims directly contradict the provided documentation."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only one claim about getfixturemarker's internal implementation was removed, and several line number references were dropped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_32",
    "repo": "pytest",
    "question": "How does Metafunc.parametrize() work end-to-end, from parsing arguments to generating the final cartesian product of CallSpec2 instances?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `Metafunc.parametrize()` method implements parametrization through the following algorithm:\n\n1. **Argument Parsing**: Calls `ParameterSet._for_parametrize()` which parses `argnames` (splitting comma-separated strings via `_parse_parametrize_args`, handling trailing commas for single-arg tuple notation) and converts `argvalues` into a list of `ParameterSet` namedtuples (with `values`, `marks`, `id` fields). If argvalues is empty, creates a single parameter set with NOTSET values and applies a mark from the `empty_parameterset_option` config setting (skip/xfail/fail_at_collect).\n\n2. **Validation**: Checks 'request' is not in argnames (raises fail). Validates via `_validate_if_using_arg_names()` that each argname is in `self.fixturenames`. If not, it checks `get_default_arg_names()` (which inspects function signature for POSITIONAL_OR_KEYWORD/KEYWORD_ONLY parameters with defaults) to give a specific error about default values vs. missing fixtures/arguments.\n\n3. **Scope Resolution**: If scope is explicitly provided, uses `Scope.from_user()`. Otherwise, `_find_parametrized_scope()` returns `Scope.Function` unless ALL arguments are indirect, in which case it returns the narrowest (minimum) scope among the referenced fixture definitions.\n\n4. **ID Resolution**: `_resolve_parameter_set_ids()` creates an `IdMaker` instance. The `IdMaker._resolve_ids()` method yields IDs by priority: (a) `parameterset.id` if set via `pytest.param(..., id=...)`, (b) explicit IDs list entries, (c) auto-generated by joining per-argument IDs with '-'. For auto-generation, `_idval()` tries in order: user's `idfn` callable, the `pytest_make_parametrize_id` hook, type-based conversion (`_idval_from_value` handles str/bytes, None/float/int/bool/complex, re.Pattern, enum.Enum, objects with `__name__`), and finally falls back to `<argname><idx>`. Uniqueness is enforced by `make_unique_parameterset_ids()`: duplicate IDs get a numeric suffix (with an underscore separator if the ID already ends in a digit). If `strict_parametrization_ids` config is enabled, duplicates raise `CollectError` instead.\n\n5. **Directness Resolution**: `_resolve_args_directness()` from `src/_pytest/fixtures.py` maps each argname to \"indirect\" or \"direct\". For direct parameters, `DirectParamFixtureDef` instances are created (subclass of `FixtureDef` using `get_direct_param_fixture_func` which simply returns `request.param`). These are cached on the scope node's stash for scopes higher than Function.\n\n6. **Cartesian Product of CallSpecs**: The core accumulation loop iterates over existing `self._calls` (or a single empty `CallSpec2()` if none) and for each existing callspec, creates new `CallSpec2` instances via `setmulti()` for every parameter set. `CallSpec2` is a frozen dataclass with `params` (name\u2192value dict), `indices` (name\u2192param_index), `_arg2scope`, `_idlist` (list of ID parts joined by '-' in the `id` property), and `marks`. In `setmulti()`, if `id is HIDDEN_PARAM`, the `_idlist` is kept unchanged (not appended to), effectively hiding that parameter from the test name. Duplicate argnames in `params` raise `CollectError`.\n\n7. **Test Function Generation**: After parametrization, `_recompute_direct_params_indices()` re-numbers indices for direct params sequentially. The test name is formed as `f\"{name}[{callspec.id}]\"` where `callspec.id` joins `_idlist` entries with '-'. If `_idlist` is empty (all hidden), the name has no bracket suffix.\n\n8. **Mark Caching**: The `_param_mark._param_ids_from` mechanism on the frozen `Mark` dataclass allows generated IDs to be cached and reused across multiple test functions sharing the same parametrize mark, avoiding redundant ID computation.",
    "rubric": [
      "Explains argument parsing via ParameterSet._for_parametrize(), including splitting comma-separated argnames and handling trailing comma for force_tuple behavior",
      "Describes validation: checking 'request' not in argnames, and _validate_if_using_arg_names checking argnames against fixturenames with specific error for default-value parameters",
      "Explains scope resolution: explicit scope uses Scope.from_user(); otherwise _find_parametrized_scope returns Function unless all args are indirect, in which case it returns the minimum/narrowest scope of referenced fixture definitions",
      "Describes ID resolution via IdMaker: priority order is parameterset.id, then explicit ids list, then auto-generated by joining per-arg IDs with '-'; auto-generation tries idfn, then hook, then type-based _idval_from_value, then argname+idx fallback",
      "Explains make_unique_parameterset_ids handles duplicates by appending numeric suffix (with underscore if ID ends in digit), or raises CollectError if strict_parametrization_ids is enabled",
      "Describes directness resolution via _resolve_args_directness mapping each argname to 'indirect' or 'direct', and creation of DirectParamFixtureDef instances (using get_direct_param_fixture_func that returns request.param) for direct params",
      "Explains the cartesian product loop: iterates over existing self._calls (or single empty CallSpec2) and for each creates new CallSpec2 via setmulti() for every parameter set",
      "Describes CallSpec2.setmulti behavior: copies params/indices/arg2scope, raises CollectError on duplicate argnames, and handles HIDDEN_PARAM by not appending to _idlist"
    ],
    "key_files": [
      "src/_pytest/python.py",
      "src/_pytest/mark/structures.py",
      "src/_pytest/fixtures.py",
      "src/_pytest/scope.py",
      "src/_pytest/compat.py"
    ],
    "source_doc": "[docstring: src/_pytest/python.py] src._pytest.python.Metafunc\nsrc._pytest.python.Metafunc:\n  Objects passed to the :hook:`pytest_generate_tests` hook.\n\n    They help to inspect a test function and to generate tests according to\n    test configuration or values specified in the class or module where a\n    test function is defined.\n\nsrc._pytest.python.Metafunc.parametrize:\n  Add new invocations to the underlying test function using the list\n        of argvalues for the given argnames. Parametrization is performed\n        during the collection phase. If you need to setup expensive resources\n        see about setting ``indirect`` to do it at test setup time instead.\n\n        Can be called multiple times per test function (but only on different\n        argument names), in which case each call parametrizes all previous\n        parametrizations, e.g.\n\n        ::\n\n            unparametrized:         t\n            parametrize [\"x\", \"y\"]: t[x], t[y]\n            parametrize [1, 2]:     t[x-1], t[x-2], t[y-1], t[y-2]\n\n        :param argnames:\n            A comma-separated string denoting one or more argument names, or\n            a list/tuple of argument strings.\n\n        :param argvalues:\n            The list of argvalues determines how often a test is invoked with\n            different argument values.\n\n            If only one argname was specified argvalues is a list of values.\n            If N argnames were specified, argvalues must be a list of\n            N-tuples, where each tuple-element specifies a value for its\n            respective argname.\n\n            .. versionchanged:: 9.1\n\n                Passing a non-:class:`~collections.abc.Collection` iterable\n                (such as a generator or iterator) is deprecated. See\n                :ref:`parametrize-iterators` for details.\n\n        :param indirect:\n            A list of arguments' names (subset of argnames) or a boolean.\n            If True the list contains all names from the argnames. Each\n            argvalue corresponding to an argname in this list will\n            be passed as request.param to its respective argname fixture\n            function so that it can perform more expensive setups during the\n            setup phase of a test rather than at collection time.\n\n        :param ids:\n            Sequence of (or generator for) ids for ``argvalues``,\n            or a callable to return part of the id for each argvalue.\n\n            With sequences (and generators like ``itertools.count()``) the\n            returned ids should be of type ``string``, ``int``, ``float``,\n            ``bool``, or ``None``.\n            They are mapped to the corresponding index in ``argvalues``.\n            ``None`` means to use the auto-generated id.\n\n            .. versionadded:: 8.4\n                :ref:`hidden-param` means to hide the parameter set\n                from the test name. Can only be used at most 1 time, as\n                test names need to be unique.\n\n            If it is a callable it will be called for each entry in\n            ``argvalues``, and the return value is used as part of the\n            auto-generated id for the whole set (where parts are joined with\n            dashes (\"-\")).\n            This is useful to provide more specific ids for certain items, e.g.\n            dates.  Returning ``None`` will use an auto-generated id.\n\n            If no ids are provided they will be generated automatically from\n            the argvalues.\n\n        :param scope:\n            If specified it denotes the scope of the parameters.\n            The scope is used for grouping tests by parameter instances.\n            It will also override any fixture-function defined scope, allowing\n            to set a dynamic scope using test context or configuration.\n\nsrc._pytest.python.Metafunc._resolve_parameter_set_ids:\n  Resolve the actual ids for the given parameter sets.\n\n        :param argnames:\n            Argument names passed to ``parametrize()``.\n        :param ids:\n            The `ids` parameter of the ``parametrize()`` call (see docs).\n        :param parametersets:\n            The parameter sets, each containing a set of values corresponding\n            to ``argnames``.\n        :param nodeid str:\n            The nodeid of the definition item that generated this\n            parametrization.\n        :returns:\n            List with ids for each parameter set given.\n\nsrc._pytest.python.Metafunc._validate_if_using_arg_names:\n  Check if all argnames are being used, by default values, or directly/indirectly.\n\n        :param List[str] argnames: List of argument names passed to ``parametrize()``.\n        :param indirect: Same as the ``indirect`` parameter of ``parametrize()``.\n        :raises ValueError: If validation fails.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (e.g., IdMaker, _idval, CallSpec2 frozen dataclass, DirectParamFixtureDef, _recompute_direct_params_indices, HIDDEN_PARAM, _param_mark._param_ids_from, make_unique_parameterset_ids with underscore separator, strict_parametrization_ids config) that go far beyond what the documentation provides. These appear to be plausible code-level details but cannot be verified from the given documentation alone.",
      "The claim about hidden parameters states 'If `id is HIDDEN_PARAM`, the `_idlist` is kept unchanged (not appended to), effectively hiding that parameter from the test name.' The documentation mentions hidden-param (versionadded 8.4) but doesn't describe the internal mechanism this way.",
      "The answer states that duplicate IDs get 'a numeric suffix (with an underscore separator if the ID already ends in a digit)' - this specific behavior is not documented in the provided text.",
      "The documentation says ids can be 'Sequence of (or generator for) ids' with types 'string, int, float, bool, or None', while the answer's description of ID resolution priority and auto-generation mechanisms are implementation details not directly contradicted but not confirmed either.",
      "The answer's claim about _find_parametrized_scope returning the 'narrowest (minimum) scope among the referenced fixture definitions' when all arguments are indirect is plausible but not stated in the documentation (the doc only says scope 'denotes the scope of the parameters' and 'will also override any fixture-function defined scope')."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code; no documentation-only claims found.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_33",
    "repo": "pytest",
    "question": "What are the key classes, dependencies, and behavioral mechanisms behind `pytest.raises` and its exception-matching architecture?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `pytest.raises` function in `src/_pytest/raises.py` serves as a factory that creates `RaisesExc` instances when used as a context manager, or directly invokes a callable and returns `ExceptionInfo` when used as a function. Here are the key architectural and dependency details:\n\n**Core class hierarchy:**\n- `AbstractRaises` (ABC, Generic[BaseExcT_co]) is the abstract base class shared between `RaisesExc` and `RaisesGroup`.\n- `RaisesExc` (decorated with `@final`) inherits from `AbstractRaises` and handles single exception matching.\n- `RaisesGroup` (decorated with `@final`) inherits from `AbstractRaises[BaseExceptionGroup[BaseExcT_co]]` and handles exception group matching.\n- Both classes implement the abstract method `matches()` defined in `AbstractRaises`.\n\n**Key implementation details:**\n\n1. **`_just_propagate` flag**: When `RaisesExc.matches()` is called, if the type check fails (via `_check_type`), `_just_propagate` is set to `True`. In `__exit__`, when `_just_propagate` is True, the method returns `False` (letting the exception propagate unchanged) rather than raising an `AssertionError`. This means type mismatches cause the original exception to bubble up, while match/check failures raise `AssertionError`.\n\n2. **`rawmatch` optimization**: `AbstractRaises.__init__` detects if the match pattern is a \"fully escaped\" regex anchored with `^...$`. If so, it stores the unescaped literal string in `self.rawmatch`. When matching fails and `rawmatch` is set, `_check_match` uses `_diff_text` from `_pytest.assertion.util` (with `dummy_highlighter`) to produce a diff-style error message instead of the generic regex mismatch message.\n\n3. **`raises.Exception` attribute**: `raises.Exception = fail.Exception`, which is the `Failed` class from `_pytest.outcomes`. `Failed` inherits from `OutcomeException` which inherits from `BaseException` (not `Exception`). The `_Fail` class's `__call__` method raises `Failed(msg=reason, pytrace=pytrace)`.\n\n4. **`stringify_exception` function** (in `src/_pytest/_code/code.py`): Used by `AbstractRaises._check_match`. It concatenates `str(exc)` with any `__notes__`. For `BaseExceptionGroup` instances when `include_subexception_msg=False`, it uses `exc.message` instead of `str(exc)` \u2014 this is specifically done in `_check_match` to match only the group's own message.\n\n5. **Empty match warning**: Passing `match=\"\"` triggers a `PytestWarning` saying \"matching against an empty string will *always* pass\".\n\n6. **Regex compilation**: String match patterns are compiled with `re.compile()` during `__init__`. If the regex is invalid, the error is caught and re-raised via `fail()` (raising `Failed`).\n\n7. **Integration with xfail**: In `_pytest/skipping.py`, `AbstractRaises` instances can be passed as the `raises` parameter to `@pytest.mark.xfail`. The skipping plugin checks `isinstance(raises, AbstractRaises)` and calls `raises.matches(call.excinfo.value)` to determine if the xfailed test raised the expected exception.\n\n8. **Context manager return**: `RaisesExc.__enter__` creates an `ExceptionInfo.for_later()` (unfilled), and `__exit__` fills it with `excinfo.fill_unfilled(exc_info)` after a successful match. `ExceptionInfo.for_later()` creates an instance with `_excinfo=None`.",
    "rubric": [
      "Identifies `AbstractRaises` as an ABC (Generic) base class shared by `RaisesExc` and `RaisesGroup`, both decorated with `@final`",
      "Explains that `RaisesGroup` inherits from `AbstractRaises[BaseExceptionGroup[BaseExcT_co]]` while `RaisesExc` inherits from `AbstractRaises[BaseExcT_co_default]`, and both implement the abstract `matches()` method",
      "Describes the `_just_propagate` flag: set to `True` in `RaisesExc.matches()` when type check fails, causing `__exit__` to return `False` (propagate exception) rather than raising `AssertionError`",
      "Explains `rawmatch` optimization: detects fully-escaped anchored regex patterns (`^...$`), stores unescaped literal, and uses `_diff_text` from `_pytest.assertion.util` for diff-style error messages on mismatch",
      "Notes that `raises.Exception` is assigned `fail.Exception`, which is the `Failed` class inheriting from `OutcomeException(BaseException)`",
      "Describes `stringify_exception` in `_pytest/_code/code.py`: concatenates `str(exc)` with `__notes__`, using `exc.message` instead of `str(exc)` for `BaseExceptionGroup` when `include_subexception_msg=False`",
      "Mentions that `match=\"\"` triggers a `PytestWarning` about matching against an empty string always passing",
      "Explains integration with xfail: `_pytest/skipping.py` checks `isinstance(raises, AbstractRaises)` and calls `raises.matches()` to determine if the xfailed test raised the expected exception",
      "Describes context manager behavior: `__enter__` creates `ExceptionInfo.for_later()` and `__exit__` fills it via `fill_unfilled()` after successful match"
    ],
    "key_files": [
      "src/_pytest/raises.py",
      "src/_pytest/_code/code.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/skipping.py"
    ],
    "source_doc": "[docstring: src/_pytest/raises.py] src._pytest.raises.raises\nAssert that a code block/function call raises an exception type, or one of its subclasses.\n\n    :param expected_exception:\n        The expected exception type, or a tuple if one of multiple possible\n        exception types are expected. Note that subclasses of the passed exceptions\n        will also match.\n\n        This is not a required parameter, you may opt to only use ``match`` and/or\n        ``check`` for verifying the raised exception.\n\n    :kwparam str | re.Pattern[str] | None match:\n        If specified, a string containing a regular expression,\n        or a regular expression object, that is tested against the string\n        representation of the exception and its :pep:`678` `__notes__`\n        using :func:`re.search`.\n\n        To match a literal string that may contain :ref:`special characters\n        <re-syntax>`, the pattern can first be escaped with :func:`re.escape`.\n\n        (This is only used when ``pytest.raises`` is used as a context manager,\n        and passed through to the function otherwise.\n        When using ``pytest.raises`` as a function, you can use:\n        ``pytest.raises(Exc, func, match=\"passed on\").match(\"my pattern\")``.)\n\n    :kwparam Callable[[BaseException], bool] check:\n\n        .. versionadded:: 8.4\n\n        If specified, a callable that will be called with the exception as a parameter\n        after checking the type and the match regex if specified.\n        If it returns ``True`` it will be considered a match, if not it will\n        be considered a failed match.\n\n\n    Use ``pytest.raises`` as a context manager, which will capture the exception of the given\n    type, or any of its subclasses::\n\n        >>> import pytest\n        >>> with pytest.raises(ZeroDivisionError):\n        ...    1/0\n\n    If the code block does not raise the expected exception (:class:`ZeroDivisionError` in the example\n    above), or no exception at all, the check will fail instead.\n\n    You can also use the keyword argument ``match`` to assert that the\n    exception matches a text or regex::\n\n        >>> with pytest.raises(ValueError, match='must be 0 or None'):\n        ...     raise ValueError(\"value must be 0 or None\")\n\n        >>> with pytest.raises(ValueError, match=r'must be \\d+$'):\n        ...     raise ValueError(\"value must be 42\")\n\n    The ``match`` argument searches the formatted exception string, which includes any\n    `PEP-678 <https://peps.python.org/pep-0678/>`__ ``__notes__``:\n\n        >>> with pytest.raises(ValueError, match=r\"had a note added\"):  # doctest: +SKIP\n        ...     e = ValueError(\"value must be 42\")\n        ...     e.add_note(\"had a note added\")\n        ...     raise e\n\n    The ``check`` argument, if provided, must return True when passed the raised exception\n    for the match to be successful, otherwise an :exc:`AssertionError` is raised.\n\n        >>> import errno\n        >>> with pytest.raises(OSError, check=lambda e: e.errno == errno.EACCES):\n        ...     raise OSError(errno.EACCES, \"no permission to view\")\n\n    The context manager produces an :class:`ExceptionInfo` object which can be used to inspect the\n    details of the captured exception::\n\n        >>> with pytest.raises(ValueError) as exc_info:\n        ...     raise ValueError(\"value must be 42\")\n        >>> assert exc_info.type is ValueError\n        >>> assert exc_info.value.args[0] == \"value must be 42\"\n\n    .. warning::\n\n       Given that ``pytest.raises`` matches subclasses, be wary of using it to match :class:`Exception` like this::\n\n           # Careful, this will catch ANY exception raised.\n           with pytest.raises(Exception):\n               some_function()\n\n       Because :class:`Exception` is the base class of almost all exceptions, it is easy for this to hide\n       real bugs, where the user wrote this expecting a specific exception, but some other exception is being\n       raised due to a bug introduced during a refactoring.\n\n       Avoid using ``pytest.raises`` to catch :class:`Exception` unless certain that you really want to catch\n       **any** exception raised.\n\n    .. note::\n\n       When using ``pytest.raises`` as a context manager, it's worthwhile to\n       note that normal context manager rules apply and that the exception\n       raised *must* be the final line in the scope of the context manager.\n       Lines of code after that, within the scope of the context manager will\n       not be executed. For example::\n\n           >>> value = 15\n           >>> with pytest.raises(ValueError) as exc_info:\n           ...     if value > 10:\n           ...         raise ValueError(\"value must be <= 10\")\n           ...     assert exc_info.type is ValueError  # This will not execute.\n\n       Instead, the following approach must be taken (note the difference in\n       scope)::\n\n           >>> with pytest.raises(ValueError) as exc_info:\n           ...     if value > 10:\n           ...         raise ValueError(\"value must be <= 10\")\n           ...\n           >>> assert exc_info.type is ValueError\n\n    **Expecting exception groups**\n\n    When expecting exceptions wrapped in :exc:`BaseExceptionGroup` or\n    :exc:`ExceptionGroup`, you should instead use :class:`pytest.RaisesGroup`.\n\n    **Using with** ``pytest.mark.parametrize``\n\n    When using :ref:`pytest.mark.parametrize ref`\n    it is possible to parametrize tests such that\n    some runs raise an exception and others do not.\n\n    See :ref:`parametrizing_conditional_raising` for an example.\n\n    .. seealso::\n\n        :ref:`assertraises` for more examples and detailed discussion.\n\n    .. note::\n        Similar to caught exception objects in Python, explicitly clearing\n        local references to returned ``ExceptionInfo`` objects can\n        help the Python interpreter speed up its garbage collection.\n\n        Clearing those references breaks a reference cycle\n        (``ExceptionInfo`` --> caught exception --> frame stack raising\n        the exception --> current frame stack --> local variables -->\n        ``ExceptionInfo``) which makes Python keep all objects referenced\n        from that cycle (including all local variables in the current\n        frame) alive until the next cyclic garbage collection run.\n        More detailed information can be found in the official Python\n        documentation for :ref:`the try statement <python:try>`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims `pytest.raises` 'creates RaisesExc instances when used as a context manager' \u2014 the documentation does not mention `RaisesExc` at all; it describes the context manager as producing an `ExceptionInfo` object. This could be an implementation detail but potentially contradicts the documented behavior if `RaisesExc` is presented as what's returned to the user.",
      "The answer describes `AbstractRaises` as an ABC shared between `RaisesExc` and `RaisesGroup`, and provides extensive implementation details (e.g., `_just_propagate`, `rawmatch`, `stringify_exception`, `ResultHolder`, `possible_match`) that are not verifiable from the provided documentation. These may be accurate code-level details but cannot be confirmed from the doc alone.",
      "The answer states 'raises.Exception = fail.Exception, which is the Failed class from _pytest.outcomes' \u2014 this is an implementation detail not mentioned in the documentation and cannot be verified from it.",
      "The answer claims type mismatches cause the original exception to bubble up while match/check failures raise AssertionError \u2014 the documentation says 'If the code block does not raise the expected exception... the check will fail instead' and for `check`: 'otherwise an AssertionError is raised'. The documentation doesn't explicitly distinguish between type mismatch propagation vs AssertionError for match/check failures, so this claim is plausible but unverifiable.",
      "The documentation says `expected_exception` is 'not a required parameter' \u2014 the answer doesn't contradict this but frames `RaisesExc` as handling single exception matching without mentioning optionality.",
      "The answer mentions `ExceptionInfo.for_later()` and filling with `fill_unfilled` \u2014 the documentation mentions `ExceptionInfo` is produced by the context manager but doesn't describe these internal methods."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; minor removals for truncated code references and doc-only rationale phrases.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_34",
    "repo": "pytest",
    "question": "Why does WarningsChecker.__exit__ use a nested found_str() closure and split its any() checks into two separate passes?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `WarningsChecker.__exit__` method in `src/_pytest/recwarn.py` employs several performance optimizations:\n\n1. **Lazy evaluation of error messages via `found_str()` closure**: The `found_str()` function wraps `pformat([record.message for record in self], indent=2)` in a local function that is only called when a failure condition is actually triggered. This avoids the expensive `pformat` call on the success path (the common case).\n\n2. **Short-circuit evaluation with `any()` and generator expressions**: The validation uses `any()` with generator expressions that short-circuit on the first matching element. The first check is only `issubclass(w.category, self.expected_warning)` (cheap class hierarchy check). Only if that succeeds does it proceed to the second check which invokes `self.matches(w)` \u2014 which additionally runs `re.search()` (more expensive). This two-level approach avoids regex evaluation entirely when no warnings of the correct category were emitted at all.\n\n3. **Short-circuit `and` in the `matches()` method**: Inside `matches()`, the `issubclass` check comes before `re.search` using Python's `and` short-circuit semantics, so regex matching is skipped for warnings of the wrong category. Additionally, `self.match_expr is None` is checked first (an O(1) identity comparison) before `re.search` is invoked.\n\n4. **Early return for BaseExceptions**: The method immediately returns without doing any warning validation if a BaseException (that is not a regular Exception) or Exit was raised, avoiding unnecessary list iteration on such exceptions.\n\n5. **`warn_explicit` used for re-emission**: Unmatched warnings are re-emitted using `warnings.warn_explicit()` rather than `warnings.warn()` to preserve the original `filename`, `lineno`, `module`, and `source` metadata. This avoids the overhead of stack frame inspection that `warnings.warn()` would perform to determine the caller location.\n\n6. **`pop()` method early return on exact match**: In `WarningsRecorder.pop()`, an exact `w.category == cls` comparison is done first, allowing an immediate return that bypasses continuing iteration through the remaining warnings list.\n\n7. **`simplefilter('always')` in `__enter__`**: The `WarningsRecorder` sets the filter to 'always' upon entry. This is done once at context entry rather than per-warning, keeping the per-warning overhead minimal.\n\nThe `WarningsRecorder` inherits from `warnings.catch_warnings` (with `record=True` passed to super().__init__), meaning the standard library handles the low-level warning interception, filter saving/restoration, and list population. The `WarningsChecker` is marked `@final`.",
    "rubric": [
      "Explains that found_str() wraps pformat in a closure so the expensive formatting is only evaluated when a failure actually occurs (lazy evaluation)",
      "Explains that the first any() check uses only issubclass (cheap category check), short-circuiting before the second any() which additionally calls re.search via matches()",
      "Mentions that matches() itself short-circuits: issubclass is checked before re.search, and match_expr is None is checked before re.search",
      "Notes the early return when a BaseException (non-Exception) or Exit is raised, skipping all warning validation",
      "Explains that warn_explicit is used instead of warn to re-emit unmatched warnings, preserving original metadata and avoiding stack frame inspection overhead",
      "Mentions WarningsRecorder.pop() does an exact equality check (w.category == cls) for early return before falling through to issubclass matching"
    ],
    "key_files": [
      "src/_pytest/recwarn.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/deprecated.py"
    ],
    "source_doc": "[docstring: src/_pytest/recwarn.py] src._pytest.recwarn.warns\nAssert that code raises a particular class of warning.\n\n    Specifically, the parameter ``expected_warning`` can be a warning class or tuple\n    of warning classes, and the code inside the ``with`` block must issue at least one\n    warning of that class or classes.\n\n    This helper produces a list of :class:`warnings.WarningMessage` objects, one for\n    each warning emitted (regardless of whether it is an ``expected_warning`` or not).\n    Since pytest 8.0, unmatched warnings are also re-emitted when the context closes.\n\n    This function should be used as a context manager::\n\n        >>> import pytest\n        >>> with pytest.warns(RuntimeWarning):\n        ...    warnings.warn(\"my warning\", RuntimeWarning)\n\n    The ``match`` keyword argument can be used to assert\n    that the warning matches a text or regex::\n\n        >>> with pytest.warns(UserWarning, match='must be 0 or None'):\n        ...     warnings.warn(\"value must be 0 or None\", UserWarning)\n\n        >>> with pytest.warns(UserWarning, match=r'must be \\d+$'):\n        ...     warnings.warn(\"value must be 42\", UserWarning)\n\n        >>> with pytest.warns(UserWarning):  # catch re-emitted warning\n        ...     with pytest.warns(UserWarning, match=r'must be \\d+$'):\n        ...         warnings.warn(\"this is not here\", UserWarning)\n        Traceback (most recent call last):\n          ...\n        Failed: Regex pattern did not match any of the 1 warnings emitted.\n         Regex: ...\n         Emitted warnings: ...UserWarning...\n\n    **Using with** ``pytest.mark.parametrize``\n\n    When using :ref:`pytest.mark.parametrize ref` it is possible to parametrize tests\n    such that some runs raise a warning and others do not.\n\n    This could be achieved in the same way as with exceptions, see\n    :ref:`parametrizing_conditional_raising` for an example.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses detailed implementation internals (specific line numbers, method implementations, performance optimizations) that are not present in the provided documentation chunk. The documentation only describes the public API and behavior of pytest.warns.",
      "Claims about specific line numbers (e.g., line 319, 323, 328, 294, 312-317, 345, 218-219, 244) cannot be verified from the provided documentation.",
      "The claim about unmatched warnings being re-emitted is consistent with the documentation statement 'Since pytest 8.0, unmatched warnings are also re-emitted when the context closes', though the implementation detail about warn_explicit vs warn cannot be verified.",
      "The answer appears to be about implementation details that go far beyond what the documentation describes, making most claims neither supported nor contradicted - they are plausible code-level extrapolations but unverifiable from this doc alone."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for interpretive rationale about @final's purpose and the specific exceptions being handled.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_35",
    "repo": "pytest",
    "question": "Where is the Scope enum defined and where is it used across the codebase for fixture and parametrize scope resolution?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Scope` enum class is defined in `src/_pytest/scope.py` and is used throughout the pytest codebase to represent fixture scopes. Here's where it's located and how it connects to the broader system:\n\n**Location**: `src/_pytest/scope.py`\n\n**Key design details**:\n- `Scope` is decorated with `@total_ordering` from `functools`, which means only `__lt__` needs to be defined and the rest of the comparison operators are derived.\n- The `__lt__` method uses `_SCOPE_INDICES`, a module-level dict mapping each `Scope` enum member to its integer index (built from `_ALL_SCOPES = list(Scope)`).\n- `ScopeName` is defined as `Literal[\"session\", \"package\", \"module\", \"class\", \"function\"]`.\n- `HIGH_SCOPES` is defined as `[x for x in Scope if x is not Scope.Function]`.\n\n**Where `Scope.from_user` is called**:\n1. In `src/_pytest/fixtures.py` (line 1016) within `FixtureDef.__init__`, where a string scope from a fixture definition is converted via `Scope.from_user(scope, descr=f\"Fixture '{func.__name__}'\", where=baseid)`.\n2. In `src/_pytest/python.py` (line 1329) within the `Metafunc._parametrize` method, where a scope name given to `@pytest.mark.parametrize` is converted via `Scope.from_user(scope, descr=f\"parametrize() call in {self.function.__name__}\")`.\n\n**Where `next_lower` is called**:\n- In `src/_pytest/fixtures.py` line 295, `reorder_items_atscope` recursively calls itself with `scope.next_lower()` to reorder test items at progressively lower scopes.\n\n**Where `Scope` is used for node resolution**:\n- `get_scope_node` in `src/_pytest/fixtures.py` (line 125) maps each `Scope` value to a specific node class: `Scope.Function` \u2192 `nodes.Item`, `Scope.Class` \u2192 `_pytest.python.Class`, `Scope.Module` \u2192 `_pytest.python.Module`, `Scope.Package` \u2192 `_pytest.python.Package`, `Scope.Session` \u2192 `_pytest.main.Session`.\n\n**Dynamic scope support**:\n- In `FixtureDef.__init__`, if `scope` is a callable, it's evaluated via `_eval_scope_callable(scope, argname, config)` which calls the callable with `fixture_name` and `config` keyword arguments, expecting it to return a string scope name.",
    "rubric": [
      "Identifies that Scope is defined in src/_pytest/scope.py",
      "Mentions that Scope uses @total_ordering decorator from functools",
      "Explains that __lt__ uses _SCOPE_INDICES dict mapping enum members to integer indices",
      "Notes that Scope.from_user is called in FixtureDef.__init__ in src/_pytest/fixtures.py to convert string scope",
      "Notes that Scope.from_user is called in Metafunc._parametrize in src/_pytest/python.py for parametrize scope",
      "Mentions that next_lower is used in reorder_items_atscope for recursive item reordering",
      "Describes get_scope_node mapping Scope values to node classes (Item, Class, Module, Package, Session)",
      "Mentions dynamic scope support via callable evaluated in _eval_scope_callable"
    ],
    "key_files": [
      "src/_pytest/scope.py",
      "src/_pytest/fixtures.py",
      "src/_pytest/python.py",
      "testing/test_scope.py"
    ],
    "source_doc": "[docstring: src/_pytest/scope.py] src._pytest.scope.Scope\nsrc._pytest.scope.Scope:\n  Represents one of the possible fixture scopes in pytest.\n\n    Scopes are ordered from lower to higher, that is:\n\n              ->>> higher ->>>\n\n    Function < Class < Module < Package < Session\n\n              <<<- lower  <<<-\n\nsrc._pytest.scope.Scope.next_lower:\n  Return the next lower scope.\n\nsrc._pytest.scope.Scope.next_higher:\n  Return the next higher scope.\n\nsrc._pytest.scope.Scope.from_user:\n  Given a scope name from the user, return the equivalent Scope enum. Should be used\n        whenever we want to convert a user provided scope name to its enum object.\n\n        If the scope name is invalid, construct a user friendly message and call pytest.fail.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.14,
    "strip_verify_summary": "Minor doc leakage: module docstring rationale about circular references and a comment about 'optimal fixture reuse' are not verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_36",
    "repo": "pytest",
    "question": "How does pytest collect, execute, and report results for unittest.TestCase subclasses?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "Pytest supports unittest-style tests through a dedicated plugin module (`src/_pytest/unittest.py`) that integrates stdlib `unittest.TestCase` subclasses into the pytest collection, execution, and reporting framework. Here's how it works:\n\n**Collection/Discovery:**\n1. The `pytest_pycollect_makeitem` hook in `src/_pytest/unittest.py` checks if a collected class is a subclass of `unittest.TestCase` by looking up `sys.modules[\"unittest\"]`. It also skips abstract classes (`inspect.isabstract(obj)`).\n2. Matching classes are wrapped in `UnitTestCase` (which extends `Class` from `src/_pytest/python.py`).\n3. `UnitTestCase.collect()` uses `unittest.TestLoader().getTestCaseNames()` to discover test methods, then yields `TestCaseFunction` items for each test. It also handles Twisted's `trial.unittest` by checking if `runTest` differs from `ut.TestCase.runTest`.\n\n**Fixture Integration:**\n- `UnitTestCase` sets `nofuncargs = True`, which prevents pytest's funcarg mechanism from being applied to unittest tests.\n- `setUpClass`/`tearDownClass` are registered as autouse class-scoped fixtures via `_register_unittest_setup_class_fixture()`, with unique names like `_unittest_setUpClass_fixture_{cls.__qualname__}`.\n- `setup_method`/`teardown_method` are registered as autouse function-scoped fixtures via `_register_unittest_setup_method_fixture()`.\n\n**Test Execution:**\n- `TestCaseFunction` extends `Function` (from `src/_pytest/python.py`) and acts as both a pytest Item AND a unittest TestResult object.\n- In `runtest()`, it calls `testcase(result=self)`, passing itself as the result \u2014 the unittest framework then calls `addError()`, `addFailure()`, `addSkip()`, `addSuccess()`, `addExpectedFailure()`, `addUnexpectedSuccess()`, and `addSubTest()` on the `TestCaseFunction` instance.\n- When `--pdb` is used and the test isn't skipped, `tearDown` is replaced with a no-op lambda and the original is stored in `self._explicit_tearDown` to be called later during pytest's teardown phase.\n\n**Assertion Rewriting:**\n- Pytest's assertion rewriting works via `AssertionRewritingHook` (an import hook in `src/_pytest/assertion/rewrite.py` implementing `importlib.abc.MetaPathFinder` and `importlib.abc.Loader`) that intercepts module imports and rewrites `assert` statements at the AST level.\n- The `AssertionRewriter` class processes AST modules via the `rewrite_asserts` function to provide detailed assertion failure information.\n- The hook is installed into `sys.meta_path` by `install_importhook()` in `src/_pytest/assertion/__init__.py`.\n\n**Traceback Filtering:**\n- `TestCaseFunction._traceback_filter()` filters out frames from unittest internals (frames where `__unittest` is set in `f_globals`) for cleaner error reporting.\n\n**Subtest Support:**\n- `addSubTest()` creates `SubtestReport` objects (from `src/_pytest/subtests.py`) using `SubtestContext` for each subtest, then invokes `pytest_runtest_logreport` for proper reporting.",
    "rubric": [
      "Explains that pytest_pycollect_makeitem checks if a class is a unittest.TestCase subclass (via sys.modules['unittest']) and skips abstract classes",
      "Mentions that matching classes are wrapped in UnitTestCase (which extends Class) and uses unittest.TestLoader().getTestCaseNames() to discover test methods, yielding TestCaseFunction items",
      "Explains that UnitTestCase sets nofuncargs=True to prevent pytest's funcarg mechanism from applying",
      "Describes how setUpClass/tearDownClass are registered as autouse class-scoped fixtures and setup_method/teardown_method as autouse function-scoped fixtures",
      "Explains that TestCaseFunction extends Function and acts as a unittest TestResult, with runtest() calling testcase(result=self) so unittest calls addError/addFailure/addSkip/addSuccess etc. on the TestCaseFunction instance",
      "Mentions the --pdb handling where tearDown is replaced with a no-op and stored in _explicit_tearDown for later execution",
      "Describes that _traceback_filter removes unittest internal frames (where __unittest is set in f_globals)",
      "Mentions assertion rewriting via AssertionRewritingHook installed in sys.meta_path that rewrites assert statements at AST level"
    ],
    "key_files": [
      "src/_pytest/unittest.py",
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/assertion/__init__.py",
      "src/_pytest/python.py",
      "src/_pytest/subtests.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.0.rst] py.test 2.0.0: asserts++, unittest++, reporting++, config++, docs++\nWelcome to pytest-2.0.0, a major new release of \"py.test\", the rapid\neasy Python testing tool.  There are many new features and enhancements,\nsee below for summary and detailed lists.  A lot of long-deprecated code\nhas been removed, resulting in a much smaller and cleaner\nimplementation.  See the new docs with examples here:\n\n    http://pytest.org/en/stable/index.html\n\nA note on packaging: pytest used to part of the \"py\" distribution up\nuntil version py-1.3.4 but this has changed now:  pytest-2.0.0 only\ncontains py.test related code and is expected to be backward-compatible\nto existing test code. If you want to install pytest, just type one of::\n\n    pip install -U pytest\n    easy_install -U pytest\n\nMany thanks to all issue reporters and people asking questions or\ncomplaining.  Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt\nfor their great coding contributions and many others for feedback and help.\n\nbest,\nholger krekel",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the @py_assert naming convention claim and some documentation-style characterizations (PEP references, specific line numbers) needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_37",
    "repo": "pytest",
    "question": "What are the key architectural pieces behind pytest's PDB integration, IO capture during conftest loading, conftest discovery, and pytestmark collection on classes?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "Here's the architecture of the key implementations in the current codebase:\n\n**`pytest.set_trace()` (PDB during capture)**\n\nThe `pytest.set_trace` function is exposed in `src/pytest/__init__.py` as a reference to `pytestPDB.set_trace`, where `pytestPDB` is imported from `_pytest.debugging`. The `pytestPDB` class (in `src/_pytest/debugging.py`) has the following key architecture:\n\n1. During `pytest_configure`, `pdb.set_trace` is monkey-patched to `pytestPDB.set_trace`, with the original saved in `pytestPDB._saved` (a list of tuples containing the original `pdb.set_trace`, pluginmanager, and config).\n2. `pytestPDB.set_trace()` gets the caller's frame via `sys._getframe().f_back`, calls `_init_pdb(\"set_trace\")`, and then calls `_pdb.set_trace(frame)`.\n3. `_init_pdb()` retrieves the `CaptureManager` plugin via `cls._pluginmanager.getplugin(\"capturemanager\")` and calls `capman.suspend(in_=True)` to drop IO capturing. It also prints a header like \"PDB set_trace (IO-capturing turned off)\" when global capturing is active.\n4. A `PytestPdbWrapper` class is dynamically created that wraps the actual `pdb.Pdb` class, adding `do_continue` that resumes capturing via `capman.resume()`, and `do_quit` that calls `outcomes.exit(\"Quitting debugger\")`.\n5. The `pytest_enter_pdb` hook is called when entering pdb, and `pytest_leave_pdb` when leaving (on continue).\n6. Cleanup restores the original `pdb.set_trace` via `config.add_cleanup(fin)`.\n\n**Capturing during conftest import**\n\nIn `src/_pytest/capture.py`, the `pytest_load_initial_conftests` hook wrapper starts global capturing *before* yielding to allow conftest loading to proceed. If an exception occurs, captured stdout/stderr is written to the real streams.\n\n**Sibling conftests not loaded**\n\nThe `_loadconftestmodules` method in `src/_pytest/config/__init__.py` is called from `_set_initial_conftests` to load conftest files for target directories.\n\n**Pytestmark on classes**\n\nThe `get_unpacked_marks` function in `src/_pytest/mark/structures.py` handles pytestmark at both class and module levels. For classes, it traverses `obj.__mro__` to collect marks from all parent classes. For non-class objects (like functions), it uses `getattr(obj, \"pytestmark\", [])` to collect marks.",
    "rubric": [
      "Explains that pdb.set_trace is monkey-patched to pytestPDB.set_trace during pytest_configure, with the original saved in pytestPDB._saved for later restoration via config.add_cleanup",
      "Describes that pytestPDB.set_trace gets the caller's frame via sys._getframe().f_back, calls _init_pdb which suspends capturing via capman.suspend(in_=True), and prints a header indicating IO-capturing status",
      "Explains the dynamically created PytestPdbWrapper class that wraps pdb.Pdb, overriding do_continue (resumes capture via capman.resume), do_quit (calls outcomes.exit), and do_debug (tracks recursive debug level)",
      "Mentions the pytest_enter_pdb and pytest_leave_pdb hooks called when entering/leaving the debugger",
      "Describes that pytest_load_initial_conftests in capture.py is a wrapper hook that starts global capturing before yielding (conftest loading), then writes captured output to real streams on exception",
      "Explains that _loadconftestmodules walks parent directories of anchor paths to discover and load conftest.py files, using _is_in_confcutdir to filter",
      "Describes that get_unpacked_marks traverses obj.__mro__ in reversed order for classes (when consider_mro=True), or uses getattr(obj, 'pytestmark', []) for non-class objects"
    ],
    "key_files": [
      "src/_pytest/debugging.py",
      "src/_pytest/capture.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/mark/structures.py",
      "src/pytest/__init__.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.0.rst] Fixes\n- fix issue126 - introduce py.test.set_trace() to trace execution via\n  PDB during the running of tests even if capturing is ongoing.\n- fix issue124 - make reporting more resilient against tests opening\n  files on filedescriptor 1 (stdout).\n- fix issue109 - sibling conftest.py files will not be loaded.\n  (and Directory collectors cannot be customized anymore from a Directory's\n  conftest.py - this needs to happen at least one level up).\n- fix issue88 (finding custom test nodes from command line arg)\n- fix issue93 stdout/stderr is captured while importing conftest.py\n- fix bug: unittest collected functions now also can have \"pytestmark\"\n  applied at class/module level",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.26,
    "strip_verify_summary": "Several claims relied on stripped comments/docstrings, release notes, truncated code, and test files not shown; these were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_38",
    "repo": "pytest",
    "question": "Why does pytest implement so many defensive mechanisms around early initialization \u2014 like capturing during conftest loading, two-pass plugin blocking, and detecting `python -O`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The following design decisions are visible in the current codebase, addressing early capturing, plugin blocking, xfail+setup/teardown interaction, and python -O/-OO safety:\n\n**1. Early Capture Design (initial capturing):**\nIn `src/_pytest/capture.py`, the `pytest_load_initial_conftests` hook is implemented as a `@hookimpl(wrapper=True)` that starts global capturing *before* conftest loading occurs (via `yield`), and suspends it after. If an exception occurs during conftest loading, captured output is flushed to stdout/stderr before re-raising, preventing lost output.\n\n**2. Plugin Blocking via `-p no:X` (preventing/unregistering plugins from command line):**\nThe `consider_preparse` method in `PytestPluginManager` (config/__init__.py) is called **twice** with different `exclude_only` settings:\n- First in `get_config()` with `exclude_only=True` \u2014 this processes `-p no:X` arguments *before* loading `default_plugins`.\n- Then again with `exclude_only=False` \u2014 this handles both blocking and loading of custom plugins.\n\nThe `consider_pluginarg` method implements safeguards: `essential_plugins` (\"mark\", \"main\", \"runner\", \"fixtures\", \"helpconfig\") cannot be disabled and raise `UsageError` if attempted. Blocking `cacheprovider` also automatically blocks `stepwise`. It also blocks both `name` and `pytest_name` variants.\n\n**3. xfail + setup/teardown:**\nIn `src/_pytest/skipping.py`, the `pytest_runtest_makereport` hook has a critical `elif call.when == \"call\"` check. This means:\n- If setup/teardown *fails* on an xfail-marked test, it enters the `if call.excinfo:` branch and gets marked `outcome=\"skipped\"` with `wasxfail` \u2014 reporting as xfail.\n- If setup/teardown *succeeds* on an xfail test, the `call.when` is \"setup\" or \"teardown\" (not \"call\"), so neither the strict-xpass nor normal-xpass branches execute \u2014 the report stays as normal pass.\n- Only during the \"call\" phase can xpass (strict or not) be detected.\n\n**4. python -O/-OO safety:**\nThe `_assertion_supported()` function (config/__init__.py) literally executes `assert False` in a try/except. If Python is running with `-O` (assertions disabled), it returns `False`. The `_warn_about_missing_assertion` method then issues a `PytestConfigWarning` with different messages depending on whether the mode is \"plain\" (warning about false positives: \"ASSERTIONS ARE NOT EXECUTED and FAILING TESTS WILL PASS\") or \"rewrite\" (warning that non-rewritten assertions will be ignored). The `PYC_EXT` in assertion/rewrite.py uses `__debug__` to determine `.pyc` vs `.pyo` extension.\n\n**5. \"not in\" assertion improvement:**\nThe `_notin_text` function in `src/_pytest/assertion/util.py` provides detailed assertion messages for `\"not in\"` operations. It finds the term's position in the text, constructs a \"correct\" text without the term, then diffs them to show where the unexpected substring is located.\n\n**6. Plugin version display (--traceconfig/--version):**\nThe `getpluginversioninfo` function in `src/_pytest/helpconfig.py` calls `config.pluginmanager.list_plugin_distinfo()` to get registered third-party plugins with their distribution info, displaying `project_name-version at location`. The `pytest_report_header` hook shows this info when `--debug` or `--traceconfig` is used, and additionally lists all active plugins with their file paths when `--traceconfig` is specified.",
    "rubric": [
      "Explains that early capture (pytest_load_initial_conftests in capture.py) starts global capturing before conftest loading via yield, and flushes captured output to stdout/stderr on exception to prevent lost output",
      "Explains that consider_preparse is called twice: first with exclude_only=True to block plugins before default_plugins are loaded, then with exclude_only=False to handle both blocking and loading of custom plugins",
      "Identifies essential_plugins (mark, main, runner, fixtures, helpconfig) that cannot be disabled, raising UsageError if attempted",
      "Mentions that blocking cacheprovider also automatically blocks stepwise, and that both name and pytest_name variants are blocked",
      "Explains how xfail interacts with setup/teardown: only during call.when == 'call' can xpass be detected; setup/teardown failures on xfail tests get marked as skipped/xfail",
      "Describes _assertion_supported() which literally executes 'assert False' in try/except to detect python -O, and the different warning messages for 'plain' vs 'rewrite' assertion modes",
      "Mentions PYC_EXT using __debug__ to determine .pyc vs .pyo extension for assertion rewriting safety"
    ],
    "key_files": [
      "src/_pytest/capture.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/skipping.py",
      "src/_pytest/helpconfig.py",
      "src/_pytest/assertion/util.py",
      "doc/en/announce/release-2.0.1.rst"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.1.rst] py.test 2.0.1: bug fixes\nWelcome to pytest-2.0.1, a maintenance and bug fix release of pytest,\na mature testing tool for Python, supporting CPython 2.4-3.2, Jython\nand latest PyPy interpreters.  See extensive docs with tested examples here:\n\n    http://pytest.org/\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nMany thanks to all issue reporters and people asking questions or\ncomplaining.  Particular thanks to Floris Bruynooghe and Ronny Pfannschmidt\nfor their great coding contributions and many others for feedback and help.\n\nbest,\nholger krekel",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive detailed claims about specific source code files, line numbers, function names, and implementation details that are not present in or supported by the provided documentation chunk.",
      "The documentation only mentions pytest 2.0.1 as a 'maintenance and bug fix release' with thanks to specific contributors, but does not describe any of the specific design decisions, code locations, or implementation details claimed in the answer.",
      "The answer attributes specific contributions (e.g., 'not in assertion improvement' to Floris Bruynooghe) that are plausible given the documentation's thanks to him, but are not explicitly stated in the doc.",
      "The answer appears to be based on actual pytest source code rather than the provided documentation, making it an extrapolation beyond the doc's scope rather than a contradiction."
    ],
    "strip_verify_leakage": 0.22,
    "strip_verify_summary": "Several claims rely on release notes for historical context (issue numbers, attributions, design rationale about logging module), while the majority of technical claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_39",
    "repo": "pytest",
    "question": "Where does data flow through early capture setup, plugin version display, plugin blocking, and xfail handling for setup/teardown phases?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data/control flow for the features mentioned in the release 2.0.1 changelog travels through several key paths in the pytest codebase:\n\n**Initial Capturing Flow (issue about logging module on early conftest):**\nThe initial capturing is implemented in `src/_pytest/capture.py` in the `pytest_load_initial_conftests` function, which is decorated with `@hookimpl(wrapper=True)`. The data flow is:\n1. `Config.parse()` in `config/__init__.py` calls `self.hook.pytest_load_initial_conftests(early_config=self, args=args, parser=self._parser)`\n2. The capture module's wrapper hook creates a `CaptureManager` with the method from `ns.capture` (default \"fd\")\n3. It registers the `CaptureManager` as plugin named \"capturemanager\" and registers `capman.stop_global_capturing` as a cleanup callback via `early_config.add_cleanup()`\n4. It calls `capman.start_global_capturing()` BEFORE yielding (which triggers actual conftest loading)\n5. After yield, it calls `capman.suspend_global_capture()`\n6. If an exception occurs, it reads captured output and writes to stdout/stderr before re-raising\n\nThe Config's own `pytest_load_initial_conftests` calls `self.pluginmanager._set_initial_conftests()` which iterates through anchors and calls `_loadconftestmodules()`.\n\n**Plugin version display (issue12 - --version and --traceconfig):**\nIn `src/_pytest/helpconfig.py`, the `pytest_report_header` function shows plugin versions when `config.option.debug` or `config.option.traceconfig` is set. It calls `getpluginversioninfo(config)` which uses `config.pluginmanager.list_plugin_distinfo()` to get plugin/dist pairs and formats them as `\"{dist.project_name}-{dist.version} at {loc}\"`. In `src/_pytest/terminal.py`, `_plugin_nameversions()` strips the \"pytest-\" prefix from plugin names for brevity.\n\n**Plugin blocking mechanism (-p no:plugin):**\nIn `src/_pytest/config/__init__.py`, `consider_preparse()` iterates through args looking for `-p` options. If the argument starts with `no:`, `consider_pluginarg()` calls `self.set_blocked(name)`. Plugins in `essential_plugins` (\"mark\", \"main\", \"runner\", \"fixtures\", \"helpconfig\") cannot be disabled - it raises `UsageError`.\n\n**Xfail for setup/teardown (issue9):**\nIn `src/_pytest/skipping.py`, `pytest_runtest_makereport` is a wrapper hook decorated with `@hookimpl(wrapper=True)`. When an xfail-marked test's setup/teardown FAILS (`call.excinfo` is truthy), the report outcome is set to \"skipped\" with `wasxfail` attribute, making it report as xfail. When setup/teardown PASSES, the condition `elif call.when == \"call\"` means xpass logic is NOT applied to setup/teardown phases - the outcome remains \"passed\" without `wasxfail`, so it reports as normally passing.",
    "rubric": [
      "Describes that `pytest_load_initial_conftests` in capture.py creates a CaptureManager, registers it, starts global capturing before yield, and suspends after yield",
      "Mentions that Config's `pytest_load_initial_conftests` calls `_set_initial_conftests` on the plugin manager to load conftest files",
      "Explains that captured output is read and written to stdout/stderr if an exception occurs during conftest loading",
      "Describes that `pytest_report_header` in helpconfig.py shows plugin versions when debug/traceconfig is set, using `getpluginversioninfo` which calls `list_plugin_distinfo`",
      "Mentions that `_plugin_nameversions` in terminal.py strips the 'pytest-' prefix from plugin names",
      "Explains that `consider_preparse`/`consider_pluginarg` handles `-p no:X` by calling `set_blocked`, but raises UsageError for essential_plugins (mark, main, runner, fixtures, helpconfig)",
      "Describes that `pytest_runtest_makereport` in skipping.py sets outcome to 'skipped' with wasxfail when an xfail-marked test's setup/teardown fails",
      "Explains that xpass logic only applies when `call.when == 'call'`, so passing setup/teardown does not trigger xpass behavior"
    ],
    "key_files": [
      "src/_pytest/capture.py",
      "src/_pytest/skipping.py",
      "src/_pytest/helpconfig.py",
      "src/_pytest/config/__init__.py",
      "src/_pytest/terminal.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.1.rst] Changes between 2.0.0 and 2.0.1\n- refine and unify initial capturing so that it works nicely\n  even if the logging module is used on an early-loaded conftest.py\n  file or plugin.\n- fix issue12 - show plugin versions with \"--version\" and\n  \"--traceconfig\" and also document how to add extra information\n  to reporting test header\n- fix issue17 (import-* reporting issue on python3) by\n  requiring py>1.4.0 (1.4.1 is going to include it)\n- fix issue10 (numpy arrays truth checking) by refining\n  assertion interpretation in py lib\n- fix issue15: make nose compatibility tests compatible\n  with python3 (now that nose-1.0 supports python3)\n- remove somewhat surprising \"same-conftest\" detection because\n  it ignores conftest.py when they appear in several subdirs.\n- improve assertions (\"not in\"), thanks Floris Bruynooghe\n- improve behaviour/warnings when running on top of \"python -OO\"\n  (assertions and docstrings are turned off, leading to potential\n  false positives)\n- introduce a pytest_cmdline_processargs(args) hook\n  to allow dynamic computation of command line arguments.\n  This fixes a regression because py.test prior to 2.0\n  allowed to set command line options from conftest.py\n  files which so far pytest-2.0 only allowed from ini-files now.\n- fix issue7: assert failures in doctest modules.\n  unexpected failures in doctests will not generally\n  show nicer, i.e. within the doctest failing context.\n- fix issue9: setup/teardown functions for an xfail-marked\n  test will report as xfail if they fail but report as normally\n  passing (not xpassing) if they succeed.  This only is true\n  for \"direct\" setup/teardown invocations because teardown_class/\n  teardown_module cannot closely relate to a single test.\n- fix issue14: no logging errors at process exit\n- refinements to \"collecting\" output on non-ttys\n- refine internal plugin registration and --traceconfig output\n- introduce a mechanism to prevent/unregister plugins from the\n  command line, see http://pytest.org/en/stable/how-to/plugins.html#cmdunregister\n- activate resultlog plugin by default\n- fix regression wrt yielded tests which due to the\n  collection-before-running semantics were not\n  setup as with pytest 1.3.4.  Note, however, that\n  the recommended and much cleaner way to do test\n  parametrization remains the \"pytest_generate_tests\"\n  mechanism, see the docs.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive code-level implementation details (specific line numbers, function names, file paths) that are not present in the documentation and cannot be verified from it alone. These appear to be plausible extrapolations based on knowledge of the pytest codebase rather than direct derivations from the changelog.",
      "The documentation states for issue9 that setup/teardown functions 'will report as xfail if they fail but report as normally passing (not xpassing) if they succeed', which the answer correctly reflects in its explanation of the xfail mechanism.",
      "The documentation mentions 'a mechanism to prevent/unregister plugins from the command line' which the answer elaborates on with specific implementation details about 'no:' prefix and essential_plugins - the essential_plugins list and specific behavior around cacheprovider/stepwise cannot be verified from this doc.",
      "The documentation says 'show plugin versions with --version and --traceconfig' - the answer discusses --traceconfig but focuses on config.option.debug rather than --version specifically, though it doesn't contradict the doc."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with line numbers and one claim about cacheprovider/stepwise blocking that's in truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_40",
    "repo": "pytest",
    "question": "How does pytest's skip/xfail expression evaluation and mark processing system work end-to-end?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The xfail/skip expression evaluation system in pytest is implemented in `src/_pytest/skipping.py` through a layered design with three key functions: `evaluate_condition`, `evaluate_skip_marks`, and `evaluate_xfail_marks`. Here's how it works:\n\n**Expression Evaluation (`evaluate_condition`):**\nWhen a string condition is provided (e.g., `@pytest.mark.skipif(\"mymodule.__version__[0] == '1'\"`):\n1. A globals dictionary is constructed with `os`, `sys`, `platform`, and `config` (the pytest Config object) as base variables.\n2. The `pytest_markeval_namespace` hook is called and its returned dictionaries are merged into globals using `reversed()` over results.\n3. If the item has an `obj` attribute, `item.obj.__globals__` is merged into the globals dict.\n4. The condition string is `compile()`d with filename `<{mark.name} condition>` and `eval()`'d against those globals.\n5. On `SyntaxError`, a formatted error message is produced showing the condition string with a caret pointing to the error offset. On other exceptions, `traceback.format_exception_only` is used for reporting.\n\n**Skip/Xfail Results as Frozen Dataclasses:**\n- `Skip` is a frozen dataclass with just a `reason` field (default: \"unconditional skip\").\n- `Xfail` is a frozen dataclass with `__slots__` containing `reason`, `run`, `strict`, and `raises`.\n- The `strict` field cascades: first from the mark kwargs, then from `strict_xfail` ini (aliased as `xfail_strict`), then from `strict` ini.\n\n**Hook Integration:**\n- `pytest_runtest_setup` in skipping.py is decorated with `@hookimpl(tryfirst=True)` to evaluate skip marks before the test setup occurs. If a skip condition is true, it raises `skip.Exception` (which is `Skipped`) with `_use_item_location=True`.\n- The xfail result is stored in `item.stash[xfailed_key]` where `xfailed_key` is a `StashKey[Xfail | None]()` \u2014 using pytest's type-safe heterogeneous stash system.\n- `pytest_runtest_call` is a wrapper hook that re-evaluates xfail marks if the stash value is None.\n- `pytest_runtest_makereport` modifies the test report outcome based on xfail status: setting `rep.wasxfail` and changing outcome to \"skipped\" for expected failures or \"failed\" with `[XPASS(strict)]` longrepr for strict xfail passes.\n\n**The `--runxfail` Flag:**\nWhen `--runxfail` is used, `pytest_configure` replaces `pytest.xfail` with a `nop` function that does nothing but preserves the `Exception` attribute (`nop.Exception = xfail.Exception`) \u2014 effectively disabling imperative xfail while maintaining the expected interface. A cleanup function restores the original.",
    "rubric": [
      "Explains that evaluate_condition builds a globals dict with os, sys, platform, and config, then merges pytest_markeval_namespace hook results (using reversed()) and item.obj.__globals__",
      "Mentions the condition string is compiled with filename '<{mark.name} condition>' and eval'd against the constructed globals",
      "Describes error handling: SyntaxError produces a formatted message with caret at offset, other exceptions use traceback.format_exception_only",
      "Notes Skip is a frozen dataclass with a 'reason' field defaulting to 'unconditional skip', and Xfail is a frozen dataclass with __slots__ containing reason, run, strict, and raises",
      "Explains the strict field cascade: mark kwargs -> strict_xfail ini (aliased as xfail_strict) -> strict ini",
      "Mentions pytest_runtest_setup is decorated with @hookimpl(tryfirst=True) and raises skip.Exception with _use_item_location=True when skip condition is met",
      "Explains xfail result is stored in item.stash using a StashKey[Xfail | None] and can be re-evaluated in pytest_runtest_call if None",
      "Describes how pytest_runtest_makereport modifies outcomes: sets wasxfail and changes to 'skipped' for expected failures, or 'failed' with '[XPASS(strict)]' for strict xfail passes",
      "Explains the --runxfail flag replaces pytest.xfail with a nop function that preserves the Exception attribute, and registers a cleanup to restore it"
    ],
    "key_files": [
      "src/_pytest/skipping.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/runner.py",
      "src/_pytest/stash.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.2.rst] py.test 2.0.2: bug fixes, improved xfail/skip expressions, speed ups\nWelcome to pytest-2.0.2, a maintenance and bug fix release of pytest,\na mature testing tool for Python, supporting CPython 2.4-3.2, Jython\nand latest PyPy interpreters.  See the extensive docs with tested examples here:\n\n    http://pytest.org/\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nMany thanks to all issue reporters and people asking questions\nor complaining, particularly Jurko for his insistence,\nLaura, Victor and Brianna for helping with improving\nand Ronny for his general advise.\n\nbest,\nholger krekel",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation is a brief release announcement for pytest 2.0.2 mentioning 'improved xfail/skip expressions' but provides no technical details about the implementation. The generated answer provides extensive implementation details (about evaluate_condition, dataclasses, hook integration, --runxfail flag, etc.) that are entirely unsupported by the provided documentation.",
      "The answer claims 'this is the key feature from release 2.0.2 that allows access to module-level variables in string expressions' \u2014 this specific attribution to release 2.0.2 is not stated in the documentation, which only mentions 'improved xfail/skip expressions' in the title without elaboration.",
      "The answer describes modern pytest internals (StashKey, frozen dataclasses, pytest_markeval_namespace hook from pytest 6.2) which are far beyond what the 2.0.2 release announcement covers, though these are plausible descriptions of the current codebase rather than contradictions."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor documentation-only claims about version history (pytest 6.2, release 2.0.2) and design rationale were removed; core code-verifiable claims preserved.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_41",
    "repo": "pytest",
    "question": "What gets included in the evaluation namespace when pytest evaluates a string condition in skipif/xfail markers, and how are errors handled?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The skipif/xfail expression handling in pytest is implemented in `src/_pytest/skipping.py` through the `evaluate_condition()` function. When a string condition is provided (as opposed to a boolean), it is evaluated using Python's `eval()` with a specially constructed globals dictionary. This globals dictionary contains:\n\n1. **Built-in modules**: `os`, `sys`, `platform` are always available\n2. **Config object**: `item.config` is available as `config`\n3. **Plugin-provided namespace**: Values from the `pytest_markeval_namespace` hook\n4. **Module globals**: Accessed via `item.obj.__globals__` \u2014 this adds all names from the test item's module globals to the evaluation namespace\n\nThe module globals are only added if the item has an `obj` attribute (checked with `hasattr(item, \"obj\")`), and they are applied last, meaning they can override the built-in names and plugin-provided names.\n\nFor error handling:\n- **SyntaxError**: The condition string is compiled with `compile(condition, filename, \"eval\")` where filename is formatted as `<{mark.name} condition>`. On syntax error, a formatted message with the caret position (`exc.offset`) is reported via `pytest.fail(pytrace=False)`.\n- **Other exceptions** (e.g., NameError): These are caught and reported using `traceback.format_exception_only()`.\n\nThe system uses two frozen dataclasses: `Skip` (with a default reason of \"unconditional skip\") and `Xfail` (with fields `reason`, `run`, `strict`, and `raises`). The `Xfail` class uses `__slots__`. For `Xfail`, the `strict` parameter cascades: first from the mark kwargs, then from the `strict_xfail` ini option (aliased as `xfail_strict`), then from the generic `strict` ini option.\n\nThe `pytest_runtest_setup` hook in skipping.py is decorated with `@hookimpl(tryfirst=True)`. If a skip condition is met, it raises `skip.Exception` (which is `Skipped`) with `_use_item_location=True`.",
    "rubric": [
      "Mentions that os, sys, and platform modules are included in the globals dictionary",
      "Mentions that item.config is available as 'config' in the namespace",
      "Mentions that the pytest_markeval_namespace hook contributes to the namespace",
      "Mentions that module globals from item.obj.__globals__ are added (only if item has 'obj' attribute)",
      "Explains that module globals are added last and can override earlier entries",
      "Explains that SyntaxError is caught and reported with a caret/offset indicator via pytest.fail(pytrace=False)",
      "Explains that other exceptions are caught and reported using traceback.format_exception_only",
      "Mentions the condition string is compiled using compile() with a filename like '<{mark.name} condition>'"
    ],
    "key_files": [
      "src/_pytest/skipping.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/hookspec.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.2.rst] Changes between 2.0.1 and 2.0.2\n- tackle issue32 - speed up test runs of very quick test functions\n  by reducing the relative overhead\n\n- fix issue30 - extended xfail/skipif handling and improved reporting.\n  If you have a syntax error in your skip/xfail\n  expressions you now get nice error reports.\n\n  Also you can now access module globals from xfail/skipif\n  expressions so that this for example works now::\n\n    import pytest\n    import mymodule\n    @pytest.mark.skipif(\"mymodule.__version__[0] == \"1\")\n    def test_function():\n        pass\n\n  This will not run the test function if the module's version string\n  does not start with a \"1\".  Note that specifying a string instead\n  of a boolean expressions allows py.test to report meaningful information\n  when summarizing a test run as to what conditions lead to skipping\n  (or xfail-ing) tests.\n\n- fix issue28 - setup_method and pytest_generate_tests work together\n  The setup_method fixture method now gets called also for\n  test function invocations generated from the pytest_generate_tests\n  hook.\n\n- fix issue27 - collectonly and keyword-selection (-k) now work together\n  Also, if you do \"py.test --collectonly -q\" you now get a flat list\n  of test ids that you can use to paste to the py.test commandline\n  in order to execute a particular test.\n\n- fix issue25 avoid reported problems with --pdb and python3.2/encodings output\n\n- fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP\n  Starting with Python3.2 os.symlink may be supported. By requiring\n  a newer py lib version the py.path.local() implementation acknowledges\n  this.\n\n- fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular\n  thanks to Laura Creighton who also reviewed parts of the documentation.\n\n- fix slightly wrong output of verbose progress reporting for classes\n  (thanks Amaury)\n\n- more precise (avoiding of) deprecation warnings for node.Class|Function accesses\n\n- avoid std unittest assertion helper code in tracebacks (thanks Ronny)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states the skipif example will NOT run the test if the version string does NOT start with '1', but the decorator shown is `@pytest.mark.skipif(\"mymodule.__version__[0] == \"1\")` which would skip when it DOES equal '1'. The documentation text appears to have a logical inconsistency (likely a typo in the docs themselves), and the generated answer doesn't address this.",
      "The generated answer provides extensive implementation details (specific function names like `evaluate_condition()`, dataclass names like `Skip` and `Xfail`, hook decorators like `@hookimpl(tryfirst=True)`, `__slots__` usage, `pytest_markeval_namespace` hook, etc.) that are not mentioned in the documentation. These appear to be plausible code-level details derived from actual source code rather than the provided documentation chunk.",
      "The documentation only mentions that module globals are now accessible from xfail/skipif expressions; the answer's claim about `item.obj.__globals__` being the mechanism is a plausible implementation detail but not verified by the doc.",
      "The answer's description of error handling (SyntaxError with caret position, compile() usage, traceback.format_exception_only()) is plausible but not directly supported by the documentation, which only says 'you now get nice error reports' for syntax errors."
    ],
    "strip_verify_leakage": 0.13,
    "strip_verify_summary": "Minor doc leakage: version history claim (pytest 6.2), rationale for __slots__ and tryfirst, and explanation of _use_item_location purpose.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_42",
    "repo": "pytest",
    "question": "Why does skip handling in TestReport.from_item_and_call use _getreprcrash() instead of the full repr_failure/getrepr path used for failures?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The design decision for lightweight skip handling is implemented in `TestReport.from_item_and_call()` in `src/_pytest/reports.py`, where skip exceptions are handled with a lightweight path that only calls `excinfo._getreprcrash()` instead of the expensive `excinfo.getrepr()` / `item.repr_failure()` path used for failures.\n\n**Why this is faster:**\n\n1. For skipped tests, the `longrepr` is stored as a simple tuple `(path, lineno, message)` obtained via `excinfo._getreprcrash()`. This method returns a lightweight `ReprFileLocation` with just filename, line number, and exception message string.\n\n2. In contrast, failed tests go through `_format_failed_longrepr()` which calls `item.repr_failure()` \u2192 `item._repr_failure_py()` \u2192 `excinfo.getrepr()`. This triggers a full exception representation pipeline that constructs `ReprTraceback`/`ExceptionChainRepr` objects.\n\n3. Additional design decisions support lighter skip handling: the `Skipped` class sets `__module__ = 'builtins'`, and the `_Skip.__call__()` uses `__tracebackhide__ = True` to mark its frame as hidden.\n\n4. The `_use_item_location` flag further optimizes skip reporting by using the item's own location data rather than parsing through the exception traceback, which is particularly useful for marker-based skips evaluated in `pytest_runtest_setup`.",
    "rubric": [
      "Explains that skipped tests store longrepr as a lightweight tuple (path, lineno, message) via excinfo._getreprcrash(), avoiding the expensive full traceback representation",
      "Explains that failed tests go through _format_failed_longrepr() \u2192 item.repr_failure() / item._repr_failure_py() \u2192 excinfo.getrepr(), which builds full ExceptionChainRepr/ReprTraceback objects",
      "Mentions that this is a performance optimization since skips don't need detailed traceback information",
      "Mentions the _use_item_location flag that further optimizes skip reporting by using the item's own location instead of parsing the exception traceback",
      "Mentions supporting design decisions like Skipped.__module__ = 'builtins' or __tracebackhide__ = True in _Skip.__call__"
    ],
    "key_files": [
      "src/_pytest/reports.py",
      "src/_pytest/_code/code.py",
      "src/_pytest/outcomes.py",
      "src/_pytest/skipping.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.3.rst] py.test 2.0.3: bug fixes and speed ups\nWelcome to pytest-2.0.3, a maintenance and bug fix release of pytest,\na mature testing tool for Python, supporting CPython 2.4-3.2, Jython\nand latest PyPy interpreters.  See the extensive docs with tested examples here:\n\n    http://pytest.org/\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nThere also is a bugfix release 1.6 of pytest-xdist, the plugin\nthat enables seamless distributed and \"looponfail\" testing for Python.\n\nbest,\nholger krekel",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only mentions 'bug fixes and speed ups' in the title without providing any specific details about the nature of the speed ups. The generated answer provides extensive implementation-level details about skip handling optimization that are not supported by or derivable from this documentation chunk.",
      "The answer appears to be derived from source code analysis rather than from the provided documentation, which contains no information about TestReport.from_item_and_call(), excinfo._getreprcrash(), or any of the other implementation details mentioned.",
      "While the claims about code implementation may be accurate based on the actual pytest source code, they cannot be verified from this documentation alone, making them extrapolations beyond what the doc supports."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code structure; a few rely on release notes references, comments about design intent ('hack'), and truncated code internals.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_43",
    "repo": "pytest",
    "question": "Where in the codebase are the pytest 2.0.3 fixes implemented \u2014 specifically for invalid XML characters, skip reasons in JUnit XML, test class collection, and skip speed optimization?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The fixes mentioned in the pytest 2.0.3 release notes are located across several key files in the current codebase:\n\n1. **Invalid characters in JUnit XML (issue 37)**: The `bin_xml_escape()` function in `src/_pytest/junitxml.py` handles escaping invalid XML characters. It uses a regex pattern matching the XML spec's valid character range (`#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]`), and also disallows `#x7F` (DEL). Invalid characters are replaced with visual representations like `#x07` (for chars \u2264 0xFF) or `#x0FFFF` (for higher code points). This function is called in multiple places throughout junitxml.py \u2014 on test names, property values, attributes, error messages, skip reasons, captured output content, and failure messages.\n\n2. **Skip reason/meta in JUnit XML**: The `_NodeReporter.append_skipped()` method in `src/_pytest/junitxml.py` writes skip information into the XML. For regular skips, the longrepr tuple `(filename, lineno, skipreason)` is unpacked, the `\"Skipped: \"` prefix is stripped if present, and a `<skipped>` element is created with `type=\"pytest.skip\"` and the reason as the `message` attribute, both bin_xml_escaped. For xfail skips, it similarly strips a `\"reason: \"` prefix and uses `type=\"pytest.xfail\"`. The skip longrepr tuple is constructed in `TestReport.from_item_and_call()` in `src/_pytest/reports.py`, where `excinfo._getreprcrash()` extracts the crash location, and the message flows from there.\n\n3. **Test-prefixed classes deriving from object (issue 34)**: The `istestclass()` method in `PyCollector` in `src/_pytest/python.py` uses `classnamefilter()` and `inspect.isabstract()` to filter test classes. The `Class.collect()` method checks for `__init__` and `__new__` constructors via `hasinit()` and `hasnew()` functions which compare against `object.__init__` and `object.__new__`. Additionally, the `IGNORED_ATTRIBUTES` frozenset pre-computes attributes from `dir(types.ModuleType(\"empty_module\"))` and an empty class (`_EmptyClass`) to skip common built-in attributes during collection.\n\n4. **Speed up skips**: For skipped tests, `TestReport.from_item_and_call()` in `src/_pytest/reports.py` creates a lightweight tuple `(path, lineno, message)` as `longrepr` using only `excinfo._getreprcrash()` from `src/_pytest/_code/code.py`, rather than calling the full `getrepr()` method.\n\n5. **Hook tracebacks**: The `CallInfo.from_call()` classmethod in `src/_pytest/runner.py` wraps hook/setup/call/teardown invocations, catching exceptions and storing them as `ExceptionInfo` objects. The `reraise` parameter controls which exceptions propagate directly (Exit and optionally KeyboardInterrupt via `get_reraise_exceptions()`).",
    "rubric": [
      "Identifies bin_xml_escape() in junitxml.py as handling invalid XML character escaping, including that it uses a regex based on the XML spec's valid character range and replaces invalid chars with visual representations like #x07",
      "Identifies that bin_xml_escape is called in multiple places in junitxml.py: on test names, property values, attributes, error messages, skip reasons, captured output, and failure messages",
      "Identifies _NodeReporter.append_skipped() in junitxml.py as writing skip info, noting the 'Skipped: ' prefix stripping for regular skips and 'reason: ' prefix stripping for xfails, and the use of type='pytest.skip' and type='pytest.xfail'",
      "Identifies istestclass() in PyCollector (python.py) for test class filtering using classnamefilter and inspect.isabstract, plus hasinit()/hasnew() functions that compare against object.__init__ and object.__new__",
      "Identifies IGNORED_ATTRIBUTES frozenset in python.py built from dir() of an empty module and _EmptyClass to skip built-in attributes during collection",
      "Identifies that for skipped tests, TestReport.from_item_and_call() in reports.py creates a lightweight (path, lineno, message) tuple using excinfo._getreprcrash() instead of the full getrepr()",
      "Identifies CallInfo.from_call() in runner.py as the wrapper that catches exceptions and stores them as ExceptionInfo, with a reraise parameter controlling exception propagation"
    ],
    "key_files": [
      "src/_pytest/junitxml.py",
      "src/_pytest/reports.py",
      "src/_pytest/python.py",
      "src/_pytest/runner.py",
      "src/_pytest/_code/code.py",
      "src/_pytest/outcomes.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.0.3.rst] Changes between 2.0.2 and 2.0.3\n- fix issue38: nicer tracebacks on calls to hooks, particularly early\n  configure/sessionstart ones\n\n- fix missing skip reason/meta information in junitxml files, reported\n  via http://lists.idyll.org/pipermail/testing-in-python/2011-March/003928.html\n\n- fix issue34: avoid collection failure with \"test\" prefixed classes\n  deriving from object.\n\n- don't require zlib (and other libs) for genscript plugin without\n  --genscript actually being used.\n\n- speed up skips (by not doing a full traceback representation\n  internally)\n\n- fix issue37: avoid invalid characters in junitxml's output",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; main issues are specific line numbers and some design rationale statements not recoverable without docs/comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_44",
    "repo": "pytest",
    "question": "How does pytest's assertion rewriting mechanism intercept imports, transform assert statements, and produce detailed error messages?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The pytest assertion rewriting mechanism works by implementing an import hook (`AssertionRewritingHook`, which inherits from `importlib.abc.MetaPathFinder` and `importlib.abc.Loader`) that intercepts module imports and rewrites `assert` statements in test modules at the AST level before execution. Here's the detailed algorithm:\n\n**1. Hook Installation:**\nThe `install_importhook()` function in `_pytest/assertion/__init__.py` creates an `AssertionRewritingHook` instance and inserts it at position 0 of `sys.meta_path`.\n\n**2. Module Interception (`find_spec`):**\nWhen a module is imported, `find_spec()` is called. It first uses `_early_rewrite_bailout()` as a fast filter \u2014 checking if the module basename matches test file patterns (e.g., `test_*.py`, `*_test.py`), is `conftest`, or is marked for rewriting via `_must_rewrite`. If the module passes these checks, it returns a ModuleSpec with `self` (the hook) as the loader.\n\n**3. Module Execution (`exec_module`):**\nThe hook first checks for a cached rewritten `.pyc` file (stored in `__pycache__` with a special tag `{sys.implementation.cache_tag}-pytest-{version}`). The `_read_pyc()` function validates the cache by checking: magic number (4 bytes), flags (must be `\\x00\\x00\\x00\\x00`), mtime (4 bytes, little-endian), and size (4 bytes, little-endian) \u2014 a 16-byte header total. If no valid cache exists, it calls `_rewrite_test()` which parses the source into an AST, calls `rewrite_asserts()`, and compiles back to bytecode.\n\n**4. AST Rewriting (`AssertionRewriter.run`):**\nThe `run()` method first checks if rewriting is disabled by looking for `\"PYTEST_DONT_REWRITE\"` in the module docstring. It then inserts two special imports at the top of the module (after docstrings and `__future__` imports):\n- `import builtins as @py_builtins`\n- `import _pytest.assertion.rewrite as @pytest_ar`\n\nThese use the `@` character which is invalid in Python identifiers.\n\n**5. Assert Transformation (`visit_Assert`):**\nEach `assert` statement is replaced with:\n- Assignment statements that compute intermediate values into variables named `@py_assert0`, `@py_assert1`, etc.\n- An `if not <condition>:` block that constructs a detailed error message using %-formatting (with placeholders like `%(py0)s`) and raises `AssertionError` with the formatted explanation.\n- A cleanup statement setting all temporary variables to `None`.\n\n**6. Explanation Building:**\nThe rewriter uses a stack-based format context system (`push_format_context()`/`pop_format_context()`) to build %-formatted explanation strings. Each sub-expression's value is captured via `_saferepr()` and inserted into the message. For comparisons, `_call_reprcompare()` is used which delegates to the `pytest_assertrepr_compare` hook if available.\n\n**7. Concurrency Safety:**\nWhen writing cached `.pyc` files, `_write_pyc()` first writes to a temporary file named `{pyc}.{os.getpid()}` and then uses `os.replace()` to move it to the final location.\n\n**8. Guard Against Recursion:**\nA `_writing_pyc` flag prevents the hook from trying to rewrite modules during the pyc-writing process itself (checked at the top of `find_spec()`, which returns `None` if `self._writing_pyc` is `True`).",
    "rubric": [
      "Explains that AssertionRewritingHook is installed at position 0 of sys.meta_path and inherits from importlib.abc.MetaPathFinder and importlib.abc.Loader",
      "Describes the find_spec method's early bailout logic: checking module basename against test file patterns, conftest, and _must_rewrite set",
      "Explains exec_module checks for cached .pyc files (with a pytest-specific tag) and falls back to _rewrite_test which parses source into AST, calls rewrite_asserts, and compiles to bytecode",
      "Describes the .pyc cache validation in _read_pyc: checking magic number, flags (4 zero bytes), mtime, and size in a 16-byte header",
      "Explains that run() checks for PYTEST_DONT_REWRITE in the module docstring and inserts special imports (builtins as @py_builtins, _pytest.assertion.rewrite as @pytest_ar) after docstrings and __future__ imports",
      "Describes visit_Assert: replaces assert with assignment statements for intermediate values (using @py_assert variables), an if-not-condition block that raises AssertionError, and a cleanup setting variables to None",
      "Explains the %-formatting explanation system using push_format_context/pop_format_context stack and explanation_param creating %(pyN)s placeholders",
      "Mentions _call_reprcompare for comparison expressions delegating to pytest_assertrepr_compare hook",
      "Describes concurrency safety: _write_pyc writes to a temp file with PID suffix then uses os.replace for atomic rename",
      "Mentions the _writing_pyc flag that prevents recursive rewriting during pyc writing (checked at the top of find_spec)"
    ],
    "key_files": [
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/assertion/__init__.py",
      "src/_pytest/assertion/util.py",
      "src/_pytest/config/__init__.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.1.0.rst] py.test 2.1.0: perfected assertions and bug fixes\nWelcome to the release of pytest-2.1, a mature testing tool for Python,\nsupporting CPython 2.4-3.2, Jython and latest PyPy interpreters.  See\nthe improved extensive docs (now also as PDF!) with tested examples here:\n\n     http://pytest.org/\n\nThe single biggest news about this release are **perfected assertions**\ncourtesy of Benjamin Peterson.  You can now safely use ``assert``\nstatements in test modules without having to worry about side effects\nor python optimization (\"-OO\") options.  This is achieved by rewriting\nassert statements in test modules upon import, using a PEP302 hook.\nSee https://docs.pytest.org/en/stable/how-to/assert.html for\ndetailed information.  The work has been partly sponsored by my company,\nmerlinux GmbH.\n\nFor further details on bug fixes and smaller enhancements see below.\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nbest,\nholger krekel / https://merlinux.eu/",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (PEP451, AssertionRewritingHook class, find_spec, exec_module, _read_pyc, _write_pyc, AST rewriting details, @py_assert variables, format context system, concurrency safety, etc.) that go far beyond what the documentation states. The documentation only says that assert statements are rewritten 'upon import, using a PEP302 hook' and refers readers to a URL for detailed information.",
      "The documentation mentions only PEP302; the answer adds PEP451 which is not mentioned in the documentation (though this is plausible as a later evolution of the import hook system).",
      "The answer's core claim that assertion rewriting uses a PEP302 import hook to rewrite assert statements in test modules is consistent with the documentation.",
      "The detailed algorithmic steps (cache format, @-prefixed variable names, _saferepr, atomic file writes, etc.) are neither supported nor contradicted by the documentation \u2014 they appear to be derived from source code knowledge rather than from this document."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in PEP references, design rationales (name clash avoidance, atomicity, race conditions, issue numbers), but the vast majority of claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_45",
    "repo": "pytest",
    "question": "What are the implementation details and dependencies connecting the assertion rewriting system and the JUnit XML output system in pytest?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The pytest release 2.1.1 referenced in the documentation concerns two key systems in pytest: the assertion rewriting mechanism and the JUnit XML output. Here's what the code reveals about their implementation and dependencies:\n\n**JUnit XML system (`src/_pytest/junitxml.py`):**\n- The `LogXML` class is the main plugin that generates JUnit XML reports. It is registered in `pytest_configure` and stored via a `StashKey` (`xml_key`). The default `junit_family` is `\"xunit2\"` (set in `pytest_addoption`).\n- The system-out/system-err tags are generated by `_NodeReporter.write_captured_output()`, which reads `report.capstdout`, `report.capstderr`, and `report.caplog` from the `TestReport` and writes them into XML `<system-out>` or `<system-err>` elements depending on the `junit_logging` configuration option. The default `junit_logging` value is `\"no\"`.\n- The `_write_content` method creates an `ET.Element` with the tag name being `\"system-out\"` or `\"system-err\"` and sets its text to the `bin_xml_escape`'d content.\n- `bin_xml_escape` escapes invalid XML characters by replacing them with `#xNN` format, filtering against a regex of valid character ranges.\n- The output XML wraps everything in a `<testsuites name=\"pytest tests\">` root element containing a `<testsuite>` element. The testsuite includes attributes: name, errors, failures, skipped, tests, time, timestamp, hostname.\n- `LogXML` tracks double failures (call failure + teardown error) via `cnt_double_fail_tests`, subtracting that from the total test count.\n- The `families` dict defines valid testcase attributes: xunit1 allows `[\"classname\", \"name\", \"file\", \"line\", \"url\"]`, while xunit2 allows only `[\"classname\", \"name\"]`.\n- `_NodeReporter.finalize()` clears `__dict__` and replaces `to_xml` with a lambda returning cached data.\n\n**Assertion rewriting system (`src/_pytest/assertion/`):**\n- The `AssertionRewritingHook` class (in `rewrite.py`) implements `importlib.abc.MetaPathFinder` and `importlib.abc.Loader` to intercept module imports and rewrite assert statements.\n- It is installed at position 0 in `sys.meta_path` by `install_importhook()` in `__init__.py`.\n- The `AssertionRewriter` class uses AST to transform assert statements to provide detailed assertion error messages.\n- Rewritten pyc files are cached with the tag format `{sys.implementation.cache_tag}-pytest-{version}` (stored as `PYTEST_TAG`).\n- The assertion module supports a `pytest_assertion_pass` hook (gated behind `enable_assertion_pass_hook` ini option).\n- Truncation of assertion explanations defaults to `DEFAULT_MAX_LINES = 8` lines and `DEFAULT_MAX_CHARS = 640` characters (8*80).\n\n**Connection between the systems:**\n- Both `assertion` and `junitxml` are listed in `default_plugins` tuple in `src/_pytest/config/__init__.py`, meaning they're loaded by default.\n- The `TestReport` class (in `src/_pytest/reports.py`) bridges them: assertion failures produce `longrepr` on reports, while captured output (which junitxml reads via `capstdout`/`capstderr`/`caplog` properties) comes from sections stored in the report. The `capstdout` property joins sections matching the prefix \"Captured stdout\".\n- The junitxml plugin accesses failure info via `report.longrepr.reprcrash.message` (a `ReprFileLocation`) for producing the `<failure>` element's message attribute.",
    "rubric": [
      "Identifies LogXML as the main JUnit XML plugin class registered in pytest_configure with a StashKey",
      "Explains that _NodeReporter.write_captured_output() reads report.capstdout/capstderr/caplog to generate system-out/system-err XML elements",
      "Mentions bin_xml_escape for escaping invalid XML characters",
      "Describes the XML structure: testsuites root element containing testsuite with attributes like name, errors, failures, skipped, tests, time, timestamp, hostname",
      "Notes the families dict difference: xunit1 allows classname/name/file/line/url while xunit2 allows only classname/name",
      "Identifies AssertionRewritingHook as implementing MetaPathFinder and Loader to intercept imports",
      "Explains the hook is installed at position 0 in sys.meta_path",
      "Mentions the pyc cache tag format includes sys.implementation.cache_tag and pytest version",
      "Notes both plugins are listed in default_plugins for automatic loading",
      "Explains TestReport bridges the systems: longrepr for assertion failures and capstdout/capstderr/caplog properties for captured output",
      "Mentions junitxml accesses failure info via report.longrepr.reprcrash.message for the failure element's message attribute"
    ],
    "key_files": [
      "src/_pytest/junitxml.py",
      "src/_pytest/assertion/__init__.py",
      "src/_pytest/assertion/rewrite.py",
      "src/_pytest/reports.py",
      "src/_pytest/assertion/truncate.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.1.1.rst] py.test 2.1.1: assertion fixes and improved junitxml output\npytest-2.1.1 is a backward compatible maintenance release of the\npopular py.test testing tool.  See extensive docs with examples here:\n\n     http://pytest.org/\n\nMost bug fixes address remaining issues with the perfected assertions\nintroduced with 2.1.0 - many thanks to the bug reporters and to Benjamin\nPeterson for helping to fix them.  Also, junitxml output now produces\nsystem-out/err tags which lead to better displays of tracebacks with Jenkins.\n\nAlso a quick note to package maintainers and others interested: there now\nis a \"pytest\" man page which can be generated with \"make man\" in doc/.\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nbest,\nholger krekel / https://merlinux.eu/",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in design rationale claims (intentional XML entity avoidance, XML spec reference, release notes mention); most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_46",
    "repo": "pytest",
    "question": "Why does pytest reorder test items and sort fixture closures by scope, and how does the caching mechanism in FixtureDef avoid redundant fixture execution?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The pytest fixture scoping and caching mechanism works as follows, based on the actual code:\n\n**1. Scope-based caching avoids redundant fixture execution:** `FixtureDef.execute()` (in `src/_pytest/fixtures.py`) checks `self.cached_result` before running the fixture function. The `cached_result` is a 3-tuple of `(value, cache_key, exception_info)`. The `cache_key` is computed by `FixtureDef.cache_key()` which returns `getattr(request, 'param', None)` \u2014 meaning for non-parametrized fixtures the key is always `None`, so the cached value is always reused within the fixture's lifetime (determined by its scope). When the cache key matches, the cached value is returned immediately without re-invoking the fixture function.\n\n**2. The `Scope` enum enables ordered comparison for scope enforcement:** The `Scope` enum in `src/_pytest/scope.py` uses `@total_ordering` with values ordered Function < Class < Module < Package < Session. The `SubRequest._check_scope()` method prevents a higher-scoped fixture from depending on a lower-scoped one (e.g., a session-scoped fixture requesting a function-scoped fixture).\n\n**3. Fixture closure sorting by scope (higher first):** In `FixtureManager.getfixtureclosure()`, `fixturenames_closure.sort(key=sort_by_scope, reverse=True)` ensures higher-scoped fixtures (session, package, module) are set up first, so shared fixtures are established before narrower-scoped fixtures that depend on them.\n\n**4. Test item reordering groups tests sharing parametrized fixtures together:** `reorder_items()` and `reorder_items_atscope()` reorganize the test execution order so that tests sharing the same high-scoped parametrized fixture values are run adjacently. The algorithm processes `HIGH_SCOPES` (all scopes except Function, defined in `src/_pytest/scope.py`) and uses `ParamArgKey` dataclass which includes the argname, param_index, scoped_item_path, and item_cls to identify which tests share the same fixture instance. The optimization short-circuits when there are fewer than 3 items (`if scope is Scope.Function or len(items) < 3: return items`).\n\n**5. Finalization cascade ensures proper teardown ordering:** `FixtureDef.finish()` sets `cached_result = None` and clears `_finalizers` even if finalization fails. When a fixture depends on another fixture, the `execute()` method adds a `functools.partial(self.finish, request=request)` finalizer to each parent fixture, so if a parent fixture is torn down (e.g., due to reparametrization), the dependent fixture is torn down first.\n\n**6. The `SetupState` stack drives scope-aware teardown:** `SetupState.teardown_exact()` (in `src/_pytest/runner.py`) pops nodes from the stack that aren't shared with the next item, running their finalizers (which include fixture teardowns). This means session-scoped fixtures survive across all items, module-scoped survive within a module, etc.\n\n**7. Special handling for `cached_result` comparison edge cases:** In `FixtureDef.execute()`, the cache key comparison wraps `==` in a `try/except (ValueError, RuntimeError)` to handle objects that don't support standard equality, falling back to identity comparison (`is`).",
    "rubric": [
      "Explains that FixtureDef.execute() checks cached_result (a 3-tuple of value, cache_key, exception_info) before running the fixture function, returning the cached value when the cache key matches",
      "Explains that cache_key() returns getattr(request, 'param', None), so non-parametrized fixtures always have key None and are reused within their scope lifetime",
      "Mentions the try/except (ValueError, RuntimeError) fallback to identity comparison ('is') for cache key equality checks to handle objects like numpy arrays",
      "Explains that the Scope enum uses @total_ordering with Function < Class < Module < Package < Session ordering, enabling SubRequest._check_scope() to prevent higher-scoped fixtures from depending on lower-scoped ones",
      "Explains that fixture closure sorting (sort_by_scope with reverse=True) ensures higher-scoped fixtures are set up before narrower-scoped ones",
      "Explains that reorder_items/reorder_items_atscope reorganize test execution so tests sharing the same high-scoped parametrized fixture values run adjacently, minimizing setup/teardown cycles",
      "Mentions the short-circuit optimization in reorder_items_atscope: returns early when scope is Function or fewer than 3 items",
      "Explains the finalization cascade: FixtureDef.finish() nullifies cached_result, and execute() adds a partial(self.finish) finalizer to parent fixtures so dependent fixtures are torn down first",
      "Explains that SetupState.teardown_exact() pops non-shared nodes from the stack when transitioning between items, so higher-scoped fixtures survive across items that share them"
    ],
    "key_files": [
      "src/_pytest/fixtures.py",
      "src/_pytest/scope.py",
      "src/_pytest/runner.py",
      "src/_pytest/unittest.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.3.0.rst] pytest-2.3: improved fixtures / better unittest integration\npytest-2.3 comes with many major improvements for fixture/funcarg management\nand parametrized testing in Python.  It is now easier, more efficient and\nmore predictable to re-run the same tests with different fixture\ninstances.  Also, you can directly declare the caching \"scope\" of\nfixtures so that dependent tests throughout your whole test suite can\nreuse database or other expensive fixture objects with ease.  Lastly,\nit's possible for fixture functions (formerly known as funcarg\nfactories) to use other fixtures, allowing for a completely modular and\nreusable fixture design.\n\nFor detailed info and tutorial-style examples, see:\n\n    http://pytest.org/en/stable/explanation/fixtures.html\n\nMoreover, there is now support for using pytest fixtures/funcargs with\nunittest-style suites, see here for examples:\n\n    http://pytest.org/en/stable/how-to/unittest.html\n\nBesides, more unittest-test suites are now expected to \"simply work\"\nwith pytest.\n\nAll changes are backward compatible and you should be able to continue\nto run your test suites and 3rd party plugins that worked with\npytest-2.2.4.\n\nIf you are interested in the precise reasoning (including examples) of the\npytest-2.3 fixture evolution, please consult\nhttp://pytest.org/en/stable/funcarg_compare.html\n\nFor general info on installation and getting started:\n\n    http://pytest.org/en/stable/getting-started.html\n\nDocs and PDF access as usual at:\n\n    http://pytest.org\n\nand more details for those already in the knowing of pytest can be found\nin the CHANGELOG below.\n\nParticular thanks for this release go to Floris Bruynooghe, Alex Okrushko\nCarl Meyer, Ronny Pfannschmidt, Benjamin Peterson and Alex Gaynor for helping\nto get the new features right and well integrated.  Ronny and Floris\nalso helped to fix a number of bugs and yet more people helped by\nproviding bug reports.\n\nhave fun,\nholger krekel",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes high-level features of pytest 2.3 (fixture scoping, caching, modularity, unittest integration). The generated answer goes far beyond the documentation into specific code-level implementation details (line numbers, method names, enum values, algorithm descriptions) that cannot be verified from the provided documentation chunk alone.",
      "The answer's claims about specific line numbers, method signatures, and implementation details (e.g., FixtureDef.execute() at line 1069, Scope enum at lines 34-40, SubRequest._check_scope() at lines 783-802, etc.) are neither supported nor contradicted by the documentation \u2014 they appear to be derived from actual source code rather than this doc.",
      "The high-level motivation described in point 1 (scope-based caching for performance with expensive fixtures like databases) is consistent with the documentation's statement about declaring caching 'scope' for reusing 'database or other expensive fixture objects'.",
      "The mention of fixture functions using other fixtures (modularity) in the answer aligns with the documentation's statement about fixture functions being able to 'use other fixtures, allowing for a completely modular and reusable fixture design'.",
      "No claims in the answer directly contradict the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale statements and the historical reference (pytest 2.3, issue #6497, numpy arrays) rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_47",
    "repo": "pytest",
    "question": "Where is per-directory customization of assertion comparison reporting implemented, and how does the hook dispatch become directory-aware?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The per-directory assertion reporting customization is implemented through the `pytest_assertrepr_compare` hook, which can be defined in per-directory `conftest.py` files. The mechanism works through the following chain:\n\n1. **`item.ihook` property** (defined in `src/_pytest/nodes.py`): This property calls `self.session.gethookproxy(self.path)`, returning a path-sensitive hook proxy.\n\n2. **`Session.gethookproxy()`** (defined in `src/_pytest/main.py`): This method gets conftest modules relevant to a path and computes which conftest modules should be excluded. If any modules need to be excluded, it creates an `FSHookProxy`; otherwise it returns the global `config.hook`.\n\n3. **`FSHookProxy`** (defined in `src/_pytest/main.py`): A proxy class whose `__getattr__` calls `self.pm.subset_hook_caller(name, remove_plugins=self.remove_mods)` to create a hook caller that excludes certain plugins. It caches hook callers on the instance dict via `self.__dict__[name] = x` for performance.\n\n4. **`PytestPluginManager._dirpath2confmods`** (initialized in `src/_pytest/config/__init__.py`): A dict mapping directory paths to their list of conftest modules. The conftest loading mechanism walks through parent directories, importing each `conftest.py` it finds.\n\n5. **The actual hook dispatch** happens in `pytest_runtest_protocol()` (defined in `src/_pytest/assertion/__init__.py` with `@hookimpl(wrapper=True, tryfirst=True)`): It captures `ihook = item.ihook` and defines `callbinrepr()`, which calls `ihook.pytest_assertrepr_compare(config=item.config, op=op, left=left, right=right)`. This function is stored in `util._reprcompare` so the assertion rewrite module can call it during assertion failures.\n\nA test `test_assertrepr_loaded_per_dir` in `testing/test_assertion.py` verifies this behavior by creating directories with different `conftest.py` files each defining `pytest_assertrepr_compare` returning different summaries, and asserting that each test file sees only its directory's custom assertion message.",
    "rubric": [
      "Mentions `ihook` property on Node (in nodes.py) which calls `self.session.gethookproxy(self.path)` to get a path-sensitive hook proxy",
      "Mentions `Session.gethookproxy()` (in main.py) which computes which conftest modules to exclude based on the path and creates FSHookProxy when needed",
      "Mentions `FSHookProxy` class (in main.py) whose `__getattr__` calls `subset_hook_caller` to create hook callers excluding irrelevant conftest plugins, with caching via `self.__dict__`",
      "Mentions `pytest_runtest_protocol` in assertion/__init__.py captures `item.ihook` and defines `callbinrepr` which calls `ihook.pytest_assertrepr_compare`, stored in `util._reprcompare`",
      "Mentions the conftest module resolution mechanism (e.g., `_getconftestmodules` or `_conftest_plugins`) that determines which conftest.py files apply to a given path"
    ],
    "key_files": [
      "src/_pytest/assertion/__init__.py",
      "src/_pytest/main.py",
      "src/_pytest/nodes.py",
      "src/_pytest/config/__init__.py",
      "testing/test_assertion.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.1.3.rst] py.test 2.1.3: just some more fixes\npytest-2.1.3 is a minor backward compatible maintenance release of the\npopular py.test testing tool.  It is commonly used for unit, functional-\nand integration testing.  See extensive docs with examples here:\n\n     http://pytest.org/\n\nThe release contains another fix to the perfected assertions introduced\nwith the 2.1 series as well as the new possibility to customize reporting\nfor assertion expressions on a per-directory level.\n\nIf you want to install or upgrade pytest, just type one of::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nThanks to the bug reporters and to Ronny Pfannschmidt, Benjamin Peterson\nand Floris Bruynooghe who implemented the fixes.\n\nbest,\nholger krekel",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; removed release notes reference, adjusted line numbers and details not visible in truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pytest_gen_48",
    "repo": "pytest",
    "question": "How are the major pytest 2.2.0 features \u2014 parametrization, mark-based selection, duration profiling, and eager teardown \u2014 implemented?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The pytest 2.2.0 release features (parametrization, mark selection, duration profiling, and eager teardown) are implemented as follows:\n\n## 1. Parametrization (`@pytest.mark.parametrize` and `metafunc.parametrize()`)\n\nThe `Metafunc.parametrize()` method (in `src/_pytest/python.py`) handles test parametrization. It works by:\n\n1. Parsing argument names via `ParameterSet._for_parametrize()` which splits comma-separated string argnames and validates parameter counts match.\n2. Resolving IDs for each parameter set via `_resolve_parameter_set_ids()`.\n3. Determining argument \"directness\" (direct vs indirect) via `_resolve_args_directness()` from `src/_pytest/fixtures.py`.\n4. Building the cartesian product of calls when `parametrize()` is called multiple times: iterating over existing `self._calls` (or a single empty `CallSpec2()`) and for each, creating new `CallSpec2` entries for each parameter set.\n\nThe `CallSpec2.setmulti()` method creates new call specs, raising a `CollectError` if a duplicate parametrization of the same argument is detected. The test ID is constructed by joining `_idlist` parts with dashes (\"-\").\n\nThe `pytest_generate_tests` hook bridges the decorator to the API: `for marker in metafunc.definition.iter_markers(name=\"parametrize\"): metafunc.parametrize(*marker.args, **marker.kwargs, _param_mark=marker)`.\n\n## 2. Mark Selection (`-m markexpr`)\n\nThe `-m` option (in `src/_pytest/mark/__init__.py`) stores the expression in `config.option.markexpr`. During collection modification (`pytest_collection_modifyitems`), `deselect_by_mark()` is called, which:\n1. Compiles the expression string into an `Expression` object using `Expression.compile()` (in `src/_pytest/mark/expression.py`), which parses it via a `Scanner` into a Python AST and compiles it to a code object.\n2. Evaluates against each item using `MarkMatcher.from_markers()`, which builds a dict mapping mark names to lists of `Mark` objects. The matcher's `__call__` method also supports keyword arguments, checking mark kwargs match.\n3. Deselected items trigger the `pytest_deselected` hook.\n\n## 3. `--strict` / `--strict-markers`\n\nWhen `strict_markers` is enabled (via `--strict-markers` or the legacy `--strict` option), the `MarkGenerator.__getattr__()` in `src/_pytest/mark/structures.py` checks if a mark name is registered in the `markers` ini-variable. If not registered, it calls `fail()` with the message `\"{name!r} not found in \\`markers\\` configuration option\"`. Without strict mode, only a `PytestUnknownMarkWarning` is issued. It also detects misspellings of \"parametrize\" (checking for \"parameterize\", \"parametrise\", \"parameterise\") and provides a helpful error.\n\n## 4. Duration Profiling (`--durations=N`)\n\nImplemented in `src/_pytest/runner.py`. The `--durations` option takes an integer N (0 for all). In `pytest_terminal_summary()`:\n1. Collects all reports with a `duration` attribute from `terminalreporter.stats`.\n2. Sorts them by duration in descending order.\n3. Truncates to N entries (if N > 0).\n4. Filters out entries below `durations_min` threshold (default 0.005 seconds, or 0.0 if verbosity >= 2 i.e. `-vv`).\n5. Prints each as `\"{duration:02.2f}s {when:<8} {nodeid}\"` showing the phase (setup/call/teardown) and node ID.\n\nDuration is recorded in `CallInfo.from_call()` using `timing.Instant()` to measure elapsed time, which is then transferred to `TestReport` via `from_item_and_call()`.\n\n## 5. Eager Teardown\n\nThe `SetupState.teardown_exact(nextitem)` method implements eager teardown. It uses `nextitem` to determine which nodes from the current stack are no longer needed. If `nextitem` shares ancestor collectors with the current item, only nodes not shared are torn down immediately.",
    "rubric": [
      "Explains that Metafunc.parametrize() parses argnames via ParameterSet._for_parametrize(), resolves IDs, determines directness, and builds cartesian product of calls across multiple parametrize invocations",
      "Mentions that CallSpec2.setmulti() creates new call specs and raises CollectError on duplicate parametrization of the same argument",
      "Describes the pytest_generate_tests hook iterating over 'parametrize' markers on the test definition and calling metafunc.parametrize()",
      "Explains that -m markexpr is handled by deselect_by_mark() which compiles the expression via Expression.compile() into a code object and evaluates it against MarkMatcher instances",
      "Describes that MarkMatcher.from_markers() builds a mapping of mark names to Mark objects for evaluation",
      "Explains that --strict-markers causes MarkGenerator.__getattr__() to fail if the mark is not in the 'markers' ini-config, otherwise only a warning is issued",
      "Describes that --durations=N collects reports with duration attributes, sorts descending, truncates to N (0=all), filters by durations_min threshold, and prints phase/nodeid",
      "Mentions that duration is measured using timing.Instant() in CallInfo.from_call() and transferred to TestReport",
      "Explains that eager teardown is implemented in SetupState.teardown_exact(nextitem), which tears down only nodes not shared with nextitem's ancestor collectors"
    ],
    "key_files": [
      "src/_pytest/python.py",
      "src/_pytest/runner.py",
      "src/_pytest/mark/__init__.py",
      "src/_pytest/mark/structures.py",
      "src/_pytest/mark/expression.py"
    ],
    "source_doc": "[doc_file: doc/en/announce/release-2.2.0.rst] py.test 2.2.0: test marking++, parametrization++ and duration profiling\npytest-2.2.0 is a test-suite compatible release of the popular\npy.test testing tool.  Plugins might need upgrades. It comes\nwith these improvements:\n\n* easier and more powerful parametrization of tests:\n\n  - new @pytest.mark.parametrize decorator to run tests with different arguments\n  - new metafunc.parametrize() API for parametrizing arguments independently\n  - see examples at http://pytest.org/en/stable/example/how-to/parametrize.html\n  - NOTE that parametrize() related APIs are still a bit experimental\n    and might change in future releases.\n\n* improved handling of test markers and refined marking mechanism:\n\n  - \"-m markexpr\" option for selecting tests according to their mark\n  - a new \"markers\" ini-variable for registering test markers for your project\n  - the new \"--strict\" bails out with an error if using unregistered markers.\n  - see examples at http://pytest.org/en/stable/example/markers.html\n\n* duration profiling: new \"--duration=N\" option showing the N slowest test\n  execution or setup/teardown calls. This is most useful if you want to\n  find out where your slowest test code is.\n\n* also 2.2.0 performs more eager calling of teardown/finalizers functions\n  resulting in better and more accurate reporting when they fail\n\nBesides there is the usual set of bug fixes along with a cleanup of\npytest's own test suite allowing it to run on a wider range of environments.\n\nFor general information, see extensive docs with examples here:\n\n     http://pytest.org/\n\nIf you want to install or upgrade pytest you might just type::\n\n    pip install -U pytest # or\n    easy_install -U pytest\n\nThanks to Ronny Pfannschmidt, David Burns, Jeff Donner, Daniel Nouri, Alfredo Deza and all who gave feedback or sent bug reports.\n\nbest,\nholger krekel",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals of line numbers and one design rationale claim.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_01",
    "repo": "astropy",
    "question": "What is the architecture of the `galactocentric_frame_defaults` class and how does it integrate with the `Galactocentric` frame?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `galactocentric_frame_defaults` class is a singleton class (inheriting from `ScienceState`) that manages global default parameters for the `Galactocentric` coordinate frame. Here is a detailed architectural overview based on the actual source code:\n\n**Class Hierarchy and Singleton Pattern:**\n- `galactocentric_frame_defaults` inherits from `ScienceState` (in `astropy/utils/state.py`), which raises `RuntimeError(\"This class is a singleton. Do not instantiate.\")` in `__init__`. All methods are `@classmethod`.\n- `ScienceState.set()` creates a `_ScienceStateContext` object that stores the previous `_value` and restores it on `__exit__`, enabling context manager usage. The `__enter__` method does nothing (returns `None` implicitly).\n- `ScienceState.get()` calls `cls.validate(cls._value)`, meaning validation runs on every `.get()` call, not just on `.set()`.\n\n**Registry Design with `_StateProxy`:**\n- The `_registry` class attribute is a dict keyed by parameter set names ('v4.0', 'pre-v4.0'). Each entry contains 'parameters' and 'references' sub-dicts wrapped in `_StateProxy` objects.\n- `_StateProxy` inherits from `collections.abc.MappingView` and provides read-only access via `types.MappingProxyType`. Critically, its `__deepcopy__` method returns `copy.deepcopy(self._mapping, memo=memo)` \u2014 this means `get_from_registry` can return a mutable deep copy of the originally read-only data.\n- The `_latest_value` class attribute is hardcoded to `\"v4.0\"`, and `_value` defaults to `None`. When `validate` receives `None`, it resolves to `_latest_value`.\n\n**Specific Parameter Values:**\n- For 'v4.0': `galcen_distance=8.122 kpc`, `galcen_v_sun=[12.9, 245.6, 7.78] km/s`, `z_sun=20.8 pc`.\n- For 'pre-v4.0': `galcen_distance=8.3 kpc`, `galcen_v_sun=[11.1, 232.24, 7.25] km/s` (computed as `[11.1, 220 + 12.24, 7.25]` in source), `z_sun=27.0 pc`.\n- Both parameter sets share the same `galcen_coord`: ICRS at `ra=266.4051\u00b0, dec=-28.936175\u00b0`.\n- `galcen_v_sun` is stored as a `CartesianRepresentation` (not `CartesianDifferential`), using `r.CartesianRepresentation`.\n- The module-level `_ROLL0 = Angle(58.5986320306 * u.degree)` is accessed via `Galactocentric.get_roll0()` and used in the ICRS\u2194Galactocentric transformation matrix.\n\n**`Galactocentric` Frame Integration:**\n- `Galactocentric` inherits from `BaseCoordinateFrame` with `default_representation = r.CartesianRepresentation` and `default_differential = r.CartesianDifferential`.\n- Its `__init__` calls `galactocentric_frame_defaults.get()` to retrieve default parameters, then creates `frame_attribute_references` by copying `galactocentric_frame_defaults.references`. If a user explicitly provides a frame attribute as a kwarg, its reference is removed from `frame_attribute_references` via `.pop(k, None)`.\n- Frame attributes are defined as descriptor instances: `galcen_coord` as `CoordinateAttribute(frame=ICRS)`, `galcen_distance` as `QuantityAttribute(unit=u.kpc)`, `galcen_v_sun` as `CartesianRepresentationAttribute(unit=u.km/u.s)`, `z_sun` as `QuantityAttribute(unit=u.pc)`, and `roll` as `QuantityAttribute(unit=u.deg)`.\n\n**Validate Method Polymorphism:**\n- `validate` accepts three types: `str` (looks up registry), `dict` (uses directly as parameters), or a `Galactocentric` instance (extracts frame attributes using `getattr(value, k)` for each key in `frame_attributes`).\n- For `Galactocentric` instance input, it also copies `frame_attribute_references` to `cls._references` and builds `cls._state`.\n- For any other type, it raises `ValueError`.\n\n**Registration Mechanism:**\n- `register()` validates that the `parameters` dict contains the required keys `{\"galcen_coord\", \"galcen_distance\", \"galcen_v_sun\", \"z_sun\", \"roll\"}` and raises `ValueError` with the missing keys if any are absent.\n- `references=None` is converted to `{}`. Additional keyword arguments (`**meta`) are added to the state dict.\n- There is no protection against overwriting existing registry entries.\n\n**Transform Architecture:**\n- ICRS\u2194Galactocentric transforms use `AffineTransform`, registered via `@frame_transform_graph.transform()` decorators.\n- The `get_matrix_vectors` function computes a rotation matrix `A = H @ mat0 @ mat1 @ mat2` using galactic center coordinates and roll, plus a translation offset incorporating `galcen_distance` and `z_sun`.\n- A loopback (Galactocentric\u2192Galactocentric) transform is created via `frame_transform_graph._add_merged_transform(Galactocentric, ICRS, Galactocentric)`.\n\n**Notable:** `galactocentric_frame_defaults` is NOT in the module's `__all__` (which only contains `[\"Galactocentric\"]`), but is explicitly imported in `builtin_frames/__init__.py`. The `parameters` and `references` properties are read-only `@classproperty` decorators (from `astropy.utils.decorators`), which only support read-only access.",
    "rubric": [
      "Explains that galactocentric_frame_defaults inherits from ScienceState, which is a singleton (raises RuntimeError on instantiation) with all classmethod-based interface",
      "Describes the _registry dict structure keyed by parameter set names ('v4.0', 'pre-v4.0') containing 'parameters' and 'references' sub-dicts wrapped in _StateProxy objects",
      "Explains that _StateProxy inherits from MappingView, provides read-only access via MappingProxyType, but its __deepcopy__ returns a mutable deep copy of the underlying mapping",
      "Describes the validate method's polymorphism: accepts str (registry lookup), dict (direct use), or Galactocentric instance (extracts frame attributes), raises ValueError otherwise",
      "Explains that ScienceState.set() creates a _ScienceStateContext that saves the previous _value and restores it on __exit__, enabling context manager usage",
      "Explains that ScienceState.get() calls cls.validate(cls._value) on every call, and when _value is None it resolves to _latest_value ('v4.0')",
      "Describes how Galactocentric.__init__ calls galactocentric_frame_defaults.get() for defaults and copies references, removing references for explicitly-provided kwargs via .pop(k, None)",
      "Mentions the register() method validates required keys {'galcen_coord', 'galcen_distance', 'galcen_v_sun', 'z_sun', 'roll'} and raises ValueError if any are missing",
      "Notes that ICRS\u2194Galactocentric transforms use AffineTransform with get_matrix_vectors computing rotation matrices from galactic center coordinates and a loopback transform via _add_merged_transform",
      "Mentions specific parameter values (e.g., v4.0: galcen_distance=8.122 kpc, galcen_v_sun=[12.9, 245.6, 7.78] km/s) and that galcen_v_sun is stored as CartesianRepresentation"
    ],
    "key_files": [
      "astropy/coordinates/builtin_frames/galactocentric.py",
      "astropy/utils/state.py",
      "astropy/coordinates/attributes.py",
      "astropy/coordinates/baseframe.py"
    ],
    "source_doc": "[docstring: astropy/coordinates/builtin_frames/galactocentric.py] astropy.coordinates.builtin_frames.galactocentric.galactocentric_frame_defaults\nastropy.coordinates.builtin_frames.galactocentric.galactocentric_frame_defaults:\n  Global setting of default values for the frame attributes in the `~astropy.coordinates.Galactocentric` frame.\n\n    These constancts may be updated in future versions of ``astropy``. Note\n    that when using `~astropy.coordinates.Galactocentric`, changing values\n    here will not affect any attributes that are set explicitly by passing\n    values in to the `~astropy.coordinates.Galactocentric`\n    initializer. Modifying these defaults will only affect the frame attribute\n    values when using the frame as, e.g., ``Galactocentric`` or\n    ``Galactocentric()`` with no explicit arguments.\n\n    This class controls the parameter settings by specifying a string name,\n    with the following pre-specified options:\n\n    - 'pre-v4.0': The current default value, which sets the default frame\n      attribute values to their original (pre-astropy-v4.0) values.\n    - 'v4.0': The attribute values as updated in Astropy version 4.0.\n    - 'latest': An alias of the most recent parameter set (currently: 'v4.0')\n\n    Alternatively, user-defined parameter settings may be registered, with\n    :meth:`~astropy.coordinates.galactocentric_frame_defaults.register`,\n    and used identically as pre-specified parameter sets. At minimum,\n    registrations must have unique names and a dictionary of parameters\n    with keys \"galcen_coord\", \"galcen_distance\", \"galcen_v_sun\", \"z_sun\",\n    \"roll\". See examples below.\n\n    This class also tracks the references for all parameter values in the\n    attribute ``references``, as well as any further information the registry.\n    The pre-specified options can be extended to include similar\n    state information as user-defined parameter settings -- for example, to add\n    parameter uncertainties.\n\n    The preferred method for getting a parameter set and metadata, by name, is\n    :meth:`~astropy.coordinates.galactocentric_frame_defaults.get_from_registry`\n    since it ensures the immutability of the registry.\n\n    See :ref:`astropy:astropy-coordinates-galactocentric-defaults` for more\n    information.\n\n    Examples\n    --------\n    The default `~astropy.coordinates.Galactocentric` frame parameters can be\n    modified globally::\n\n        >>> from astropy.coordinates import galactocentric_frame_defaults\n        >>> _ = galactocentric_frame_defaults.set('v4.0') # doctest: +SKIP\n        >>> Galactocentric() # doctest: +SKIP\n        <Galactocentric Frame (galcen_coord=<ICRS Coordinate: (ra, dec) in deg\n            (266.4051, -28.936175)>, galcen_distance=8.122 kpc, galcen_v_sun=(12.9, 245.6, 7.78) km / s, z_sun=20.8 pc, roll=0.0 deg)>\n        >>> _ = galactocentric_frame_defaults.set('pre-v4.0') # doctest: +SKIP\n        >>> Galactocentric() # doctest: +SKIP\n        <Galactocentric Frame (galcen_coord=<ICRS Coordinate: (ra, dec) in deg\n            (266.4051, -28.936175)>, galcen_distance=8.3 kpc, galcen_v_sun=(11.1, 232.24, 7.25) km / s, z_sun=27.0 pc, roll=0.0 deg)>\n\n    The default parameters can also be updated by using this class as a context\n    manager::\n\n        >>> with galactocentric_frame_defaults.set('pre-v4.0'):\n        ...     print(Galactocentric()) # doctest: +FLOAT_CMP\n        <Galactocentric Frame (galcen_coord=<ICRS Coordinate: (ra, dec) in deg\n            (266.4051, -28.936175)>, galcen_distance=8.3 kpc, galcen_v_sun=(11.1, 232.24, 7.25) km / s, z_sun=27.0 pc, roll=0.0 deg)>\n\n    Again, changing the default parameter values will not affect frame\n    attributes that are explicitly specified::\n\n        >>> import astropy.units as u\n        >>> with galactocentric_frame_defaults.set('pre-v4.0'):\n        ...     print(Galactocentric(galcen_distance=8.0*u.kpc)) # doctest: +FLOAT_CMP\n        <Galactocentric Frame (galcen_coord=<ICRS Coordinate: (ra, dec) in deg\n            (266.4051, -28.936175)>, galcen_distance=8.0 kpc, galcen_v_sun=(11.1, 232.24, 7.25) km / s, z_sun=27.0 pc, roll=0.0 deg)>\n\n    Additional parameter sets may be registered, for instance to use the\n    Dehnen & Binney (1998) measurements of the solar motion. We can also\n    add metadata, such as the 1-sigma errors. In this example we will modify\n    the required key \"parameters\", change the recommended key \"references\" to\n    match \"parameters\", and add the extra key \"error\" (any key can be added)::\n\n        >>> state = galactocentric_frame_defaults.get_from_registry(\"v4.0\")\n        >>> state[\"parameters\"][\"galcen_v_sun\"] = (10.00, 225.25, 7.17) * (u.km / u.s)\n        >>> state[\"references\"][\"galcen_v_sun\"] = \"https://ui.adsabs.harvard.edu/full/1998MNRAS.298..387D\"\n        >>> state[\"error\"] = {\"galcen_v_sun\": (0.36, 0.62, 0.38) * (u.km / u.s)}\n        >>> galactocentric_frame_defaults.register(name=\"DB1998\", **state)\n\n    Just as in the previous examples, the new parameter set can be retrieved with::\n\n        >>> state = galactocentric_frame_defaults.get_from_registry(\"DB1998\")\n        >>> print(state[\"error\"][\"galcen_v_sun\"])  # doctest: +FLOAT_CMP\n        [0.36 0.62 0.38] km / s\n\nastropy.coordinates.builtin_frames.galactocentric.galactocentric_frame_defaults.get_from_registry:\n  Return Galactocentric solar parameters and metadata given string names\n        for the parameter sets. This method ensures the returned state is a\n        mutable copy, so any changes made do not affect the registry state.\n\n        Returns\n        -------\n        state : dict\n            Copy of the registry for the string name.\n            Should contain, at minimum:\n\n            - \"parameters\": dict\n                Galactocentric solar parameters\n            - \"references\" : Dict[str, Union[str, Sequence[str]]]\n                References for \"parameters\".\n                Fields are str or sequence of str.\n\n        Raises\n        ------\n        KeyError\n            If invalid string input to registry\n            to retrieve solar parameters for Galactocentric frame.\n\nastropy.coordinates.builtin_frames.galactocentric.galactocentric_frame_defaults.register:\n  Register a set of parameters.\n\n        Parameters\n        ----------\n        name : str\n            The registration name for the parameter and metadata set.\n        parameters : dict\n            The solar parameters for Galactocentric frame.\n        references : dict or None, optional\n            References for contents of `parameters`.\n            None becomes empty dict.\n        **meta : dict, optional\n            Any other properties to register.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 'pre-v4.0' is described as 'The current default value' in the documentation, which matches the doc's description of 'pre-v4.0' as 'The current default value, which sets the default frame attribute values to their original (pre-astropy-v4.0) values.' However, the answer also states '_latest_value' is hardcoded to 'v4.0' and that when validate receives None it resolves to '_latest_value'. This creates a tension: the doc says 'latest' is an alias for 'v4.0', and 'pre-v4.0' is 'the current default value'. The answer's claim about _latest_value='v4.0' is consistent with 'latest' aliasing 'v4.0', but the relationship to the actual default is ambiguous.",
      "The answer makes many detailed implementation claims (e.g., _StateProxy inheriting from collections.abc.MappingView, using types.MappingProxyType, __deepcopy__ implementation, _ROLL0 = Angle(58.5986320306 * u.degree), H @ mat0 @ mat1 @ mat2 rotation matrix, frame_transform_graph._add_merged_transform, @classproperty decorators, etc.) that go well beyond what the documentation states. These are plausible code-level details but cannot be verified from the provided documentation.",
      "The answer states 'There is no protection against overwriting existing registry entries' - this is not mentioned in the documentation and cannot be verified from it.",
      "The answer claims galactocentric_frame_defaults is NOT in __all__ - this cannot be verified from the documentation provided.",
      "The answer claims ScienceState raises RuntimeError('This class is a singleton. Do not instantiate.') in __init__ - not verifiable from the docs provided."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_02",
    "repo": "astropy",
    "question": "Why does BaseRepresentationOrDifferential make so many unusual design choices \u2014 like bypassing __init__ in _apply, using identity checks in __eq__, and setting __array_priority__ so high?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `BaseRepresentationOrDifferential` class uses several deliberate design decisions:\n\n1. **`__array_priority__ = 50000`**: This is set as `__array_priority__: Final = 50000`. The class also defines `__mul__`, `__rmul__`, and `__truediv__` methods that delegate to `_scale_operation`, ensuring that when a representation interacts with other array-like objects in operations, the representation's methods take precedence.\n\n2. **`_apply` bypasses `__init__` via `super().__new__(self.__class__)`**: The `_apply` method creates a new instance using `super().__new__(self.__class__)` instead of calling the constructor, avoiding re-running the `__init__` logic (broadcasting, validation, copying, attr_classes initialization). The method directly sets `_`-prefixed private attributes on the new object by iterating over `self.components`.\n\n3. **`__eq__` uses `self.__class__ is not value.__class__` (identity, not isinstance)**: The equality operator uses `is` rather than `isinstance` to enforce strict class identity. This means different representation subclasses cannot be compared even though both inherit from the same base class.\n\n4. **`__eq__` also calls `np.broadcast(self, value)` before comparison**: This pre-checks that the shapes are broadcastable and raises a `ValueError` if not, before proceeding with component-wise comparison.\n\n5. **Shape setter skips components with `val.size <= 1`**: The shape setter skips reshaping scalar/size-1 components (via `if val.size <= 1: continue`). It also implements rollback logic \u2014 if reshaping one component fails, it restores all previously reshaped components to their original shape.\n\n6. **Broadcasting handling in `__init__`**: The `__init__` method calls `np.broadcast_arrays(*attrs, subok=True)` and then applies the logic `(bc_attr.copy() if copy else bc_attr) if bc_attr.shape != attr.shape else attr` \u2014 when `copy=True` and the shape changed due to broadcasting, it makes a copy of the broadcasted array.\n\n7. **`_scale_operation` in BaseRepresentation preserves angular components**: When scaling (multiply/divide), only non-angular components are operated on \u2014 components whose `attr_classes` value is a subclass of `Angle` are left unchanged.\n\n8. **`BaseDifferential.__init_subclass__` auto-generates `attr_classes`**: If a differential subclass doesn't define `attr_classes`, it is automatically created by prefixing base representation component names with `d_` and setting all types to `u.Quantity` (not the specific angle classes used by the base representation).\n\n9. **Masking consistency across components**: When any component has a mask (`hasattr(attr, 'mask')`), `_ensure_masked()` wraps all components in `Masked`. The `__setitem__` does NOT auto-mask when setting with a masked value \u2014 only when explicitly assigning `np.ma.masked` (checked via `value is np.ma.masked`).\n\n10. **Duplicate representation handling**: When two representations register with the same name, the first time a duplicate is detected, the existing entry is removed from `REPRESENTATION_CLASSES` and re-registered using the fully-qualified name (`_fqn_class` returns `cls.__module__ + '.' + cls.__qualname__`). The new class is also registered with its fully-qualified name. A `DuplicateRepresentationWarning` is issued. The `get_reprdiff_cls_hash` function uses `@functools.cache` and computes a hash from the current state of `REPRESENTATION_CLASSES` and `DIFFERENTIAL_CLASSES`.",
    "rubric": [
      "__array_priority__ is set to 50000 to ensure representation's __mul__/__rmul__/__truediv__ (which delegate to _scale_operation) take precedence over ndarray/Quantity in mixed operations",
      "_apply bypasses __init__ by using super().__new__(self.__class__) and directly sets _-prefixed private attributes, avoiding re-running broadcasting, validation, and attr_classes initialization",
      "__eq__ uses 'is' identity check (self.__class__ is not value.__class__) rather than isinstance, enforcing strict class identity so different subclasses cannot be compared",
      "__eq__ calls np.broadcast(self, value) to pre-check shape compatibility before doing component-wise comparison",
      "The shape setter skips components with val.size <= 1 and implements rollback logic to restore previously reshaped components if a later reshape fails",
      "__init__ broadcasts components via np.broadcast_arrays and conditionally copies: (bc_attr.copy() if copy else bc_attr) if shape changed, else uses original attr",
      "_scale_operation in BaseRepresentation preserves angular components (subclasses of Angle) unchanged, only operating on non-angular ones",
      "BaseDifferential.__init_subclass__ auto-generates attr_classes by prefixing base component names with 'd_' and setting all types to u.Quantity instead of the base's angle classes",
      "When any component has a mask, _ensure_masked wraps all components in Masked; __setitem__ only auto-masks when value is literally np.ma.masked, not for any masked value",
      "Duplicate representation names are handled by re-registering both old and new classes under fully-qualified names (_fqn_class), with a DuplicateRepresentationWarning"
    ],
    "key_files": [
      "astropy/coordinates/representation/base.py",
      "astropy/utils/masked/core.py",
      "astropy/utils/shapes.py",
      "astropy/coordinates/representation/cartesian.py"
    ],
    "source_doc": "[docstring: astropy/coordinates/representation/base.py] astropy.coordinates.representation.base.BaseRepresentationOrDifferential\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential:\n  3D coordinate representations and differentials.\n\n    Parameters\n    ----------\n    comp1, comp2, comp3 : `~astropy.units.Quantity` or subclass\n        The components of the 3D point or differential.  The names are the\n        keys and the subclasses the values of the ``attr_classes`` attribute.\n    copy : bool, optional\n        If `True` (default), arrays will be copied; if `False`, they will be\n        broadcast together but not use new memory.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.get_name:\n  Name of the representation or differential.\n\n        Returns the ``.name`` attribute.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.from_cartesian:\n  Create a representation of this class from a supplied Cartesian one.\n\n        Parameters\n        ----------\n        other : `~astropy.coordinates.CartesianRepresentation`\n            The representation to turn into this class\n\n        Returns\n        -------\n        representation : `~astropy.coordinates.BaseRepresentation` subclass instance\n            A new representation of this class's type.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.to_cartesian:\n  Convert the representation to its Cartesian form.\n\n        Note that any differentials get dropped.\n        Also note that orientation information at the origin is *not* preserved by\n        conversions through Cartesian coordinates. For example, transforming\n        an angular position defined at distance=0 through cartesian coordinates\n        and back will lose the original angular coordinates::\n\n            >>> import astropy.units as u\n            >>> import astropy.coordinates as coord\n            >>> rep = coord.SphericalRepresentation(\n            ...     lon=15*u.deg,\n            ...     lat=-11*u.deg,\n            ...     distance=0*u.pc)\n            >>> rep.to_cartesian().represent_as(coord.SphericalRepresentation)\n            <SphericalRepresentation (lon, lat, distance) in (rad, rad, pc)\n                (0., 0., 0.)>\n\n        Returns\n        -------\n        cartrepr : `~astropy.coordinates.CartesianRepresentation`\n            The representation in Cartesian form.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.components:\n  A tuple with the in-order names of the coordinate components.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.__eq__:\n  Equality operator.\n\n        This implements strict equality and requires that the representation\n        classes are identical and that the representation data are exactly equal.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential._apply:\n  Create a new representation or differential with ``method`` applied\n        to the component data.\n\n        In typical usage, the method is any of the shape-changing methods for\n        `~numpy.ndarray` (``reshape``, ``swapaxes``, etc.), as well as those\n        picking particular elements (``__getitem__``, ``take``, etc.), which\n        are all defined in `~astropy.utils.shapes.ShapedLikeNDArray`. It will be\n        applied to the underlying arrays (e.g., ``x``, ``y``, and ``z`` for\n        `~astropy.coordinates.CartesianRepresentation`), with the results used\n        to create a new instance.\n\n        Internally, it is also used to apply functions to the components\n        (in particular, `~numpy.broadcast_to`).\n\n        Parameters\n        ----------\n        method : str or callable\n            If str, it is the name of a method that is applied to the internal\n            ``components``. If callable, the function is applied.\n        *args : tuple\n            Any positional arguments for ``method``.\n        **kwargs : dict\n            Any keyword arguments for ``method``.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.shape:\n  The shape of the instance and underlying arrays.\n\n        Like `~numpy.ndarray.shape`, can be set to a new shape by assigning a\n        tuple.  Note that if different instances share some but not all\n        underlying data, setting the shape of one instance can make the other\n        instance unusable.  Hence, it is strongly recommended to get new,\n        reshaped instances with the ``reshape`` method.\n\n        Raises\n        ------\n        ValueError\n            If the new shape has the wrong total number of elements.\n        AttributeError\n            If the shape of any of the components cannot be changed without the\n            arrays being copied.  For these cases, use the ``reshape`` method\n            (which copies any arrays that cannot be reshaped in-place).\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential._ensure_masked:\n  Ensure Masked components.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential.get_mask:\n  Calculate the mask, by combining masks from the given attributes.\n\n        Parameters\n        ----------\n        *attrs : str\n            Attributes from which to get the masks to combine. If not given,\n            use all components of the class.\n\n        Returns\n        -------\n        mask : ~numpy.ndarray of bool\n            The combined, read-only mask. If the instance is not masked, it\n            is an array of `False` with the correct shape.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential._values:\n  Turn the coordinates into a record array with the coordinate values.\n\n        The record array fields will have the component names.\n\nastropy.coordinates.representation.base.BaseRepresentationOrDifferential._units:\n  Return a dictionary with the units of the coordinate components.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer (points 1-10) go far beyond what is documented in the provided documentation chunk. They appear to be derived from reading the actual source code rather than the docstrings provided.",
      "The documentation only provides high-level docstrings for methods like `__eq__`, `_apply`, `shape`, etc. The answer provides detailed implementation specifics (like `__array_priority__ = 50000`, `super().__new__()` usage, `np.broadcast()` calls, scalar component skipping, etc.) that cannot be verified from the documentation alone.",
      "Point 3 about `__eq__` using `is` rather than `isinstance` is consistent with the doc saying 'strict equality and requires that the representation classes are identical', but the specific implementation detail of using `is` is not in the docs.",
      "Point 5 about shape setter behavior is plausible given the documented raises (ValueError, AttributeError) but the specifics about scalar components are not documented.",
      "The documentation does say `__eq__` 'implements strict equality and requires that the representation classes are identical' which is consistent with but less specific than the answer's claim about identity checks.",
      "The documentation does say `_apply` applies methods to component data and creates new instances, which is consistent with but less specific than the answer's claims about bypassing __init__."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Most claims are verifiable from code but contain rationale/design-intent explanations that come from stripped documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_03",
    "repo": "astropy",
    "question": "Where does data flow through ERFA ufuncs in CartesianRepresentation's math operations, and how does the class hierarchy and caching mechanism support this?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `CartesianRepresentation` class has a specific data flow architecture centered around ERFA ufuncs (from `erfa.ufunc`) for its core mathematical operations.\n\n**Class hierarchy**: `CartesianRepresentation` \u2192 `BaseRepresentation` \u2192 `BaseRepresentationOrDifferential` \u2192 `MaskableShapedLikeNDArray` \u2192 `ShapedLikeNDArray` \u2192 `NDArrayShapeMethods`\n\n**Key data flow in `transform()`**:\n1. `self.get_xyz(xyz_axis=-1)` retrieves coordinates with the 3-component axis as the last axis\n2. `erfa_ufunc.rxp(matrix, p)` performs the matrix-vector multiplication\n3. A new `CartesianRepresentation` is created from the result with `xyz_axis=-1, copy=False`\n4. Each attached differential is transformed via `d.transform(matrix, self, rep)` where `self` is the original base and `rep` is the transformed base\n5. The result is assembled via `rep.with_differentials(new_diffs)`\n\n**Key data flow in `norm()`**: Uses `erfa_ufunc.pm()` on `self.get_xyz(xyz_axis=-1)`, overriding the base class implementation.\n\n**Key data flow in `dot()`**: Uses `erfa_ufunc.pdp()` on both vectors with `xyz_axis=-1`.\n\n**Key data flow in `cross()`**: Uses `erfa_ufunc.pxp()` and returns a new `CartesianRepresentation` from the result.\n\n**`CartesianDifferential.transform()`** ignores its `base` and `transformed_base` parameters entirely, and directly applies `erfa_ufunc.rxp` to its `get_d_xyz(xyz_axis=-1)` result.\n\n**The `_xyz` caching mechanism**: When a single array is passed to `__init__`, the `_xyz` attribute caches the full 3-coordinate array. `get_xyz()` returns this cached version directly if the requested `xyz_axis` matches `_xyz_axis`, or uses `np.moveaxis` otherwise. When individual x, y, z are given, `_xyz` is `None` and `get_xyz()` uses `np.stack`.\n\n**`mean()` and `sum()`** both call `self._raise_if_has_differentials()` then delegate to `self._apply(\"mean\"/\"sum\", ...)`. The `_apply` method in `BaseRepresentationOrDifferential` applies the named method to each component individually and creates a new instance.\n\n**`_combine_operation` in `CartesianRepresentation`** first calls `_raise_if_has_differentials`, then converts `other` via `other.to_cartesian()`, and applies the operator component-by-component, returning `NotImplemented` if conversion fails.",
    "rubric": [
      "Identifies the class hierarchy: CartesianRepresentation \u2192 BaseRepresentation \u2192 BaseRepresentationOrDifferential \u2192 MaskableShapedLikeNDArray \u2192 ShapedLikeNDArray",
      "Explains transform() flow: get_xyz(xyz_axis=-1) \u2192 erfa_ufunc.rxp(matrix, p) \u2192 new CartesianRepresentation(p, xyz_axis=-1, copy=False), then transforms each attached differential via d.transform(matrix, self, rep) and assembles via rep.with_differentials(new_diffs)",
      "Explains norm() uses erfa_ufunc.pm() on get_xyz(xyz_axis=-1)",
      "Explains dot() uses erfa_ufunc.pdp() on both vectors' get_xyz(xyz_axis=-1)",
      "Explains cross() uses erfa_ufunc.pxp() and returns a new CartesianRepresentation from the result",
      "Explains CartesianDifferential.transform() ignores base and transformed_base parameters, directly applying erfa_ufunc.rxp to get_d_xyz(xyz_axis=-1)",
      "Describes _xyz caching: when a single array is passed, _xyz caches the full 3-coordinate array; get_xyz returns it directly if xyz_axis matches _xyz_axis, otherwise uses np.moveaxis; when x,y,z are given separately, _xyz is None and get_xyz uses np.stack",
      "Explains mean() and sum() call _raise_if_has_differentials then delegate to _apply which applies the method to each component individually",
      "Explains _combine_operation first raises if has differentials, converts other via to_cartesian(), applies operator component-by-component, returns NotImplemented on failure"
    ],
    "key_files": [
      "astropy/coordinates/representation/cartesian.py",
      "astropy/coordinates/representation/base.py",
      "astropy/utils/masked/core.py"
    ],
    "source_doc": "[docstring: astropy/coordinates/representation/cartesian.py] astropy.coordinates.representation.cartesian.CartesianRepresentation\nastropy.coordinates.representation.cartesian.CartesianRepresentation:\n  Representation of points in 3D cartesian coordinates.\n\n    Parameters\n    ----------\n    x, y, z : `~astropy.units.Quantity` or array\n        The x, y, and z coordinates of the point(s). If ``x``, ``y``, and ``z``\n        have different shapes, they should be broadcastable. If not quantity,\n        ``unit`` should be set.  If only ``x`` is given, it is assumed that it\n        contains an array with the 3 coordinates stored along ``xyz_axis``.\n    unit : unit-like\n        If given, the coordinates will be converted to this unit (or taken to\n        be in this unit if not given.\n    xyz_axis : int, optional\n        The axis along which the coordinates are stored when a single array is\n        provided rather than distinct ``x``, ``y``, and ``z`` (default: 0).\n\n    differentials : dict, `~astropy.coordinates.CartesianDifferential`, optional\n        Any differential classes that should be associated with this\n        representation. The input must either be a single\n        `~astropy.coordinates.CartesianDifferential` instance, or a dictionary of\n        `~astropy.coordinates.CartesianDifferential` s with keys set to a string representation of\n        the SI unit with which the differential (derivative) is taken. For\n        example, for a velocity differential on a positional representation, the\n        key would be ``'s'`` for seconds, indicating that the derivative is a\n        time derivative.\n\n    copy : bool, optional\n        If `True` (default), arrays will be copied. If `False`, arrays will\n        be references, though possibly broadcast to ensure matching shapes.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.get_xyz:\n  Return a vector array of the x, y, and z coordinates.\n\n        Parameters\n        ----------\n        xyz_axis : int, optional\n            The axis in the final array along which the x, y, z components\n            should be stored (default: 0).\n\n        Returns\n        -------\n        xyz : `~astropy.units.Quantity`\n            With dimension 3 along ``xyz_axis``.  Note that, if possible,\n            this will be a view.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.transform:\n  Transform the cartesian coordinates using a 3x3 matrix.\n\n        This returns a new representation and does not modify the original one.\n        Any differentials attached to this representation will also be\n        transformed.\n\n        Parameters\n        ----------\n        matrix : ndarray\n            A 3x3 transformation matrix, such as a rotation matrix.\n\n        Examples\n        --------\n        We can start off by creating a cartesian representation object:\n\n            >>> from astropy import units as u\n            >>> from astropy.coordinates import CartesianRepresentation\n            >>> rep = CartesianRepresentation([1, 2] * u.pc,\n            ...                               [2, 3] * u.pc,\n            ...                               [3, 4] * u.pc)\n\n        We now create a rotation matrix around the z axis:\n\n            >>> from astropy.coordinates import rotation_matrix\n            >>> rotation = rotation_matrix(30 * u.deg, axis='z')\n\n        Finally, we can apply this transformation:\n\n            >>> rep_new = rep.transform(rotation)\n            >>> rep_new.xyz  # doctest: +FLOAT_CMP\n            <Quantity [[ 1.8660254 , 3.23205081],\n                       [ 1.23205081, 1.59807621],\n                       [ 3.        , 4.        ]] pc>\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.norm:\n  Vector norm.\n\n        The norm is the standard Frobenius norm, i.e., the square root of the\n        sum of the squares of all components with non-angular units.\n\n        Note that any associated differentials will be dropped during this\n        operation.\n\n        Returns\n        -------\n        norm : `astropy.units.Quantity`\n            Vector norm, with the same shape as the representation.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.mean:\n  Vector mean.\n\n        Returns a new CartesianRepresentation instance with the means of the\n        x, y, and z components.\n\n        Refer to `~numpy.mean` for full documentation of the arguments, noting\n        that ``axis`` is the entry in the ``shape`` of the representation, and\n        that the ``out`` argument cannot be used.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.sum:\n  Vector sum.\n\n        Returns a new CartesianRepresentation instance with the sums of the\n        x, y, and z components.\n\n        Refer to `~numpy.sum` for full documentation of the arguments, noting\n        that ``axis`` is the entry in the ``shape`` of the representation, and\n        that the ``out`` argument cannot be used.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.dot:\n  Dot product of two representations.\n\n        Note that any associated differentials will be dropped during this\n        operation.\n\n        Parameters\n        ----------\n        other : `~astropy.coordinates.BaseRepresentation` subclass instance\n            If not already cartesian, it is converted.\n\n        Returns\n        -------\n        dot_product : `~astropy.units.Quantity`\n            The sum of the product of the x, y, and z components of ``self``\n            and ``other``.\n\nastropy.coordinates.representation.cartesian.CartesianRepresentation.cross:\n  Cross product of two representations.\n\n        Parameters\n        ----------\n        other : `~astropy.coordinates.BaseRepresentation` subclass instance\n            If not already cartesian, it is converted.\n\n        Returns\n        -------\n        cross_product : `~astropy.coordinates.CartesianRepresentation`\n            With vectors perpendicular to both ``self`` and ``other``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The class hierarchy (CartesianRepresentation \u2192 BaseRepresentation \u2192 BaseRepresentationOrDifferential \u2192 MaskableShapedLikeNDArray \u2192 ShapedLikeNDArray \u2192 NDArrayShapeMethods) is not mentioned in the documentation and cannot be verified from it, though it is plausible code-level detail.",
      "The use of ERFA ufuncs (erfa_ufunc.rxp, erfa_ufunc.pm, erfa_ufunc.pdp, erfa_ufunc.pxp) for transform, norm, dot, and cross operations is not mentioned in the documentation - these are implementation details that cannot be verified from the provided docs.",
      "The _xyz caching mechanism and internal details about _apply, _raise_if_has_differentials, _combine_operation are all implementation details not present in the documentation.",
      "The claim about CartesianDifferential.transform() ignoring base and transformed_base parameters is not verifiable from the documentation.",
      "The claim about BaseRepresentation.transform() converting to CartesianRepresentation first is not stated in the documentation.",
      "The documentation for transform() says 'This returns a new representation and does not modify the original one. Any differentials attached to this representation will also be transformed.' - the answer's description of differential transformation is more detailed than what the docs state but doesn't contradict it.",
      "The documentation for norm() says 'the square root of the sum of the squares of all components with non-angular units' - the answer claims it uses erfa_ufunc.pm() bypassing the base class implementation, which is an implementation detail not verifiable from docs but doesn't contradict the documented behavior.",
      "The answer's description of mean() and sum() behavior is consistent with the documentation which states they return new instances with means/sums of x, y, z components."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims relied on ERFA documentation descriptions and base class implementations not visible in stripped code; these were rewritten to include only code-verifiable facts.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_04",
    "repo": "astropy",
    "question": "How does the Cosmology class initialization and parameter system work, including subclass registration, equality vs equivalence checking, and the clone mechanism?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `Cosmology` class in `astropy/cosmology/_src/core.py` is implemented as a frozen dataclass with `ABCMeta` as its metaclass, using a custom `dataclass_decorator` that applies `dataclass(frozen=True, repr=True, eq=False, init=True)` followed by `_with_signature` which precomputes and caches `inspect.signature(cls)` on the class.\n\n**How the class initialization system works:**\n\n1. **Dataclass transformation**: The `@dataclass_decorator` applies Python's `@dataclass(frozen=True, repr=True, eq=False, init=True)` and then precomputes the class signature via `_with_signature`. The `eq=False` means equality checking is custom (via `__eq__`).\n\n2. **Field descriptors**: The `name` field uses a custom `_NameField` descriptor that converts values to `str` via `__set__` (storing in `instance._name`), and the `meta` field uses `MetaData` from `astropy.utils.metadata`. After class creation, `meta`'s dataclass field has `compare=False` and `repr=False` set explicitly.\n\n3. **Parameter registration in `__init_subclass__`**: When a subclass is created, `all_parameters(cls)` is called which walks the entire MRO in reverse, collecting all class variables that are `Parameter` instances or `dataclass.Field` objects containing a `Parameter` default. These are split into `_parameters` (non-derived) and `_parameters_derived` (derived), stored as `MappingProxyType` dicts.\n\n4. **Subclass registration**: Non-abstract subclasses call `_register_cls()` which adds the class to `_COSMOLOGY_CLASSES` dict (keyed by `__qualname__`) and registers YAML serialization via `register_cosmology_yaml(cls)`.\n\n5. **`clone` method**: Uses `dataclasses.replace(self, meta=self.meta | meta, **kwargs)` to create new instances. If no actual change occurred (checked by `is_equivalent`), the name is reverted. Returns `self` (not a copy) if `meta is None and not kwargs`.\n\n6. **`is_equivalent` delegates to `cosmology_equal`** (in `astropy/cosmology/_src/funcs/comparison.py`) with `allow_equivalent=True`. This function calls `cosmo1.__equiv__(cosmo2)` and if it returns `NotImplemented`, tries `cosmo2.__equiv__(cosmo1)`. Any exception results in `False`.\n\n7. **`__equiv__` vs `__eq__`**: Both use `np.all(getattr(self, k) == getattr(other, k))` for parameter comparison. `__eq__` additionally checks `self.name == other.name`, while `__equiv__` does not. Both return `NotImplemented` if classes differ.\n\n8. **`FlatCosmologyMixin.__equiv__`**: Extends equivalence to the non-flat parent class by checking `issubclass(other.__class__, self.__nonflatclass__)`, comparing parameters, AND verifying `other.is_flat`.\n\n9. **`FlatCosmologyMixin._get_nonflat_cls`**: Recursively searches class bases for a `Cosmology` subclass that is NOT a `FlatCosmologyMixin` subclass. Raises `TypeError` if multiple non-flat classes found at the same level.\n\n10. **`Parameter.__set__`**: Values are validated via `self.validate(cosmology, copy.deepcopy(value))`, then made read-only with `value.setflags(write=False)` if ndarray-like, and stored in `cosmology.__dict__[self.name]`.\n\n11. **`ParametersAttribute` descriptor**: When accessed from an instance, it dynamically builds a `MappingProxyType` mapping parameter names to their current values (via `getattr`). When accessed from the class, it returns the class-level mapping of `Parameter` objects.",
    "rubric": [
      "Explains that dataclass_decorator applies dataclass(frozen=True, repr=True, eq=False, init=True) and then _with_signature to precompute/cache inspect.signature on the class",
      "Describes the _NameField descriptor converting values to str via __set__ and storing in instance._name",
      "Explains that __init_subclass__ calls all_parameters(cls) which walks the MRO in reverse collecting Parameter instances and dataclass Fields with Parameter defaults, splitting into _parameters (non-derived) and _parameters_derived (derived) as MappingProxyType dicts",
      "Describes that non-abstract subclasses are registered via _register_cls() into _COSMOLOGY_CLASSES dict and YAML serialization",
      "Explains clone uses dataclasses.replace and returns self (not a copy) if meta is None and no kwargs, and reverts name if is_equivalent shows no change",
      "Describes is_equivalent delegating to cosmology_equal with allow_equivalent=True, which calls __equiv__ on both sides if first returns NotImplemented",
      "Explains the difference between __eq__ (checks name + parameters) and __equiv__ (checks only parameters), both returning NotImplemented for different classes",
      "Describes FlatCosmologyMixin.__equiv__ extending equivalence to the non-flat parent class by checking issubclass against __nonflatclass__, comparing parameters, and verifying other.is_flat",
      "Explains Parameter.__set__ validates via self.validate(cosmology, copy.deepcopy(value)), makes ndarrays read-only with setflags(write=False), and stores in cosmology.__dict__",
      "Describes ParametersAttribute descriptor returning Parameter objects from the class and a MappingProxyType of current values from an instance"
    ],
    "key_files": [
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/parameter/core.py",
      "astropy/cosmology/_src/funcs/comparison.py",
      "astropy/cosmology/_src/parameter/descriptors.py",
      "astropy/cosmology/_src/parameter/utils.py",
      "astropy/cosmology/_src/flrw/base.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/core.py] astropy.cosmology._src.core.Cosmology\nastropy.cosmology._src.core.Cosmology:\n  Base-class for all Cosmologies.\n\n    Parameters\n    ----------\n    *args\n        Arguments into the cosmology; used by subclasses, not this base class.\n    name : str or None (optional, keyword-only)\n        The name of the cosmology.\n    meta : dict or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n    **kwargs\n        Arguments into the cosmology; used by subclasses, not this base class.\n\n    Notes\n    -----\n    Class instances are static -- you cannot (and should not) change the\n    values of the parameters.  That is, all of the above attributes\n    (except meta) are read only.\n\n    For details on how to create performant custom subclasses, see the\n    documentation on :ref:`astropy-cosmology-fast-integrals`.\n\n    Cosmology subclasses are automatically registered in a global registry\n    and with various I/O methods. To turn off or change this registration,\n    override the ``_register_cls`` classmethod in the subclass.\n\nastropy.cosmology._src.core.Cosmology.__post_init__:\n  Post-initialization, for subclasses to override if they need.\n\n        This allows for subclasses to call ``super().__post_init__()`` without worrying\n        about whether the parent class has a ``__post_init__`` method.\n\nastropy.cosmology._src.core.Cosmology.is_flat:\n  Return bool; `True` if the cosmology is flat.\n\n        This is abstract and must be defined in subclasses.\n\nastropy.cosmology._src.core.Cosmology.clone:\n  Returns a copy of this object with updated parameters, as specified.\n\n        In general this returns an instance of the same class. For some classes (e.g.\n        flat classes like `FlatLambdaCDM`) there are more options, which are documented\n        in the relevant subclass.\n\n        Parameters\n        ----------\n        meta : mapping or None (optional, keyword-only)\n            Metadata that will update the current metadata.\n        **kwargs\n            Cosmology parameter (and name) modifications. If any parameter is\n            changed and a new name is not given, the name will be set to \"[old\n            name] (modified)\".\n\n        Returns\n        -------\n        newcosmo : `~astropy.cosmology.Cosmology` subclass instance\n            A new instance of this (or non-flat equivalent) class with updated\n            parameters as specified. If no arguments are given, then a\n            reference to this object is returned instead of a copy.\n\n        Examples\n        --------\n        To make a copy of the ``Planck13`` cosmology with a different matter\n        density (``Om0``), and a new name:\n\n            >>> from astropy.cosmology import Planck13\n            >>> Planck13.clone(name=\"Modified Planck 2013\", Om0=0.35)\n            FlatLambdaCDM(name='Modified Planck 2013', H0=<Quantity 67.77 km / (Mpc s)>,\n                          Om0=0.35, ...\n\n        If no name is specified, the new name will note the modification.\n\n            >>> Planck13.clone(Om0=0.35).name\n            'Planck13 (modified)'\n\nastropy.cosmology._src.core.Cosmology.is_equivalent:\n  Check equivalence between Cosmologies.\n\n        Two cosmologies may be equivalent even if not the same class.\n        For example, an instance of ``LambdaCDM`` might have :math:`\\Omega_0=1`\n        and :math:`\\Omega_k=0` and therefore be flat, like ``FlatLambdaCDM``.\n\n        Parameters\n        ----------\n        other : `~astropy.cosmology.Cosmology` subclass instance, positional-only\n            The object to which to compare.\n        format : bool or None or str, optional keyword-only\n            Whether to allow, before equivalence is checked, the object to be\n            converted to a |Cosmology|. This allows, e.g. a |Table| to be\n            equivalent to a Cosmology.\n            `False` (default) will not allow conversion. `True` or `None` will,\n            and will use the auto-identification to try to infer the correct\n            format. A `str` is assumed to be the correct format to use when\n            converting.\n            ``format`` is broadcast to match the shape of ``other``.\n            Note that the cosmology arguments are not broadcast against\n            ``format``, so it cannot determine the output shape.\n\n        Returns\n        -------\n        bool\n            True if cosmologies are equivalent, False otherwise.\n\n        Examples\n        --------\n        Two cosmologies may be equivalent even if not of the same class.\n        In this examples the ``LambdaCDM`` has ``Ode0`` set to the same value\n        calculated in ``FlatLambdaCDM``.\n\n            >>> import astropy.units as u\n            >>> from astropy.cosmology import LambdaCDM, FlatLambdaCDM\n            >>> cosmo1 = LambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3, 0.7)\n            >>> cosmo2 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3)\n            >>> cosmo1.is_equivalent(cosmo2)\n            True\n\n        While in this example, the cosmologies are not equivalent.\n\n            >>> cosmo3 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3, Tcmb0=3 * u.K)\n            >>> cosmo3.is_equivalent(cosmo2)\n            False\n\n        Also, using the keyword argument, the notion of equivalence is extended\n        to any Python object that can be converted to a |Cosmology|.\n\n            >>> from astropy.cosmology import Planck18\n            >>> tbl = Planck18.to_format(\"astropy.table\")\n            >>> Planck18.is_equivalent(tbl, format=True)\n            True\n\n        The list of valid formats, e.g. the |Table| in this example, may be\n        checked with ``Cosmology.from_format.list_formats()``.\n\n        As can be seen in the list of formats, not all formats can be\n        auto-identified by ``Cosmology.from_format.registry``. Objects of\n        these kinds can still be checked for equivalence, but the correct\n        format string must be used.\n\n            >>> tbl = Planck18.to_format(\"yaml\")\n            >>> Planck18.is_equivalent(tbl, format=\"yaml\")\n            True\n\nastropy.cosmology._src.core.Cosmology.__equiv__:\n  Cosmology equivalence. Use ``.is_equivalent()`` for actual check!\n\n        Parameters\n        ----------\n        other : Any, positional-only\n            The object in which to compare.\n\n        Returns\n        -------\n        bool or `NotImplemented`\n            `NotImplemented` if ``other`` is from a different class.\n            `True` if ``other`` is of the same class and has matching parameters\n            and parameter values.\n            `False` otherwise.\n\nastropy.cosmology._src.core.Cosmology.__eq__:\n  Check equality between Cosmologies.\n\n        Checks the Parameters and immutable fields (i.e. not \"meta\").\n\n        Parameters\n        ----------\n        other : `~astropy.cosmology.Cosmology` subclass instance, positional-only\n            The object in which to compare.\n\n        Returns\n        -------\n        bool\n            `True` if Parameters and names are the same, `False` otherwise.\n\nastropy.cosmology._src.core.Cosmology.__str__:\n  Return a string representation of the cosmology.\n\nastropy.cosmology._src.core.Cosmology.__astropy_table__:\n  Return a `~astropy.table.Table` of type ``cls``.\n\n        Parameters\n        ----------\n        cls : type\n            Astropy ``Table`` class or subclass.\n        copy : bool\n            Ignored.\n        **kwargs : dict, optional\n            Additional keyword arguments. Passed to ``self.to_format()``.\n            See ``Cosmology.to_format.help(\"astropy.table\")`` for allowed kwargs.\n\n        Returns\n        -------\n        `astropy.table.Table` or subclass instance\n            Instance of type ``cls``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (dataclass_decorator, _with_signature, _NameField descriptor, MetaData OrderedDict, MappingProxyType, _COSMOLOGY_CLASSES dict, register_cosmology_yaml, YAML tags, dataclasses.replace usage, cosmology_equal delegation, np.all for parameter comparison, FlatCosmologyMixin.__equiv__, FlatCosmologyMixin._get_nonflat_cls, Parameter.__set__ with deepcopy and setflags, ParametersAttribute descriptor) that are not verifiable from the provided documentation alone - these appear to be code-level implementation details that may or may not be accurate.",
      "The claim that clone 'Returns self (not a copy) if meta is None and not kwargs' is consistent with the documentation which says 'If no arguments are given, then a reference to this object is returned instead of a copy.'",
      "The claim that __eq__ checks 'self.name == other.name' while __equiv__ does not is consistent with the documentation: __eq__ 'Checks the Parameters and immutable fields (i.e. not \"meta\")' which includes name, while __equiv__ checks 'matching parameters and parameter values' without mentioning name.",
      "The claim that __equiv__ returns NotImplemented 'if other is from a different class' is supported by the documentation.",
      "The claim about clone modifying the name to '[old name] (modified)' is supported by the documentation example showing 'Planck13 (modified)'.",
      "The claim about is_equivalent allowing format conversion is supported by the documentation.",
      "Many implementation details like exact line numbers (463-464), specific internal functions, and internal data structures cannot be verified from the documentation and could potentially be hallucinated or inaccurate."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: ~20x speedup rationale, OrderedDict default behavior of MetaData, YAML tag format, and 'to handle array parameters like m_nu' rationale are not recoverable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_05",
    "repo": "astropy",
    "question": "What is `FlatCosmologyMixin` and how do it and `FlatFLRWMixin` implement flatness handling, non-flat class resolution, equivalence checking, and cloning?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "FlatCosmologyMixin is a mixin class for flat cosmologies in astropy, defined in `astropy/cosmology/_src/core.py`. It is decorated with `@dataclass_decorator` (which makes it a frozen dataclass with `eq=False` and precomputes `__signature__`) and uses `ABCMeta` as its metaclass. Here are key implementation details:\n\n1. **`__nonflatclass__` is a lazy `classproperty`**: The `_get_nonflat_cls` classmethod is wrapped as `__nonflatclass__ = classproperty(_get_nonflat_cls, lazy=True, ...)`. The `lazy=True` parameter is passed to `classproperty`. It's eagerly triggered in `__init_subclass__` via `cls.__nonflatclass__` to ensure the non-flat class resolution is performed at class creation time.\n\n2. **`_get_nonflat_cls` algorithm**: It searches `_kls.__bases__` for classes that are subclasses of `Cosmology` but NOT subclasses of `FlatCosmologyMixin`. If none are found at the current level, it recursively searches each base's bases via `cls._get_nonflat_cls(b)`. If more than one non-flat class is found at the same level, it raises `TypeError`. If none are found at all (e.g., `FlatFLRWMixin(FlatCosmologyMixin)`), it returns `None`.\n\n3. **`__equiv__` two-path logic**: When `other` is also a `FlatCosmologyMixin`, it delegates to `super().__equiv__(other)` (i.e., `Cosmology.__equiv__`), which checks same class and matching parameters. When `other` is NOT a `FlatCosmologyMixin`, it checks whether `other.__class__` is a subclass of `self.__nonflatclass__`, then verifies `self._parameters_all == other._parameters_all`, that all parameter values match (iterating over `self.parameters`), AND that `other.is_flat` is True.\n\n4. **`clone` delegates to `self.nonflat.clone()`**: When `to_nonflat=True`, the method calls `self.nonflat.clone(meta=meta, **kwargs)` \u2014 it first constructs the non-flat equivalent instance via the `nonflat` property, then clones that instance with the new parameters.\n\n5. **`FlatFLRWMixin` extends `FlatCosmologyMixin`** and redefines `Ode0` as a derived parameter (`derived=True`, `init=False`, `repr=False`) with `field(default=ParameterOde0.clone(default=0, derived=True), init=False, repr=False)`. Its `__post_init__` computes `Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)` after first forcing `Ok0 = 0.0`.\n\n6. **`FlatFLRWMixin.nonflat` is a `lazyproperty`**: It uses `inspect.signature(self.__nonflatclass__).bind_partial()` to construct the non-flat instance from parameters plus `Ode0`, then overwrites `inst.__dict__` with exact parameter values and sets `Ok0` to `self.Ok0`.\n\n7. **`FlatFLRWMixin.__init_subclass__` validates**: It checks that `Ode0` is not in `__init__` for any subclass, raising `TypeError('subclasses of `FlatFLRWMixin` cannot have `Ode0` in `__init__`')` if violated.\n\n8. **`FlatFLRWMixin.clone` adds validation**: Before delegating to super, it raises `ValueError(\"Cannot set 'Ode0' in clone unless 'to_nonflat=True'. \")` if `Ode0` is passed as a kwarg while `to_nonflat` is False.\n\n9. **FLRW's `is_flat` property** (the non-flat version) returns `bool((self.Ok0 == 0.0) and (self.Otot0 == 1.0))`, while `FlatCosmologyMixin.is_flat` always returns the literal `True` (typed as `Literal[True]`).\n\n10. **Concrete flat classes** all inherit from `FlatFLRWMixin` and their non-flat counterpart: `FlatLambdaCDM(FlatFLRWMixin, LambdaCDM)`, `FlatwCDM(FlatFLRWMixin, wCDM)`, `Flatw0waCDM(FlatFLRWMixin, w0waCDM)`, `Flatw0wzCDM(FlatFLRWMixin, w0wzCDM)`, `FlatwpwaCDM(FlatFLRWMixin, wpwaCDM)`.",
    "rubric": [
      "Explains that FlatCosmologyMixin is a mixin class for flat cosmologies, decorated with @dataclass_decorator and using ABCMeta as metaclass",
      "Describes __nonflatclass__ as a lazy classproperty wrapping _get_nonflat_cls, eagerly triggered in __init_subclass__ via cls.__nonflatclass__",
      "Explains _get_nonflat_cls algorithm: searches __bases__ for Cosmology subclasses that are NOT FlatCosmologyMixin subclasses, recurses if none found, raises TypeError if multiple found at same level, returns None if none found at all",
      "Describes __equiv__ two-path logic: delegates to super().__equiv__ for FlatCosmologyMixin instances; for non-flat instances, checks subclass of __nonflatclass__, matching _parameters_all, parameter values, and other.is_flat",
      "Explains FlatCosmologyMixin.clone delegates to self.nonflat.clone(meta=meta, **kwargs) when to_nonflat=True",
      "Describes FlatFLRWMixin redefining Ode0 as a derived parameter (derived=True, init=False, repr=False) and __post_init__ computing Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0) after forcing Ok0 = 0.0",
      "Explains FlatFLRWMixin.nonflat as a lazyproperty that constructs non-flat instance using inspect.signature bind_partial, then overwrites __dict__ with parameter values and sets Ok0",
      "Mentions FlatFLRWMixin.__init_subclass__ validates that Ode0 is not in __init__, raising TypeError if violated",
      "Mentions FlatFLRWMixin.clone raises ValueError if Ode0 is passed without to_nonflat=True",
      "Notes that FlatCosmologyMixin.is_flat always returns literal True, while FLRW.is_flat checks (Ok0 == 0.0) and (Otot0 == 1.0)",
      "Lists the concrete flat classes: FlatLambdaCDM, FlatwCDM, Flatw0waCDM, Flatw0wzCDM, FlatwpwaCDM, each inheriting from FlatFLRWMixin and their non-flat counterpart"
    ],
    "key_files": [
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/funcs/comparison.py",
      "astropy/cosmology/_src/traits/curvature.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/core.py] astropy.cosmology._src.core.FlatCosmologyMixin\nastropy.cosmology._src.core.FlatCosmologyMixin:\n  Mixin class for flat cosmologies.\n\n    Do NOT instantiate directly. Note that all instances of\n    ``FlatCosmologyMixin`` are flat, but not all flat cosmologies are\n    instances of ``FlatCosmologyMixin``. As example, ``LambdaCDM`` **may**\n    be flat (for the a specific set of parameter values), but\n    ``FlatLambdaCDM`` **will** be flat.\n\nastropy.cosmology._src.core.FlatCosmologyMixin._get_nonflat_cls:\n  Find the corresponding non-flat class.\n\n        The class' bases are searched recursively.\n\n        Parameters\n        ----------\n        kls : :class:`astropy.cosmology.Cosmology` class or None, optional\n            If `None` (default) this class is searched instead of `kls`.\n\n        Raises\n        ------\n        TypeError\n            If more than one non-flat class is found at the same level of the\n            inheritance. This is similar to the error normally raised by Python\n            for an inconsistent method resolution order.\n\n        Returns\n        -------\n        type\n            A :class:`Cosmology` subclass this class inherits from that is not a\n            :class:`FlatCosmologyMixin` subclass.\n\nastropy.cosmology._src.core.FlatCosmologyMixin.is_flat:\n  Return `True`, the cosmology is flat.\n\nastropy.cosmology._src.core.FlatCosmologyMixin.nonflat:\n  Return the equivalent non-flat-class instance of this cosmology.\n\nastropy.cosmology._src.core.FlatCosmologyMixin.clone:\n  Returns a copy of this object with updated parameters, as specified.\n\n        This cannot be used to change the type of the cosmology, except for\n        changing to the non-flat version of this cosmology.\n\n        Parameters\n        ----------\n        meta : mapping or None (optional, keyword-only)\n            Metadata that will update the current metadata.\n        to_nonflat : bool, optional keyword-only\n            Whether to change to the non-flat version of this cosmology.\n        **kwargs\n            Cosmology parameter (and name) modifications. If any parameter is\n            changed and a new name is not given, the name will be set to \"[old\n            name] (modified)\".\n\n        Returns\n        -------\n        newcosmo : `~astropy.cosmology.Cosmology` subclass instance\n            A new instance of this (or non-flat equivalent) class with updated\n            parameters as specified. If no arguments are given, then a\n            reference to this object is returned instead of a copy.\n\n        Examples\n        --------\n        To make a copy of the ``Planck13`` cosmology with a different matter\n        density (``Om0``), and a new name:\n\n            >>> from astropy.cosmology import Planck13\n            >>> Planck13.clone(name=\"Modified Planck 2013\", Om0=0.35)\n            FlatLambdaCDM(name='Modified Planck 2013', H0=<Quantity 67.77 km / (Mpc s)>,\n                          Om0=0.35, ...\n\n        If no name is specified, the new name will note the modification.\n\n            >>> Planck13.clone(Om0=0.35).name\n            'Planck13 (modified)'\n\n        The keyword 'to_nonflat' can be used to clone on the non-flat equivalent\n        cosmology. For :class:`~astropy.cosmology.FLRW` cosmologies this means\n        ``Ode0`` can be modified:\n\n            >>> Planck13.clone(to_nonflat=True, Ode0=1)\n            LambdaCDM(name='Planck13 (modified)', H0=<Quantity 67.77 km / (Mpc s)>,\n                      Om0=0.30712, Ode0=1.0, ...\n\nastropy.cosmology._src.core.FlatCosmologyMixin.__equiv__:\n  Flat-|Cosmology| equivalence.\n\n        Use `astropy.cosmology.cosmology_equal` with\n        ``allow_equivalent=True`` for actual checks!\n\n        Parameters\n        ----------\n        other : Any, positional-only\n            The object to which to compare for equivalence.\n\n        Returns\n        -------\n        bool or `NotImplemented`\n            `True` if ``other`` is of the same class / non-flat class (e.g.\n            |FlatLambdaCDM| and |LambdaCDM|) has matching parameters and\n            parameter values.\n            `False` if ``other`` is of the same class but has different\n            parameters.\n            `NotImplemented` otherwise.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims (points 1, 2, 3, 4, 5, 6, 7, 8, 9, 10) go far beyond what the documentation states and appear to be based on source code knowledge or inference rather than the provided documentation. While they are plausible implementation details, they cannot be verified from the given documentation alone.",
      "Point 2 states that if no non-flat class is found it returns `None`, but the documentation only mentions a `TypeError` being raised if more than one is found and states the return type is `type` (a Cosmology subclass). Returning `None` is not mentioned and could contradict the documented return type.",
      "Point 5 mentions `Ok0` in the formula `Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)` after forcing `Ok0 = 0.0`, which is plausible but entirely unverifiable from the documentation.",
      "Point 9 claims FLRW's `is_flat` returns `bool((self.Ok0 == 0.0) and (self.Otot0 == 1.0))` while the documentation only states FlatCosmologyMixin.is_flat returns True. The non-flat version's implementation is not documented here.",
      "Point 10 lists specific concrete flat classes which are plausible but not mentioned in the provided documentation."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: 'lazy means computed once and cached' explanation and 'to avoid machine precision issues' rationale came from docs/comments; all other claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_06",
    "repo": "astropy",
    "question": "Why does `default_cosmology` store its `_value` as a string like \"Planck18\" instead of an actual Cosmology instance, and how does this interact with the ScienceState machinery?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `default_cosmology` class exists as a global science state manager (singleton pattern) that allows temporary overriding of the default cosmology via a context manager pattern. Its design reveals several important architectural decisions:\n\n1. **Lazy resolution via `validate()` on every `get()` call**: The parent class `ScienceState.get()` always calls `cls.validate(cls._value)` rather than returning `_value` directly. This is why `_value` can be stored as a string (\"Planck18\") rather than an actual Cosmology object. The string is resolved to a Cosmology object by `_get_from_registry` each time `get()` is called. This enables lazy loading \u2014 the Planck18 cosmology data file isn't read until someone actually calls `get()`.\n\n2. **Lazy-loaded realizations via module-level `__getattr__`**: The `_get_from_registry` method retrieves cosmologies from `astropy.cosmology.realizations`, which uses a module-level `__getattr__` to lazily load cosmology data from `.ecsv` files using `Cosmology.read()`. After first access, the result is cached on the module via `setattr(sys.modules[__name__], name, cosmo)` so `__getattr__` is only called once per realization name.\n\n3. **\"no_default\" sentinel string**: The `validate()` method special-cases the string `\"no_default\"` to return `None`, allowing users to explicitly disable the default cosmology \u2014 any code calling `default_cosmology.get()` would then receive `None` and need to handle that case.\n\n4. **Singleton enforcement**: `ScienceState.__init__` raises `RuntimeError(\"This class is a singleton. Do not instantiate.\")`, ensuring the class is used only via classmethods (`get()`, `set()`).\n\n5. **Context manager for thread-unsafe state restoration**: The `set()` method returns a `_ScienceStateContext` that captures the **pre-validation** current `_value` before setting the new one. On `__exit__`, it restores this old value directly to `cls._value`. Since `get()` re-validates each time, restoring a string like \"Planck18\" still works correctly.\n\n6. **The `set()` method validates the new value before storing** (unlike `get()` which validates stored value on retrieval), meaning after `set()` is called, `_value` holds a Cosmology instance (or None for \"no_default\"), not a string.",
    "rubric": [
      "Explains that ScienceState.get() calls cls.validate(cls._value) on every access, enabling lazy resolution from string to Cosmology object",
      "Explains that _get_from_registry retrieves cosmologies from astropy.cosmology.realizations, which uses a module-level __getattr__ with PEP 562 to lazily load .ecsv data files",
      "Mentions that realizations caches loaded cosmologies via setattr(sys.modules[__name__], name, cosmo) so __getattr__ is only called once per realization",
      "Explains the 'no_default' sentinel string that validate() special-cases to return None, allowing users to disable the default cosmology",
      "Mentions that ScienceState.__init__ raises RuntimeError enforcing singleton/classmethod-only usage pattern",
      "Explains that set() validates the new value before storing it (so _value becomes a Cosmology instance after set), while the context manager captures the pre-validation _value and restores it on __exit__"
    ],
    "key_files": [
      "astropy/cosmology/_src/default.py",
      "astropy/utils/state.py",
      "astropy/cosmology/realizations.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/default.py] astropy.cosmology._src.default.default_cosmology\nastropy.cosmology._src.default.default_cosmology:\n  The default cosmology to use.\n\n    To change it::\n\n        >>> from astropy.cosmology import default_cosmology, WMAP7\n        >>> with default_cosmology.set(WMAP7):\n        ...     # WMAP7 cosmology in effect\n        ...     pass\n\n    Or, you may use a string::\n\n        >>> with default_cosmology.set('WMAP7'):\n        ...     # WMAP7 cosmology in effect\n        ...     pass\n\n    To get the default cosmology:\n\n        >>> default_cosmology.get()\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>,\n                      Om0=0.30966, ...\n\nastropy.cosmology._src.default.default_cosmology.validate:\n  Return a Cosmology given a value.\n\n        Parameters\n        ----------\n        value : None, str, or `~astropy.cosmology.Cosmology`\n\n        Returns\n        -------\n        `~astropy.cosmology.Cosmology` instance\n\n        Raises\n        ------\n        TypeError\n            If ``value`` is not a string or |Cosmology|.\n\nastropy.cosmology._src.default.default_cosmology._get_from_registry:\n  Get a registered Cosmology realization.\n\n        Parameters\n        ----------\n        name : str\n            The built-in |Cosmology| realization to retrieve.\n\n        Returns\n        -------\n        `astropy.cosmology.Cosmology`\n            The cosmology realization of `name`.\n\n        Raises\n        ------\n        ValueError\n            If ``name`` is a str, but not for a built-in Cosmology.\n        TypeError\n            If ``name`` is for a non-Cosmology object.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 1 (lazy resolution via validate): The documentation does show that `validate()` accepts None, str, or Cosmology and returns a Cosmology instance, and that `_get_from_registry` resolves strings. The claim about `ScienceState.get()` always calling `cls.validate(cls._value)` is plausible but goes beyond what the documentation states \u2014 it's an extrapolation about the parent class internals.",
      "Claim 2 (PEP 562 lazy-loaded realizations): The documentation says nothing about PEP 562, module-level `__getattr__`, `.ecsv` files, or `Cosmology.read()`. This is plausible code-level detail but entirely unsupported by the provided documentation.",
      "Claim 3 ('no_default' sentinel string): The documentation does not mention 'no_default' at all. This is an unsupported claim \u2014 it may be true in the actual code, but cannot be verified from the provided documentation.",
      "Claim 4 (Singleton enforcement via RuntimeError): Not mentioned in the documentation. Plausible but unsupported.",
      "Claim 5 (_ScienceStateContext, pre-validation value capture, thread-unsafe state restoration): The documentation only shows that `set()` is used as a context manager. The internal details about `_ScienceStateContext` and pre-validation value capture are unsupported extrapolations.",
      "Claim 6 (set() validates before storing, resulting in _value holding a Cosmology instance): Not explicitly stated in the documentation. The documentation shows `set()` being used with both Cosmology objects and strings, but says nothing about when validation occurs relative to storage."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the 'PEP 562' reference was documentation-only and was removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_07",
    "repo": "astropy",
    "question": "Where is `FlatLambdaCDM` defined, how does it enforce flatness, and how do its `efunc` and distance computation paths differ from `LambdaCDM`?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `FlatLambdaCDM` class is defined at line 624 of `astropy/cosmology/_src/flrw/lambdacdm.py` and inherits from both `FlatFLRWMixin` and `LambdaCDM` (in that MRO order). Here is where key functionality is located and how data flows:\n\n**Class hierarchy and location:**\n- `FlatLambdaCDM(FlatFLRWMixin, LambdaCDM)` at `lambdacdm.py:624`\n- `LambdaCDM(FLRW)` at `lambdacdm.py:26`\n- `FlatFLRWMixin(FlatCosmologyMixin)` at `base.py:1275`\n- `FLRW(Cosmology, BaryonComponent, TotalComponent, CriticalDensity, MatterComponent, CurvatureComponent, DarkEnergyComponent, HubbleParameter, PhotonComponent, NeutrinoComponent, ScaleFactor, DarkMatterComponent, TemperatureCMB)` at `base.py:133`\n\n**Flatness enforcement in `FlatFLRWMixin.__post_init__`** (base.py:1307-1312):\n- Sets `self.__dict__['Ode0'] = 0` before calling `super().__post_init__()`\n- After that call, explicitly sets `self.__dict__['Ok0'] = 0.0`\n- Then derives `Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)`, ensuring flatness.\n- In `FlatFLRWMixin`, `Ode0` is redefined as a derived parameter with `init=False, repr=False` (line 1286-1290).\n\n**`efunc` formula difference between LambdaCDM and FlatLambdaCDM:**\n- `LambdaCDM.efunc` (line 567): `sqrt(zp1**2 * ((Or * zp1 + Om0) * zp1 + Ok0) + Ode0)` \u2014 includes the `Ok0` curvature term\n- `FlatLambdaCDM.efunc` (line 743): `sqrt(zp1**3 * (Or * zp1 + Om0) + Ode0)` \u2014 drops the `Ok0` term and simplifies `zp1**2 * (... * zp1)` into `zp1**3`\n- The base `FLRW.efunc` (base.py:404) is even more general, calling `self.de_density_scale(z)` to scale `Ode0`, which LambdaCDM variants skip because `de_density_scale` returns 1.0 (constant `w = -1`).\n\n**Scalar inverse efunc Cython accelerators** (`scalar_inv_efuncs.pyx`):\n- `FlatLambdaCDM` uses `flcdm_inv_efunc_norel`, `flcdm_inv_efunc_nomnu`, or `flcdm_inv_efunc` (lines 42-56 of .pyx)\n- `LambdaCDM` uses `lcdm_inv_efunc_norel`, `lcdm_inv_efunc_nomnu`, or `lcdm_inv_efunc` (lines 22-38 of .pyx)\n- The flat versions omit the `Ok0` parameter entirely. E.g., `flcdm_inv_efunc_norel(z, Om0, Ode0)` takes only 3 args while `lcdm_inv_efunc_norel(z, Om0, Ode0, Ok0)` takes 4.\n- These Cython functions use `pow(x, -0.5)` because it's noted to be slightly faster than `x**(-0.5)` and faster than `1.0/sqrt(x)` (line 16 comment).\n\n**Optimized comoving distance paths** (`_optimize_flat_norad` at lambdacdm.py:121-142):\n- When `Tcmb0 == 0` and `Ok0 == 0`, `_optimize_flat_norad()` replaces `_comoving_distance_z1z2`, `_age`, and `_lookback_time` via `object.__setattr__`\n- Three special cases: Om0==0 \u2192 de Sitter (dS), Om0==1 \u2192 Einstein-de Sitter (EdS), otherwise \u2192 hypergeometric\n- The hypergeometric path uses `scipy.special.hyp2f1` with parameters (1/6, 1/2, 7/6, -x\u00b3), referencing Baes et al. 2017\n- The elliptic comoving distance path (for non-flat LambdaCDM with no radiation) uses `scipy.special.ellipkinc` and references Kantowski, Kao, & Thomas (2000)\n- `FlatLambdaCDM.__post_init__` explicitly calls `self._optimize_flat_norad()` again because the parent `LambdaCDM.__post_init__` ran before flatness was set.\n\n**`__nonflatclass__` mechanism** (`core.py:544`):\n- `__nonflatclass__` is a lazy `classproperty` that calls `_get_nonflat_cls()`, which walks the MRO bases to find the first `Cosmology` subclass that is not a `FlatCosmologyMixin` subclass\n- For `FlatLambdaCDM`, this returns `LambdaCDM`\n\n**Data flow for distance integrals:**\n- `FLRW.comoving_distance(z)` calls `self._comoving_distance_z1z2(0.0, z)` (base.py:841)\n- Default `_comoving_distance_z1z2` calls `_integral_comoving_distance_z1z2`, which uses `hubble_distance * quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)` (base.py:881-902)\n- `_integral_comoving_distance_z1z2_scalar` is decorated with `@vectorize_redshift_method(nin=2)` which converts Quantity inputs and uses `np.vectorize` for array inputs\n- `quad` is `scipy.integrate.quad` when scipy is available, or raises `ModuleNotFoundError` otherwise (via `scipy_compat.py`)\n\n**Frozen dataclass pattern:**\n- Both `LambdaCDM` and `FlatLambdaCDM` are decorated with `@dataclass_decorator`, which applies `dataclass(frozen=True, repr=True, eq=False, init=True)` (core.py:74)\n- Because the class is frozen, all attribute mutations in `__post_init__` use `object.__setattr__` to bypass the frozen restriction",
    "rubric": [
      "Identifies FlatLambdaCDM as inheriting from both FlatFLRWMixin and LambdaCDM (in that MRO order)",
      "Explains that FlatFLRWMixin.__post_init__ enforces flatness by setting Ok0=0.0 and deriving Ode0 = 1 - (Om0 + Ogamma0 + Onu0 + Ok0)",
      "Notes that FlatLambdaCDM.efunc drops the Ok0 curvature term compared to LambdaCDM.efunc (zp1**3*(Or*zp1+Om0)+Ode0 vs zp1**2*((Or*zp1+Om0)*zp1+Ok0)+Ode0)",
      "Mentions that FlatLambdaCDM uses flat-specific Cython scalar_inv_efuncs (flcdm_inv_efunc_norel/nomnu/efunc) which omit the Ok0 parameter",
      "Describes the _optimize_flat_norad method that replaces comoving distance/age/lookback_time with special-case implementations (de Sitter, Einstein-de Sitter, or hypergeometric)",
      "Notes that FlatLambdaCDM.__post_init__ must re-call _optimize_flat_norad because LambdaCDM.__post_init__ ran before flatness was fully set",
      "Mentions that attribute mutations use object.__setattr__ (or self.__dict__) to bypass the frozen dataclass restriction"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/lambdacdm.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/scipy_compat.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/lambdacdm.py] astropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM\nastropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM:\n  FLRW cosmology with a cosmological constant and no curvature.\n\n    This has no additional attributes beyond those of FLRW.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import FlatLambdaCDM\n    >>> cosmo = FlatLambdaCDM(H0=70, Om0=0.3)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\n    To get an equivalent cosmology, but of type `astropy.cosmology.LambdaCDM`,\n    use :attr:`astropy.cosmology.FlatFLRWMixin.nonflat`.\n\n    >>> print(cosmo.nonflat)\n    LambdaCDM(H0=70.0 km / (Mpc s), Om0=0.3, Ode0=0.7, ...\n\nastropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM.efunc:\n  Function used to calculate H(z), the Hubble parameter.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The redshift scaling of the Hubble constant.\n            Defined such that :math:`H(z) = H_0 E(z)`.\n\nastropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM.inv_efunc:\n  Function used to calculate :math:`\\frac{1}{H_z}`.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The inverse redshift scaling of the Hubble constant.\n            Defined such that :math:`H_z = H_0 / E`.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_08",
    "repo": "astropy",
    "question": "How does the wCDM cosmology class compute the Hubble parameter E(z), and how does it select different Cython scalar integration functions based on neutrino properties?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `wCDM` class in `astropy/cosmology/_src/flrw/w0cdm.py` implements a FLRW cosmology where the dark energy equation of state `w(z)` returns a constant value `w0`. Here's how the algorithm is implemented:\n\n**Class Structure:**\n- `wCDM` inherits from `FLRW` (base class in `base.py`) and is decorated with `@dataclass_decorator`.\n- `FlatwCDM` inherits from both `FlatFLRWMixin` and `wCDM` (in that order for MRO).\n- The `w0` parameter is defined as a `Parameter` with `default=-1.0` and `fvalidate=\"float\"`.\n\n**Key Algorithm in `efunc` (wCDM with curvature):**\nThe Hubble parameter scaling E(z) is computed as:\n```\nsqrt(zp1\u00b2 * ((Or * zp1 + Om0) * zp1 + Ok0) + Ode0 * zp1^(3*(1+w0)))\n```\nThis inlines the `de_density_scale` computation directly rather than calling `self.de_density_scale(z)` as the base FLRW class does. The base class `FLRW.efunc` calls `self.Ode0 * self.de_density_scale(z)`, but `wCDM` overrides this by substituting `zp1 ** (3.0 * (1.0 + self.w0))` directly.\n\n**FlatwCDM differs from wCDM in `efunc`:** It omits the `Ok0` (curvature) term entirely:\n```\nsqrt(zp1\u00b3 * (Or * zp1 + Om0) + Ode0 * zp1^(3*(1+w0)))\n```\n\n**Fast scalar integration via Cython:**\nIn `__post_init__`, three branches select different Cython functions from `scalar_inv_efuncs.pyx`:\n1. `Tcmb0 == 0`: `wcdm_inv_efunc_norel`, args are `(Om0, Ode0, Ok0, w0)`\n2. `not self._nu_info.has_massive_nu`: `wcdm_inv_efunc_nomnu`, args are `(Om0, Ode0, Ok0, Ogamma0+Onu0, w0)`\n3. Otherwise (massive neutrinos): `wcdm_inv_efunc`, args are `(Om0, Ode0, Ok0, Ogamma0, neff_per_nu, n_massless_nu, nu_y_list, w0)`\n\nThese Cython scalar functions are used in numerical integration (via `scipy.integrate.quad`) for computing comoving distances. The `_inv_efunc_scalar` is called in `_integral_comoving_distance_z1z2_scalar` as `quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)[0]` and also in `_lookback_time_integrand_scalar` and `_abs_distance_integrand_scalar`.\n\n**The Cython `nufunc`** (for massive neutrinos) computes: `0.22710731766 * NeffPerNu * rel_mass_sum`, where `rel_mass_sum` accumulates `pow(1.0 + (0.3173/opz * nu_y[i])^1.83, 0.54644808743)` for each massive species plus the number of massless neutrinos.\n\n**The `aszarr` utility** converts redshift inputs to numpy arrays: scalars become `np.asarray(z)`, Quantities are converted via `z.to_value(cu.redshift)[...]`, plain ndarrays pass through, and other array-likes are wrapped in `Quantity(z, cu.redshift).view(np.ndarray)`.",
    "rubric": [
      "Explains that wCDM.efunc computes sqrt(zp1\u00b2 * ((Or*zp1 + Om0)*zp1 + Ok0) + Ode0 * zp1^(3*(1+w0)))",
      "Explains that FlatwCDM.efunc omits the Ok0 curvature term: sqrt(zp1\u00b3 * (Or*zp1 + Om0) + Ode0 * zp1^(3*(1+w0)))",
      "Identifies the three branches in __post_init__: norel (Tcmb0==0), nomnu (no massive neutrinos), and full (with massive neutrinos)",
      "Lists the arguments passed to the Cython scalar functions for each branch (e.g., norel gets Om0/Ode0/Ok0/w0, nomnu adds Ogamma0+Onu0, full adds Ogamma0/neff_per_nu/n_massless_nu/nu_y_list)",
      "Mentions that wCDM inlines the dark energy density scale as zp1^(3*(1+w0)) rather than calling de_density_scale separately",
      "Notes that FlatwCDM inherits from both FlatFLRWMixin and wCDM, and the w0 Parameter defaults to -1.0"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0cdm.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/utils.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0cdm.py] astropy.cosmology._src.flrw.w0cdm.wCDM\nastropy.cosmology._src.flrw.w0cdm.wCDM:\n  FLRW cosmology with a constant dark energy EoS and curvature.\n\n    This has one additional attribute beyond those of FLRW.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    Ode0 : float\n        Omega dark energy: density of dark energy in units of the critical\n        density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at all redshifts. This is\n        pressure/density for dark energy in units where c=1. A cosmological\n        constant has w0=-1.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import wCDM\n    >>> cosmo = wCDM(H0=70, Om0=0.3, Ode0=0.7, w0=-0.9)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\nastropy.cosmology._src.flrw.w0cdm.wCDM.w:\n  Returns dark energy equation of state at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        w : ndarray\n            The dark energy equation of state.\n\n        Notes\n        -----\n        The dark energy equation of state is defined as\n        :math:`w(z) = P(z)/\\rho(z)`, where :math:`P(z)` is the pressure at\n        redshift z and :math:`\\rho(z)` is the density at redshift z, both in\n        units where c=1. Here this is :math:`w(z) = w_0`.\n\nastropy.cosmology._src.flrw.w0cdm.wCDM.de_density_scale:\n  Evaluates the redshift dependence of the dark energy density.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        I : ndarray\n            The scaling of the energy density of dark energy with redshift.\n\n        Notes\n        -----\n        The scaling factor, I, is defined by :math:`\\rho(z) = \\rho_0 I`,\n        and in this case is given by\n        :math:`I = \\left(1 + z\\right)^{3\\left(1 + w_0\\right)}`\n\nastropy.cosmology._src.flrw.w0cdm.wCDM.efunc:\n  Function used to calculate H(z), the Hubble parameter.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The redshift scaling of the Hubble constant.\n            Defined such that :math:`H(z) = H_0 E(z)`.\n\nastropy.cosmology._src.flrw.w0cdm.wCDM.inv_efunc:\n  Function used to calculate :math:`\\frac{1}{H_z}`.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The inverse redshift scaling of the Hubble constant.\n            Defined such that :math:`H_z = H_0 / E`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not describe class decorators, dataclass behavior, or the `frozen=True, repr=True, eq=False, init=True` details - these are plausible implementation details but unverifiable from the given doc.",
      "The documentation does not mention `FlatwCDM`, `FlatFLRWMixin`, or MRO details - these are plausible but unverifiable from the given doc.",
      "The documentation does not describe Cython functions, `scalar_inv_efuncs.pyx`, or the three-branch selection in `__post_init__` - these are plausible implementation details but unverifiable.",
      "The documentation does not describe the `nufunc` Komatsu et al. 2011 fitting formula or its specific constants - this is unverifiable.",
      "The documentation does not describe the `aszarr` utility or its behavior - this is unverifiable.",
      "The `efunc` formula described in the answer as `sqrt(zp1\u00b2 * ((Or * zp1 + Om0) * zp1 + Ok0) + Ode0 * zp1^(3*(1+w0)))` is consistent with the documented `de_density_scale` formula `(1+z)^{3(1+w0)}` and the general FLRW framework.",
      "The `de_density_scale` formula `I = (1+z)^{3(1+w0)}` matches the documentation exactly.",
      "The claim that `w(z) = w0` (constant) is consistent with the documentation's Notes section."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in attributions (Komatsu reference), design rationale ('for performance'), and dataclass decorator specifics.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_09",
    "repo": "astropy",
    "question": "What is the full inheritance chain and dependency structure of FlatwCDM, including how it enforces flatness and optimizes distance integrals?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `FlatwCDM` class in `astropy/cosmology/_src/flrw/w0cdm.py` is a flat FLRW cosmology with a constant dark energy equation of state (its `w` method returns `self.w0 * np.ones_like(aszarr(z))`). It inherits from `FlatFLRWMixin` and `wCDM` (in that MRO order), and is decorated with `@dataclass_decorator` which transforms it into a frozen dataclass with `eq=False`.\n\n**Key dependency chain:**\n\n1. **MRO**: `FlatwCDM` \u2192 `FlatFLRWMixin` \u2192 `wCDM` \u2192 `FLRW` \u2192 `Cosmology` (plus multiple trait mixins like `BaryonComponent`, `HubbleParameter`, etc.)\n\n2. **`FlatFLRWMixin`** (in `base.py`) inherits from `FlatCosmologyMixin` (in `core.py`). From FLRW's `Ok0` cached_property (`1.0 - self.Om0 - self.Ode0 - self.Ogamma0 - self.Onu0`) and the flat constraint (`Ok0 = 0.0`), `Ode0` is computed as `1.0 - Om0 - Ogamma0 - Onu0`.\n\n3. **`__nonflatclass__`** is resolved via `FlatCosmologyMixin._get_nonflat_cls()`, which searches the class bases for a `Cosmology` subclass that is NOT a `FlatCosmologyMixin` subclass. For `FlatwCDM`, this resolves to `wCDM`.\n\n4. **Scalar inverse efunc functions** are implemented in Cython (`scalar_inv_efuncs.pyx`). For `FlatwCDM.__post_init__`, the three cases are:\n   - `Tcmb0 == 0`: uses `fwcdm_inv_efunc_norel(z, Om0, Ode0, w0)` \u2014 no `Ok0` argument\n   - No massive neutrinos: uses `fwcdm_inv_efunc_nomnu(z, Om0, Ode0, Or0, w0)` where `Or0 = Ogamma0 + Onu0`\n   - Massive neutrinos: uses `fwcdm_inv_efunc(z, Om0, Ode0, Ogamma0, NeffPerNu, nmasslessnu, nu_y, w0)`\n\n5. **`efunc` difference between `wCDM` and `FlatwCDM`**: The flat version computes `sqrt(zp1**3 * (Or * zp1 + Om0) + Ode0 * zp1 ** (3.0 * (1 + w0)))`, while the non-flat `wCDM` version includes the curvature term: `sqrt(zp1**2 * ((Or * zp1 + Om0) * zp1 + Ok0) + Ode0 * zp1 ** (3.0 * (1.0 + self.w0)))`.\n\n6. **`w0` parameter**: Defined in `wCDM` as `Parameter(default=-1.0, fvalidate=\"float\")`.\n\n7. **`_inv_efunc_scalar` usage**: It is used in `FLRW._integral_comoving_distance_z1z2_scalar` via `quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)` for numerical integration of comoving distances. It's also used in `_lookback_time_integrand_scalar` and `_abs_distance_integrand_scalar`.\n\n8. **The `nufunc` in Cython**: The neutrino density fitting function uses constants named `KOMATSU_P=1.83`, `KOMATSU_INVP=0.54644808743`, `KOMATSU_K=0.3173`, and `NEUTRINO_FERMI_DIRAC_CORRECTION=0.22710731766`. In the Cython implementation: `k = 0.3173 / opz`, each massive neutrino contributes `pow(1.0 + (k * nu_y[i])**1.83, 0.54644808743)`, and the result is multiplied by `0.22710731766 * NeffPerNu`.\n\n9. **`aszarr` utility**: Converts redshift inputs to numpy arrays, handling Quantity objects via `z.to_value(cu.redshift)[...]`, scalars via `np.asarray(z)`, plain ndarrays returned directly, and fallback via `Quantity(z, cu.redshift, copy=None, subok=True).view(np.ndarray)`.",
    "rubric": [
      "Identifies FlatwCDM inherits from FlatFLRWMixin and wCDM (in that order), with wCDM inheriting from FLRW which inherits from Cosmology",
      "Explains that FlatFLRWMixin inherits from FlatCosmologyMixin and enforces flatness by setting Ok0=0.0 and computing Ode0 as 1.0 - Om0 - Ogamma0 - Onu0 - Ok0 in __post_init__",
      "Describes __nonflatclass__ resolution via _get_nonflat_cls which searches bases for a Cosmology subclass that is not a FlatCosmologyMixin subclass, resolving to wCDM",
      "Explains the three cases in FlatwCDM.__post_init__ for _inv_efunc_scalar: norel (fwcdm_inv_efunc_norel with Om0, Ode0, w0), nomnu (fwcdm_inv_efunc_nomnu with Om0, Ode0, Or0, w0), and massive neutrinos (fwcdm_inv_efunc with Ogamma0, NeffPerNu, nmasslessnu, nu_y, w0)",
      "Notes the flat vs non-flat efunc difference: flat version omits the Ok0 curvature term",
      "Mentions that _inv_efunc_scalar is used via quad() integration for comoving distances, lookback time, and absorption distance integrands",
      "Describes the Cython nufunc implementation using Komatsu fitting constants (p=1.83, 1/p=0.54644808743, k=0.3173, prefactor=0.22710731766)",
      "Notes that the class is decorated with dataclass_decorator which applies dataclass(frozen=True, eq=False) and precomputes the signature"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0cdm.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/utils.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0cdm.py] astropy.cosmology._src.flrw.w0cdm.FlatwCDM\nastropy.cosmology._src.flrw.w0cdm.FlatwCDM:\n  FLRW cosmology with a constant dark energy EoS and no spatial curvature.\n\n    This has one additional attribute beyond those of FLRW.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at all redshifts. This is\n        pressure/density for dark energy in units where c=1. A cosmological\n        constant has w0=-1.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import FlatwCDM\n    >>> cosmo = FlatwCDM(H0=70, Om0=0.3, w0=-0.9)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\n    To get an equivalent cosmology, but of type `astropy.cosmology.wCDM`,\n    use :attr:`astropy.cosmology.FlatFLRWMixin.nonflat`.\n\n    >>> print(cosmo.nonflat)\n    wCDM(H0=70.0 km / (Mpc s), Om0=0.3, Ode0=0.7, ...\n\nastropy.cosmology._src.flrw.w0cdm.FlatwCDM.efunc:\n  Function used to calculate H(z), the Hubble parameter.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The redshift scaling of the Hubble constant.\n            Defined such that :math:`H(z) = H_0 E(z)`.\n\nastropy.cosmology._src.flrw.w0cdm.FlatwCDM.inv_efunc:\n  Function used to calculate :math:`\\frac{1}{H_z}`.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        E : ndarray\n            The inverse redshift scaling of the Hubble constant.\n            Defined such that :math:`H(z) = H_0 E(z)`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer (MRO, Cython internals, efunc formulas, nufunc fitting constants, aszarr utility, etc.) go far beyond what the provided documentation covers. They cannot be verified from the given doc chunk alone, but they are plausible code-level implementation details.",
      "The Ode0 computation formula 'Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)' cannot be verified from the doc but is plausible given the flat cosmology constraint.",
      "The claim about nonflat equivalent being wCDM is supported by the documentation example: 'cosmo.nonflat' prints 'wCDM(H0=70.0 km / (Mpc s), Om0=0.3, Ode0=0.7, ...'",
      "The w0 default of -1.0 is consistent with the documentation stating 'A cosmological constant has w0=-1.0'.",
      "The inv_efunc docstring in the documentation says it calculates 1/H_z, not explicitly 1/E(z). The answer's discussion of _inv_efunc_scalar is plausible but unverifiable from the doc.",
      "The specific line numbers cited (e.g., line 83, line 902, line 579, line 625, line 1275) cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; minor doc leakage in Komatsu reference attribution, line numbers, and parameter doc strings.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_10",
    "repo": "astropy",
    "question": "Why does the w0waCDM cosmology class select different Cython scalar inverse expansion functions at construction time based on the physics configuration?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `w0waCDM` class uses a multi-layered performance optimization strategy centered around Cython-compiled scalar inverse expansion functions (`scalar_inv_efuncs.pyx`) that are selected at construction time in `__post_init__` based on the physics configuration. Here's why these design decisions exist:\n\n**1. Three-tier branching for physical complexity elimination:**\nIn `__post_init__`, the code checks `Tcmb0.value == 0` first (no relativistic species at all), then `not self._nu_info.has_massive_nu` (massless neutrinos only), and finally the full case with massive neutrinos. Each tier selects a different Cython function with fewer arguments and simpler computation:\n- `w0wacdm_inv_efunc_norel`: takes 5 cosmological args (Om0, Ode0, Ok0, w0, wa) \u2014 no radiation terms at all\n- `w0wacdm_inv_efunc_nomnu`: takes 6 cosmological args \u2014 combines `Ogamma0 + Onu0` into a single `Or0` parameter, and the function does not call `nufunc`\n- `w0wacdm_inv_efunc`: takes 9 cosmological args \u2014 calls `nufunc()` at every evaluation to compute the neutrino mass contribution\n\n**2. Cython for hot-loop scalar performance:**\nThe functions in `scalar_inv_efuncs.pyx` are compiled with `boundscheck=False` and use C-level `exp` and `pow` from `libc.math`. The `nufunc` is declared `cdef` (Cython-only, not Python-callable).\n\n**3. Pre-binding arguments at construction time:**\nThe cosmological parameters (Om0, Ode0, Ok0, w0, wa, etc.) are frozen into `_inv_efunc_scalar_args` as a tuple at object creation. The scalar function and its args are used within `scipy.integrate.quad` calls (visible in `_integral_comoving_distance_z1z2_scalar` and `_integral_lookback_time`). The function and its args are stored via `object.__setattr__` because the class is a frozen dataclass (`dataclass(frozen=True)`).\n\n**4. Flat variant eliminates Ok0 from the computation:**\n`Flatw0waCDM` uses different Cython functions (`fw0wacdm_inv_efunc_norel`, etc.) that don't accept `Ok0` at all. Comparing the Cython code: the non-flat `w0wacdm_inv_efunc_norel` computes `opz**2 * (opz * Om0 + Ok0) + Ode0 * Odescl`, while the flat `fw0wacdm_inv_efunc_norel` computes `opz**3 * Om0 + Ode0 * Odescl` \u2014 the curvature term is algebraically eliminated, saving a multiplication and addition per call.\n\n**5. Base class fallback:**\nThe base `FLRW.__post_init__` sets `_inv_efunc_scalar = self.inv_efunc` (the generic Python method) and `_inv_efunc_scalar_args = ()` as defaults. Subclasses like `w0waCDM` override these with their specialized Cython functions. The `w0waCDM` class does NOT override `inv_efunc` itself (only the base `FLRW.inv_efunc` exists for array inputs), but the scalar path through `_integral_comoving_distance_z1z2_scalar` and `_lookback_time_integrand_scalar` uses the optimized Cython versions. These scalar methods are invoked inside `scipy.integrate.quad`.\n\n**6. The neutrino fitting function (`nufunc`):**\nThe Cython `nufunc` uses hardwired constants (1.83, 0.54644808743, 0.3173, 0.22710731766) directly in the computation rather than declaring them as `cdef` variables. The same constants are also defined in `base.py` as `KOMATSU_P`, `KOMATSU_INVP`, `KOMATSU_K`, and `NEUTRINO_FERMI_DIRAC_CORRECTION` for the Python-level code.",
    "rubric": [
      "Explains the three-tier branching in __post_init__: no relativistic species (Tcmb0==0), massless neutrinos only, and massive neutrinos, each selecting a different Cython function with fewer arguments and simpler computation",
      "Describes how the norel variant eliminates radiation terms entirely, the nomnu variant combines Ogamma0+Onu0 into a single Or0 parameter avoiding neutrino mass computation, and the full variant calls nufunc() for massive neutrino contributions",
      "Mentions that the Cython functions in scalar_inv_efuncs.pyx use C-level math (libc.math exp/pow) and boundscheck=False for performance, and nufunc is declared cdef (not Python-callable)",
      "Explains pre-binding of cosmological parameters into _inv_efunc_scalar_args tuple at construction time, used later inside scipy.integrate.quad for comoving distance and lookback time integrals",
      "Notes that object.__setattr__ is used because the class is a frozen dataclass",
      "Describes how the flat variant (Flatw0waCDM) uses separate Cython functions that eliminate the Ok0 curvature parameter entirely from the computation",
      "Mentions the base FLRW class sets default _inv_efunc_scalar to the generic Python inv_efunc method, which subclasses override with optimized Cython versions"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0wacdm.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/parameter/converter.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0wacdm.py] astropy.cosmology._src.flrw.w0wacdm.w0waCDM\nastropy.cosmology._src.flrw.w0wacdm.w0waCDM:\n  FLRW cosmology with a CPL dark energy EoS and curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the\n    CPL form as described in Chevallier & Polarski [1]_ and Linder [2]_:\n    :math:`w(z) = w_0 + w_a (1-a) = w_0 + w_a z / (1+z)`.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    Ode0 : float\n        Omega dark energy: density of dark energy in units of the critical\n        density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at z=0 (a=1). This is pressure/density\n        for dark energy in units where c=1.\n\n    wa : float, optional\n        Negative derivative of the dark energy equation of state with respect\n        to the scale factor. A cosmological constant has w0=-1.0 and wa=0.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import w0waCDM\n    >>> cosmo = w0waCDM(H0=70, Om0=0.3, Ode0=0.7, w0=-0.9, wa=0.2)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\n    References\n    ----------\n    .. [1] Chevallier, M., & Polarski, D. (2001). Accelerating Universes with\n           Scaling Dark Matter. International Journal of Modern Physics D,\n           10(2), 213-223.\n    .. [2] Linder, E. (2003). Exploring the Expansion History of the\n           Universe. Phys. Rev. Lett., 90, 091301.\n\nastropy.cosmology._src.flrw.w0wacdm.w0waCDM.w:\n  Returns dark energy equation of state at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        w : ndarray\n            The dark energy equation of state\n\n        Notes\n        -----\n        The dark energy equation of state is defined as\n        :math:`w(z) = P(z)/\\rho(z)`, where :math:`P(z)` is the pressure at\n        redshift z and :math:`\\rho(z)` is the density at redshift z, both in\n        units where c=1. Here this is\n        :math:`w(z) = w_0 + w_a (1 - a) = w_0 + w_a \\frac{z}{1+z}`.\n\nastropy.cosmology._src.flrw.w0wacdm.w0waCDM.de_density_scale:\n  Evaluates the redshift dependence of the dark energy density.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        I : ndarray\n            The scaling of the energy density of dark energy with redshift.\n\n        Notes\n        -----\n        The scaling factor, I, is defined by :math:`\\rho(z) = \\rho_0 I`,\n        and in this case is given by\n\n        .. math::\n\n           I = \\left(1 + z\\right)^{3 \\left(1 + w_0 + w_a\\right)}\n                     \\exp \\left(-3 w_a \\frac{z}{1+z}\\right)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses implementation details (Cython compilation, scalar_inv_efuncs.pyx, __post_init__ branching, frozen dataclass internals, neutrino fitting functions, etc.) that are not present in the provided documentation. These details cannot be verified from the given doc chunk.",
      "The documentation only describes the w0waCDM class's parameters, the dark energy EoS formula w(z) = w0 + wa*z/(1+z), the de_density_scale formula, and basic usage examples. The answer goes far beyond this scope.",
      "However, nothing in the answer directly contradicts what the documentation does state. The formulas referenced (e.g., the CPL form, the de_density_scale expression) are consistent with the documentation.",
      "The answer appears to be responding to a question about performance optimization internals, which would require access to source code not provided in this documentation chunk. The claims about Cython internals, specific function names, argument counts, and performance notes are plausible but entirely unverifiable from the given documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed claims about performance rationales from comments and the Komatsu reference attribution.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_11",
    "repo": "astropy",
    "question": "Where does Flatw0waCDM differ structurally from its non-flat parent w0waCDM, and how does flatness get enforced throughout the class hierarchy?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Flatw0waCDM` class is defined in `astropy/cosmology/_src/flrw/w0wacdm.py` and inherits from both `FlatFLRWMixin` and `w0waCDM` (MRO: `FlatFLRWMixin` first, then `w0waCDM`). It does not define `w0` or `wa` parameters itself \u2014 those are inherited from `w0waCDM`, where `w0` defaults to `-1.0` and `wa` defaults to `0.0`, both using `fvalidate=\"float\"`.\n\nThe key structural differences from its non-flat parent `w0waCDM`:\n\n1. **`Ode0` handling** \u2014 `FlatFLRWMixin` redefines `Ode0` as a derived parameter (not an `__init__` parameter), ensuring flatness is enforced.\n\n2. **`Ode0` is computed in `FlatFLRWMixin.__post_init__`** to enforce the flatness constraint.\n\n3. **The flat versions use different Cython scalar inverse efuncs** that omit `Ok0` from their arguments. In `Flatw0waCDM.__post_init__`, the three dispatch branches select `fw0wacdm_inv_efunc_norel`, `fw0wacdm_inv_efunc_nomnu`, or `fw0wacdm_inv_efunc` from `scalar_inv_efuncs.pyx`, compared to the non-flat `w0waCDM` which uses `w0wacdm_inv_efunc_norel/nomnu/efunc` (which include `Ok0` as an argument).\n\n4. **The Cython `fw0wacdm_inv_efunc_norel`** computes `pow(opz**3 * Om0 + Ode0 * Odescl, -0.5)` (no curvature term), while the non-flat `w0wacdm_inv_efunc_norel` computes `pow(opz**2 * (opz * Om0 + Ok0) + Ode0 * Odescl, -0.5)` (with curvature).\n\n5. **The `nonflat` property** \u2014 `__nonflatclass__` is automatically determined by `FlatCosmologyMixin._get_nonflat_cls` which finds the first base that is a `Cosmology` subclass but not a `FlatCosmologyMixin` subclass (visible in core.py). For `Flatw0waCDM`, this resolves to `w0waCDM`.\n\n6. **`Otot0` and `Otot(z)`** in `FlatFLRWMixin` enforce flatness (always returning 1.0).\n\n7. **The `_inv_efunc_scalar` is stored via `object.__setattr__`** because the class is a frozen dataclass (`dataclass_decorator` applies `dataclass(frozen=True, repr=True, eq=False, init=True)` per core.py).\n\n8. The `w()` and `de_density_scale()` methods are inherited from `w0waCDM` without override. `w()` returns `self.w0 + self.wa * z / (z + 1.0)`, and `de_density_scale()` returns `zp1 ** (3 * (1 + self.w0 + self.wa)) * exp(-3 * self.wa * z / zp1)`. Both use `aszarr()` from `astropy/cosmology/_src/utils.py` to convert input redshift.\n\n9. The comoving distance integration path is: `comoving_distance()` \u2192 `_comoving_distance_z1z2()` \u2192 `_integral_comoving_distance_z1z2()` \u2192 `_integral_comoving_distance_z1z2_scalar()` which calls `quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)`.",
    "rubric": [
      "Explains that Flatw0waCDM inherits from both FlatFLRWMixin and w0waCDM, with FlatFLRWMixin first in the MRO",
      "Notes that w0 and wa parameters are inherited from w0waCDM (defaults -1.0 and 0.0, fvalidate='float'), not redefined in Flatw0waCDM",
      "Explains that FlatFLRWMixin redefines Ode0 as a derived parameter (not an __init__ parameter) to enforce flatness",
      "Describes that FlatFLRWMixin.__post_init__ computes Ode0 as 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0) and sets Ok0 to 0.0",
      "Identifies that the flat version uses different Cython scalar inverse efuncs (fw0wacdm_inv_efunc_norel/nomnu/efunc) that omit Ok0 from their arguments, unlike the non-flat versions (w0wacdm_inv_efunc_norel/nomnu/efunc) which include Ok0",
      "Mentions that __nonflatclass__ is resolved via FlatCosmologyMixin._get_nonflat_cls which finds the first base that is a Cosmology subclass but not a FlatCosmologyMixin subclass, yielding w0waCDM",
      "Notes that FlatFLRWMixin.Otot0 always returns 1.0 and Otot(z) returns ones_like to enforce flatness",
      "Explains that object.__setattr__ is used for _inv_efunc_scalar because the class is a frozen dataclass"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0wacdm.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0wacdm.py] astropy.cosmology._src.flrw.w0wacdm.Flatw0waCDM\nastropy.cosmology._src.flrw.w0wacdm.Flatw0waCDM:\n  FLRW cosmology with a CPL dark energy EoS and no curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the CPL form as\n    described in Chevallier & Polarski [1]_ and Linder [2]_:\n    :math:`w(z) = w_0 + w_a (1-a) = w_0 + w_a z / (1+z)`.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at z=0 (a=1). This is pressure/density\n        for dark energy in units where c=1.\n\n    wa : float, optional\n        Negative derivative of the dark energy equation of state with respect\n        to the scale factor. A cosmological constant has w0=-1.0 and wa=0.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Flatw0waCDM\n    >>> cosmo = Flatw0waCDM(H0=70, Om0=0.3, w0=-0.9, wa=0.2)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\n    To get an equivalent cosmology, but of type `astropy.cosmology.w0waCDM`,\n    use :attr:`astropy.cosmology.FlatFLRWMixin.nonflat`.\n\n    >>> print(cosmo.nonflat)\n    w0waCDM(H0=70.0 km / (Mpc s), Om0=0.3, Ode0=0.7, ...\n\n    References\n    ----------\n    .. [1] Chevallier, M., & Polarski, D. (2001). Accelerating Universes with\n           Scaling Dark Matter. International Journal of Modern Physics D,\n           10(2), 213-223.\n    .. [2] Linder, E. (2003). Exploring the Expansion History of the\n           Universe. Phys. Rev. Lett., 90, 091301.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few details about FlatFLRWMixin's Ode0/Otot behavior are in truncated code sections but the core logic is still code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_12",
    "repo": "astropy",
    "question": "How does the `w0wzCDM` cosmology class implement its dark energy equation of state, and how are its inverse E(z) computations optimized?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `w0wzCDM` class implements a FLRW cosmology with a variable dark energy equation of state `w(z) = w0 + wz * z`. Here's how it works at the code level:\n\n**Class hierarchy and decoration**: `w0wzCDM` extends `FLRW` (which itself extends `Cosmology` plus 12 trait mixins including `DarkEnergyComponent`, `HubbleParameter`, `NeutrinoComponent`, etc.). It is decorated with `@dataclass_decorator`, which applies `dataclasses.dataclass(frozen=True, repr=True, eq=False, init=True)` and then precomputes the class's `__signature__` via `_with_signature()`.\n\n**Parameters**: `w0` and `wz` are defined as `Parameter` instances with `fvalidate=\"float\"`. `w0` defaults to `-1.0` and `wz` defaults to `0.0`.\n\n**`__post_init__` optimization**: During initialization, `w0wzCDM.__post_init__` selects one of three Cython-compiled scalar inverse-efunc functions from `scalar_inv_efuncs.pyx` depending on the physics case:\n1. `w0wzcdm_inv_efunc_norel` \u2014 when `Tcmb0 == 0`, takes args `(Om0, Ode0, Ok0, w0, wz)`\n2. `w0wzcdm_inv_efunc_nomnu` \u2014 when `not self._nu_info.has_massive_nu`, takes args `(Om0, Ode0, Ok0, Ogamma0 + Onu0, w0, wz)`\n3. `w0wzcdm_inv_efunc` \u2014 otherwise (massive neutrinos present), takes args `(Om0, Ode0, Ok0, Ogamma0, neff_per_nu, n_massless_nu, nu_y_list, w0, wz)`\n\nThese are stored via `object.__setattr__` (because the dataclass is frozen) as `_inv_efunc_scalar` and `_inv_efunc_scalar_args`.\n\n**Cython scalar functions**: In `scalar_inv_efuncs.pyx`, the dark energy density scale factor is computed as `Odescl = opz**(3. * (1. + w0 - wz)) * exp(3. * wz * z)`, and the full inverse E(z) function returns `pow(... + Ode0 * Odescl, -0.5)`. The Cython file uses `#cython: language_level=3, boundscheck=False` for performance and imports `exp` and `pow` from `libc.math`.\n\n**`w(z)` method**: Returns `self.w0 + self.wz * aszarr(z)`, where `aszarr()` converts the input to a numpy array, handling `Quantity` objects by converting to dimensionless redshift via `z.to_value(cu.redshift)`, plain scalars via `np.asarray(z)`, and plain ndarrays directly.\n\n**`de_density_scale(z)` method**: Computes the analytic closed-form expression `(z + 1.0) ** (3.0 * (1.0 + self.w0 - self.wz)) * exp(3.0 * self.wz * z)` using numpy's `exp`. This overrides the base `DarkEnergyComponent.de_density_scale()` which uses numerical integration via `scipy.integrate.quad` of `_w_integrand` (defined as `1.0 + self.w(exp(ln1pz) - 1.0)`).\n\n**No `inv_efunc` override**: Unlike `wCDM` which overrides `inv_efunc` with a simpler analytic expression, `w0wzCDM` inherits the generic `FLRW.inv_efunc` which calls `self.de_density_scale(z)`. The performance-critical path uses the Cython `_inv_efunc_scalar` functions instead (called from `_integral_comoving_distance_z1z2_scalar` via `scipy.integrate.quad`, and from `_lookback_time_integrand_scalar`).\n\n**`Flatw0wzCDM`**: Inherits from `FlatFLRWMixin` and `w0wzCDM`. `Flatw0wzCDM` uses flat-specific Cython functions (`fw0wzcdm_inv_efunc_norel`, etc.) that omit the `Ok0` term.\n\n**Neutrino handling in Cython**: The `nufunc` in the `.pyx` file is a `cdef` function (not accessible from Python) that computes neutrino relative density with constants `p = 1.83`, `1/p = 0.54644808743`, `k = 0.3173`, and the prefactor `0.22710731766`.",
    "rubric": [
      "Explains that w(z) = w0 + wz * z, implemented using aszarr() to handle redshift input",
      "Describes the de_density_scale method's analytic closed-form: (1+z)^(3*(1+w0-wz)) * exp(3*wz*z), overriding the base class numerical integration",
      "Explains that __post_init__ selects one of three Cython scalar inverse-efunc functions (norel, nomnu, full) based on Tcmb0 and massive neutrino conditions",
      "Mentions that object.__setattr__ is used to set _inv_efunc_scalar and _inv_efunc_scalar_args because the dataclass is frozen",
      "Describes the Cython dark energy density scale factor formula: opz**(3*(1+w0-wz)) * exp(3*wz*z) used in scalar_inv_efuncs.pyx",
      "Notes that Flatw0wzCDM uses flat-specific Cython functions (fw0wzcdm_*) that omit the Ok0 curvature term",
      "Mentions the w0 and wz Parameter definitions with fvalidate='float' and their defaults (-1.0 and 0.0)"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0wzcdm.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/traits/darkenergy.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0wzcdm.py] astropy.cosmology._src.flrw.w0wzcdm.w0wzCDM\nastropy.cosmology._src.flrw.w0wzcdm.w0wzCDM:\n  FLRW cosmology with a variable dark energy EoS and curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the simple form:\n    :math:`w(z) = w_0 + w_z z`.\n\n    This form is not recommended for z > 1.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    Ode0 : float\n        Omega dark energy: density of dark energy in units of the critical\n        density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at z=0. This is pressure/density for\n        dark energy in units where c=1.\n\n    wz : float, optional\n        Derivative of the dark energy equation of state with respect to z.\n        A cosmological constant has w0=-1.0 and wz=0.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import w0wzCDM\n    >>> cosmo = w0wzCDM(H0=70, Om0=0.3, Ode0=0.7, w0=-0.9, wz=0.2)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\nastropy.cosmology._src.flrw.w0wzcdm.w0wzCDM.w:\n  Returns dark energy equation of state at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        w : ndarray\n            The dark energy equation of state.\n\n        Notes\n        -----\n        The dark energy equation of state is defined as\n        :math:`w(z) = P(z)/\\rho(z)`, where :math:`P(z)` is the pressure at\n        redshift z and :math:`\\rho(z)` is the density at redshift z, both in\n        units where c=1. Here this is given by :math:`w(z) = w_0 + w_z z`.\n\nastropy.cosmology._src.flrw.w0wzcdm.w0wzCDM.de_density_scale:\n  Evaluates the redshift dependence of the dark energy density.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        I : ndarray\n            The scaling of the energy density of dark energy with redshift.\n\n        References\n        ----------\n        .. [1] Linder, E. (2003). Exploring the Expansion History of the Universe.\n               Physics Review Letters, 90(9), 091301.\n\n        Notes\n        -----\n        The scaling factor, I, is defined by :math:`\\rho(z) = \\rho_0 I`,\n        and in this case is given by ([1]_)\n\n        .. math::\n\n           I = \\left(1 + z\\right)^{3 \\left(1 + w_0 - w_z\\right)}\n                     \\exp \\left(3 w_z z\\right)",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many claims about internal implementation details (Cython functions, dataclass decorators, __post_init__ optimization, neutrino handling constants, etc.) go well beyond what the documentation covers, but none of them contradict the documentation.",
      "The core documented behavior \u2014 w(z) = w0 + wz*z, the de_density_scale formula, parameter descriptions, and class purpose \u2014 are all accurately represented in the answer.",
      "The de_density_scale formula in the answer matches the documented math: (1+z)^{3(1+w0-wz)} * exp(3*wz*z).",
      "The w(z) = w0 + wz*z equation matches the documentation.",
      "The claim about Flatw0wzCDM, FlatFLRWMixin, Cython scalar functions, etc. are implementation details that cannot be verified from the provided documentation but are plausible code-level extrapolations."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; minor leakage from comments about Komatsu reference, performance rationale, and physical interpretation of constants.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_14",
    "repo": "astropy",
    "question": "Why does `wpwaCDM.__post_init__` use `object.__setattr__` and select among three different scalar inverse-efunc functions?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `wpwaCDM` class uses a design pattern of precomputing optimized scalar inverse-efunc functions during `__post_init__` for performance-critical numerical integration. The key design rationale is:\n\n1. **Frozen dataclass + `object.__setattr__`**: The cosmology classes are decorated with `dataclass_decorator` which applies `dataclass(frozen=True, repr=True, eq=False, init=True)`. Since frozen dataclasses forbid attribute assignment, `object.__setattr__` is used to bypass the freeze and cache the precomputed `_inv_efunc_scalar` and `_inv_efunc_scalar_args` attributes. This ensures immutability for user-facing parameters while allowing internal caching of computed attributes.\n\n2. **Three-tier optimization for neutrino handling**: The `__post_init__` selects among three different Cython-implemented inverse-efunc functions based on the physical scenario: (a) `wpwacdm_inv_efunc_norel` when `Tcmb0 == 0`, (b) `wpwacdm_inv_efunc_nomnu` when there are no massive neutrinos, and (c) `wpwacdm_inv_efunc` when massive neutrinos exist (which calls `nufunc`).\n\n3. **Precomputed `apiv` pivot scale factor**: The pivot scale factor `apiv = 1.0 / (1.0 + self.zp.value)` is precomputed once and stored as part of `_inv_efunc_scalar_args`. The base `FLRW.__post_init__` sets `_inv_efunc_scalar` to the generic `self.inv_efunc` method; subclasses override this with the specialized Cython function.\n\n4. **Cython scalar functions for `quad` integration**: The `_inv_efunc_scalar` is directly passed to `scipy.integrate.quad` in `_integral_comoving_distance_z1z2_scalar`. The Cython functions use `pow(x, -0.5)` and hardwired fitting constants.\n\n5. **Relationship to `w0waCDM`**: The `wpwaCDM` dark energy density scaling uses `opz**(3*(1+wp+apiv*wa))` while `w0waCDM` uses `opz**(3*(1+w0+wa))`. When `zp=0`, `apiv=1.0` and the formulas become identical (with `wp` corresponding to `w0`). The default value of `zp` is `0.0 * cu.redshift`, making `wpwaCDM` reduce to `w0waCDM` behavior by default.\n\n6. **`zp` stored as Quantity with unit `cu.redshift`**: Unlike `wp` and `wa` which use `fvalidate='float'` (storing plain floats), `zp` is defined with `unit=cu.redshift` and uses the default validator (`validate_with_unit`), storing it as a `Quantity`. This is why `.value` is used to extract the numeric value in `w()`, `de_density_scale()`, and `wpwaCDM.__post_init__()`. The `FlatwpwaCDM.__post_init__()` uses `self.zp` without `.value`, which produces a Quantity result for `apiv` that gets passed to the Cython function.\n\n7. **Python lists vs NumPy arrays for `nu_y`**: The `NeutrinoInfo` NamedTuple stores `nu_y_list` separately from `nu_y`, and the Cython functions declare their parameter as `list nu_y`, indicating lists are used for the Cython loop in `nufunc`.",
    "rubric": [
      "Explains that cosmology classes are frozen dataclasses, so `object.__setattr__` is needed to bypass immutability for internal caching",
      "Describes the three-tier optimization: norel (Tcmb0==0), nomnu (no massive neutrinos), and full (with massive neutrinos/nufunc)",
      "Mentions that `_inv_efunc_scalar` is precomputed for performance-critical numerical integration (passed to scipy.integrate.quad)",
      "Notes that the base FLRW class sets `_inv_efunc_scalar` to the generic `self.inv_efunc`, and subclasses override with specialized/Cython scalar functions",
      "Explains that `apiv` (pivot scale factor) is precomputed from `zp` and stored as part of the cached args tuple"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/wpwazpcdm.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/parameter/converter.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/wpwazpcdm.py] astropy.cosmology._src.flrw.wpwazpcdm.wpwaCDM\nastropy.cosmology._src.flrw.wpwazpcdm.wpwaCDM:\n  FLRW cosmology with a CPL dark energy EoS, a pivot redshift, and curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the CPL form as\n    described in Chevallier & Polarski [1]_ and Linder [2]_, but modified to\n    have a pivot redshift as in the findings of the Dark Energy Task Force\n    [3]_: :math:`w(a) = w_p + w_a (a_p - a) = w_p + w_a( 1/(1+zp) - 1/(1+z) )`.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    Ode0 : float\n        Omega dark energy: density of dark energy in units of the critical\n        density at z=0.\n\n    wp : float, optional\n        Dark energy equation of state at the pivot redshift zp. This is\n        pressure/density for dark energy in units where c=1.\n\n    wa : float, optional\n        Negative derivative of the dark energy equation of state with respect\n        to the scale factor. A cosmological constant has wp=-1.0 and wa=0.0.\n\n    zp : float or quantity-like ['redshift'], optional\n        Pivot redshift -- the redshift where w(z) = wp\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import wpwaCDM\n    >>> cosmo = wpwaCDM(H0=70, Om0=0.3, Ode0=0.7, wp=-0.9, wa=0.2, zp=0.4)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> z = 0.5\n    >>> dc = cosmo.comoving_distance(z)\n\n    References\n    ----------\n    .. [1] Chevallier, M., & Polarski, D. (2001). Accelerating Universes with\n           Scaling Dark Matter. International Journal of Modern Physics D,\n           10(2), 213-223.\n    .. [2] Linder, E. (2003). Exploring the Expansion History of the\n           Universe. Phys. Rev. Lett., 90, 091301.\n    .. [3] Albrecht, A., Amendola, L., Bernstein, G., Clowe, D., Eisenstein,\n           D., Guzzo, L., Hirata, C., Huterer, D., Kirshner, R., Kolb, E., &\n           Nichol, R. (2009). Findings of the Joint Dark Energy Mission Figure\n           of Merit Science Working Group. arXiv e-prints, arXiv:0901.0721.\n\nastropy.cosmology._src.flrw.wpwazpcdm.wpwaCDM.w:\n  Returns dark energy equation of state at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        w : ndarray\n            The dark energy equation of state\n\n        Notes\n        -----\n        The dark energy equation of state is defined as\n        :math:`w(z) = P(z)/\\rho(z)`, where :math:`P(z)` is the pressure at\n        redshift z and :math:`\\rho(z)` is the density at redshift z, both in\n        units where c=1. Here this is :math:`w(z) = w_p + w_a (a_p - a)` where\n        :math:`a = 1/1+z` and :math:`a_p = 1 / 1 + z_p`.\n\nastropy.cosmology._src.flrw.wpwazpcdm.wpwaCDM.de_density_scale:\n  Evaluates the redshift dependence of the dark energy density.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'] or array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        I : ndarray\n            The scaling of the energy density of dark energy with redshift.\n\n        Notes\n        -----\n        The scaling factor, I, is defined by :math:`\\rho(z) = \\rho_0 I`,\n        and in this case is given by\n\n        .. math::\n\n           a_p = \\frac{1}{1 + z_p}\n\n           I = \\left(1 + z\\right)^{3 \\left(1 + w_p + a_p w_a\\right)}\n                     \\exp \\left(-3 w_a \\frac{z}{1+z}\\right)",
    "verification_verdict": "warn",
    "verification_issues": [
      "Point 5 claims that when zp=0, apiv=1.0 and the formulas become identical with wp corresponding to w0. The documentation shows the dark energy density scaling as (1+z)^{3(1+wp+a_p*wa)} * exp(-3*wa*z/(1+z)). When zp=0, a_p=1, so the exponent becomes 3(1+wp+wa), which matches w0waCDM's (1+z)^{3(1+w0+wa)} form. This is plausible but not explicitly stated in the documentation.",
      "Point 5 claims the default value of zp is '0.0 * cu.redshift'. The documentation does not specify a default value for zp, only stating it is 'optional'. This is an unverifiable claim from the documentation alone.",
      "Points 1, 2, 3, 4, 6, 7 discuss implementation details (Cython functions, dataclass decorators, frozen attributes, nu_y lists, etc.) that are not mentioned anywhere in the provided documentation. These are plausible code-level implementation details but cannot be verified from the given docstrings.",
      "The answer discusses internal implementation patterns extensively. While these may be accurate descriptions of the actual code, the documentation provided only covers the public API (parameters, return values, mathematical formulas). None of these implementation claims are contradicted by the documentation, but they are entirely unsupported by it."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few performance rationale statements and one direct quote from comments constitute doc leakage.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_15",
    "repo": "astropy",
    "question": "Where does the data flow when `FlatwpwaCDM.comoving_distance(z)` is called, and how does it reach the Cython integration kernel?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `FlatwpwaCDM` class's `comoving_distance(z)` method ultimately computes its result through a chain that flows from `FLRW.comoving_distance()` \u2192 `_comoving_distance_z1z2(0.0, z)` \u2192 `_integral_comoving_distance_z1z2(z1, z2)` \u2192 `hubble_distance * _integral_comoving_distance_z1z2_scalar(z1, z2)` \u2192 `scipy.integrate.quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)`. The `_inv_efunc_scalar` function and its `_inv_efunc_scalar_args` are set during `FlatwpwaCDM.__post_init__()`, which calls `super().__post_init__()` first (resolving to `FlatFLRWMixin.__post_init__()`, which sets `Ok0=0.0` for flat cosmologies), and then overrides `_inv_efunc_scalar` with one of three Cython functions from `scalar_inv_efuncs.pyx`: `fwpwacdm_inv_efunc_norel` (when `Tcmb0==0`), `fwpwacdm_inv_efunc_nomnu` (no massive neutrinos), or `fwpwacdm_inv_efunc` (massive neutrinos). These flat variants differ from the non-flat `wpwacdm_*` versions by omitting the `Ok0` parameter entirely. The dark energy density scale used in the Cython functions computes `Odescl = opz**(3*(1+wp+apiv*wa)) * exp(-3*wa*z/opz)` where `apiv = 1/(1+zp)`. The `_integral_comoving_distance_z1z2_scalar` method is decorated with `@vectorize_redshift_method(nin=2)`, which handles scalar inputs directly but uses `np.vectorize` for array inputs.",
    "rubric": [
      "Mentions that `comoving_distance` delegates to `_comoving_distance_z1z2(0.0, z)` in the FLRW base class",
      "Mentions that `_comoving_distance_z1z2` calls `_integral_comoving_distance_z1z2` which multiplies hubble_distance by the scalar integration result",
      "Mentions that `_integral_comoving_distance_z1z2_scalar` uses `scipy.integrate.quad` with `self._inv_efunc_scalar` and `self._inv_efunc_scalar_args`",
      "Explains that `_inv_efunc_scalar` is set during `FlatwpwaCDM.__post_init__` to one of three Cython functions: `fwpwacdm_inv_efunc_norel`, `fwpwacdm_inv_efunc_nomnu`, or `fwpwacdm_inv_efunc` depending on Tcmb0 and massive neutrino presence",
      "Notes that `FlatFLRWMixin.__post_init__` sets Ok0=0.0 and the flat Cython functions omit Ok0 unlike the non-flat wpwacdm variants",
      "Mentions the `@vectorize_redshift_method(nin=2)` decorator handles scalar inputs directly but uses `np.vectorize` for array inputs"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/wpwazpcdm.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/scipy_compat.py",
      "astropy/cosmology/_src/utils.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/wpwazpcdm.py] astropy.cosmology._src.flrw.wpwazpcdm.FlatwpwaCDM\nastropy.cosmology._src.flrw.wpwazpcdm.FlatwpwaCDM:\n  FLRW cosmology with a CPL dark energy EoS, a pivot redshift, and no curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the CPL form as\n    described in Chevallier & Polarski [1]_ and Linder [2]_, but modified to\n    have a pivot redshift as in the findings of the Dark Energy Task Force\n    [3]_: :math:`w(a) = w_p + w_a (a_p - a) = w_p + w_a( 1/(1+zp) - 1/(1+z) )`.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    wp : float, optional\n        Dark energy equation of state at the pivot redshift zp. This is\n        pressure/density for dark energy in units where c=1.\n\n    wa : float, optional\n        Negative derivative of the dark energy equation of state with respect\n        to the scale factor. A cosmological constant has wp=-1.0 and wa=0.0.\n\n    zp : float or quantity-like ['redshift'], optional\n        Pivot redshift -- the redshift where w(z) = wp\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import FlatwpwaCDM\n    >>> cosmo = FlatwpwaCDM(H0=70, Om0=0.3, wp=-0.9, wa=0.2, zp=0.4)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> cosmo.comoving_distance(0.5)\n    <Quantity 1868.68474438 Mpc>\n\n    References\n    ----------\n    .. [1] Chevallier, M., & Polarski, D. (2001). Accelerating Universes with\n        Scaling Dark Matter. International Journal of Modern Physics D,\n        10(2), 213-223.\n    .. [2] Linder, E. (2003). Exploring the Expansion History of the\n        Universe. Phys. Rev. Lett., 90, 091301.\n    .. [3] Albrecht, A., Amendola, L., Bernstein, G., Clowe, D., Eisenstein,\n        D., Guzzo, L., Hirata, C., Huterer, D., Kirshner, R., Kolb, E., &\n        Nichol, R. (2009). Findings of the Joint Dark Energy Mission Figure\n        of Merit Science Working Group. arXiv e-prints, arXiv:0901.0721.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The detailed implementation chain (scipy.integrate.quad, Cython functions, __post_init__ behavior, vectorize_redshift_method decorator) cannot be verified from the provided documentation alone - these are implementation details that go beyond what the docstring describes.",
      "The dark energy density scale formula 'Odescl = opz**(3*(1+wp+apiv*wa)) * exp(-3*wa*z/opz)' is a plausible implementation detail but cannot be verified from the documentation provided.",
      "The claim about FlatFLRWMixin.__post_init__() setting Ok0=0.0 and computing Ode0 is reasonable for a flat cosmology but is not stated in the documentation.",
      "The documentation only shows the EoS formula w(a) = w_p + w_a(a_p - a), the parameters, and example output. The answer's claims about internal implementation are extrapolations from code knowledge rather than the provided documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; only the specific Ode0 formula from FlatFLRWMixin (truncated in provided code) needed minor adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_13",
    "repo": "astropy",
    "question": "What is the full class hierarchy and architecture of the Flatw0wzCDM cosmology class, including how it enforces flatness and optimizes distance integrals?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `Flatw0wzCDM` class is a flat FLRW cosmology with a variable dark energy equation of state of the form `w(z) = w0 + wz * z`. Here is its architecture:\n\n**Class Hierarchy (MRO):**\n`Flatw0wzCDM` inherits from `FlatFLRWMixin` and `w0wzCDM` (in that order). `w0wzCDM` inherits from `FLRW`, which inherits from `Cosmology` and multiple trait mixins: `BaryonComponent`, `TotalComponent`, `CriticalDensity`, `MatterComponent`, `CurvatureComponent`, `DarkEnergyComponent`, `HubbleParameter`, `PhotonComponent`, `NeutrinoComponent`, `ScaleFactor`, `DarkMatterComponent`, and `TemperatureCMB`. `FlatFLRWMixin` inherits from `FlatCosmologyMixin`.\n\n**Dataclass Pattern:**\nAll cosmology classes are decorated with `@dataclass_decorator`, which applies `dataclasses.dataclass(frozen=True, repr=True, eq=False, init=True)` and then precomputes the class signature via `_with_signature`.\n\n**Parameter Definitions:**\nIn `w0wzCDM`, `w0` is a `Parameter` with default `-1.0` and `fvalidate=\"float\"`. `wz` is a `Parameter` with default `0.0` and `fvalidate=\"float\"`. The base `FLRW` class defines `H0` (unit='km/(s Mpc)', fvalidate='scalar'), `Om0` (fvalidate='non-negative'), `Ode0` (fvalidate='float'), `Tcmb0` (default=0.0*u.K, unit='Kelvin'), `Neff` (default=3.04), `m_nu` (default=0.0*u.eV, unit='eV', equivalencies=u.mass_energy()), and `Ob0` (default=0.0).\n\n**Flatness Enforcement:**\nIn `FlatFLRWMixin`, `Ode0` is redefined as a derived parameter (`derived=True`, `init=False`, `repr=False`). Its `__post_init__` sets `Ok0 = 0.0` and computes `Ode0 = 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)`, eliminating curvature. The `__init_subclass__` method enforces that no subclass can have `Ode0` in `__init__`. The `Otot0` property always returns `1.0`, and `Otot(z)` returns `np.ones_like(...)`. The `__nonflatclass__` is automatically resolved to `w0wzCDM` (the non-flat parent) via `FlatCosmologyMixin._get_nonflat_cls`, which searches the MRO bases for a `Cosmology` subclass that is not a `FlatCosmologyMixin` subclass.\n\n**Fast Scalar Integration:**\nBoth `w0wzCDM.__post_init__` and `Flatw0wzCDM.__post_init__` select optimized Cython scalar inverse efunc implementations from `scalar_inv_efuncs.pyx` based on three cases:\n1. **No relativistic species** (`Tcmb0.value == 0`): uses `fw0wzcdm_inv_efunc_norel` (flat) or `w0wzcdm_inv_efunc_norel` (non-flat)\n2. **Massless neutrinos only** (`not self._nu_info.has_massive_nu`): uses `fw0wzcdm_inv_efunc_nomnu`/`w0wzcdm_inv_efunc_nomnu`, with combined radiation `Ogamma0 + Onu0`\n3. **Massive neutrinos**: uses `fw0wzcdm_inv_efunc`/`w0wzcdm_inv_efunc`, passing `Ogamma0`, `neff_per_nu`, `n_massless_nu`, and `nu_y_list`\n\nThe key difference: flat variants omit the `Ok0` parameter from the Cython function arguments and use a simplified energy density formula without the curvature term.\n\nThese scalar functions are stored via `object.__setattr__` (bypassing frozen dataclass immutability) as `_inv_efunc_scalar` and `_inv_efunc_scalar_args`.\n\n**Dark Energy Density Scaling:**\n`w0wzCDM.de_density_scale(z)` returns `(z + 1.0) ** (3.0 * (1.0 + w0 - wz)) * exp(3.0 * wz * z)`, overriding the generic numerical integral in `DarkEnergyComponent.de_density_scale` (which uses `scipy.integrate.quad` on `_w_integrand`). The Cython functions also compute the same `Odescl = opz**(3*(1+w0-wz)) * exp(3*wz*z)` inline.\n\n**Comoving Distance Computation Chain:**\n`comoving_distance(z)` \u2192 `_comoving_distance_z1z2(0, z)` \u2192 `_integral_comoving_distance_z1z2(z1, z2)` \u2192 `hubble_distance * _integral_comoving_distance_z1z2_scalar(z1, z2)`. The scalar method (decorated with `@vectorize_redshift_method(nin=2)`) calls `scipy.integrate.quad(self._inv_efunc_scalar, z1, z2, args=self._inv_efunc_scalar_args)`. `hubble_distance` is `(c / H0).to(u.Mpc)` from `HubbleParameter` trait.\n\n**`w(z)` Method:**\n`w0wzCDM.w(z)` returns `self.w0 + self.wz * aszarr(z)`, where `aszarr` converts redshift inputs (Quantity or array-like) to numpy arrays. `Flatw0wzCDM` inherits this method without overriding it.\n\n**Neutrino Information:**\nThe `NeutrinoInfo` NamedTuple (defined in base.py) stores: `n_nu`, `neff_per_nu`, `has_massive_nu`, `n_massive_nu`, `n_massless_nu`, `nu_y`, and `nu_y_list`. The `nu_y` values are computed as `m_nu / (kB_evK * Tnu0)`. The Cython `nufunc` implements a Komatsu et al. 2011 fitting function with constants p=1.83, 1/p=0.54644808743, k=0.3173, and the fermion/boson factor 0.22710731766 (= 7/8 * (4/11)^(4/3)).",
    "rubric": [
      "Identifies that Flatw0wzCDM inherits from FlatFLRWMixin and w0wzCDM, with w0wzCDM inheriting from FLRW, and FlatFLRWMixin inheriting from FlatCosmologyMixin",
      "Lists the trait mixins that FLRW inherits from (Cosmology, BaryonComponent, TotalComponent, CriticalDensity, MatterComponent, CurvatureComponent, DarkEnergyComponent, HubbleParameter, PhotonComponent, NeutrinoComponent, ScaleFactor, DarkMatterComponent, TemperatureCMB)",
      "Explains that classes are decorated with dataclass_decorator which applies dataclass(frozen=True, repr=True, eq=False, init=True) and precomputes signature",
      "Describes the w0 and wz Parameter definitions in w0wzCDM (defaults -1.0 and 0.0, fvalidate='float')",
      "Explains flatness enforcement: FlatFLRWMixin redefines Ode0 as derived (init=False), __post_init__ sets Ok0=0.0 and computes Ode0 = 1 - (Om0 + Ogamma0 + Onu0 + Ok0)",
      "Describes the three-case selection of Cython scalar_inv_efuncs (norel, nomnu, massive) for both flat and non-flat variants",
      "Notes that flat variants omit Ok0 from the Cython function arguments compared to non-flat variants",
      "Explains that object.__setattr__ is used to bypass frozen dataclass immutability when storing _inv_efunc_scalar and _inv_efunc_scalar_args",
      "Describes de_density_scale formula: (z+1)^(3*(1+w0-wz)) * exp(3*wz*z)",
      "Explains the comoving distance computation chain: comoving_distance \u2192 _comoving_distance_z1z2 \u2192 _integral_comoving_distance_z1z2 \u2192 hubble_distance * _integral_comoving_distance_z1z2_scalar, where the scalar method uses scipy.integrate.quad with _inv_efunc_scalar"
    ],
    "key_files": [
      "astropy/cosmology/_src/flrw/w0wzcdm.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/traits/darkenergy.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/traits/hubble.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/flrw/w0wzcdm.py] astropy.cosmology._src.flrw.w0wzcdm.Flatw0wzCDM\nastropy.cosmology._src.flrw.w0wzcdm.Flatw0wzCDM:\n  FLRW cosmology with a variable dark energy EoS and no curvature.\n\n    The equation for the dark energy equation of state (EoS) uses the simple form:\n    :math:`w(z) = w_0 + w_z z`.\n\n    This form is not recommended for z > 1.\n\n    Parameters\n    ----------\n    H0 : float or scalar quantity-like ['frequency']\n        Hubble constant at z = 0. If a float, must be in [km/sec/Mpc].\n\n    Om0 : float\n        Omega matter: density of non-relativistic matter in units of the\n        critical density at z=0.\n\n    w0 : float, optional\n        Dark energy equation of state at z=0. This is pressure/density for\n        dark energy in units where c=1.\n\n    wz : float, optional\n        Derivative of the dark energy equation of state with respect to z.\n        A cosmological constant has w0=-1.0 and wz=0.0.\n\n    Tcmb0 : float or scalar quantity-like ['temperature'], optional\n        Temperature of the CMB z=0. If a float, must be in [K]. Default: 0 [K].\n        Setting this to zero will turn off both photons and neutrinos\n        (even massive ones).\n\n    Neff : float, optional\n        Effective number of Neutrino species. Default 3.04.\n\n    m_nu : quantity-like ['energy', 'mass'] or array-like, optional\n        Mass of each neutrino species in [eV] (mass-energy equivalency enabled).\n        If this is a scalar Quantity, then all neutrino species are assumed to\n        have that mass. Otherwise, the mass of each species. The actual number\n        of neutrino species (and hence the number of elements of m_nu if it is\n        not scalar) must be the floor of Neff. Typically this means you should\n        provide three neutrino masses unless you are considering something like\n        a sterile neutrino.\n\n    Ob0 : float, optional\n        Omega baryons: density of baryonic matter in units of the critical\n        density at z=0.\n\n    name : str or None (optional, keyword-only)\n        Name for this cosmological object.\n\n    meta : mapping or None (optional, keyword-only)\n        Metadata for the cosmology, e.g., a reference.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Flatw0wzCDM\n    >>> cosmo = Flatw0wzCDM(H0=70, Om0=0.3, w0=-0.9, wz=0.2)\n\n    The comoving distance in Mpc at redshift z:\n\n    >>> cosmo.comoving_distance(0.5)\n    <Quantity 1849.74726272 Mpc>",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_16",
    "repo": "astropy",
    "question": "How does `cosmology_equal` implement its equality checking, including the decorator, format parsing, and the difference between `__eq__` and `__equiv__`?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `cosmology_equal` function in `astropy/cosmology/_src/funcs/comparison.py` implements element-wise equality checking for cosmologies through a layered design involving a decorator pattern, format parsing via NumPy ufuncs, and dual equality mechanisms on the `Cosmology` class.\n\n## Implementation Flow\n\n1. **Decorator Layer (`_comparison_decorator`)**: The `cosmology_equal` function is wrapped by `@_comparison_decorator`, which intercepts calls. The decorator uses `inspect.signature` on the wrapped function and counts positional-only parameters (where `p.kind == 0`, corresponding to `inspect.Parameter.POSITIONAL_ONLY`) to determine `nin` (which equals 2 for `cosmology_equal`). If more than `nin` positional arguments are passed, it raises a `TypeError`. It then calls `_parse_formats(*cosmos, format=format)` to convert inputs before passing them to the actual comparison function.\n\n2. **Format Parsing (`_parse_formats` and `_parse_format`)**: `_parse_formats` broadcasts the `format` argument (as a numpy object array) to match the number of cosmologies using `np.broadcast_to`. Objects that cannot be used safely in numpy arrays (`table.Row` and `table.Table`, stored in `_CANT_BROADCAST`) are wrapped in a frozen `_CosmologyWrapper` dataclass (which has `__slots__ = ('wrapped',)`) before being passed to `_parse_format`.\n\n   `_parse_format` is a numpy ufunc created via `@functools.partial(np.frompyfunc, nin=2, nout=1)`. Inside it:\n   - If input is a `_CosmologyWrapper`, it unwraps it first.\n   - If input is already a `Cosmology`, it validates that `format` is in `_COSMO_AOK = {None, True_, False_, 'astropy.cosmology'}` (where `True_` and `False_` are `numpy.True_` and `numpy.False_`); if not, it raises a `ValueError`.\n   - If not a `Cosmology` and `format == False_`, it raises `TypeError`.\n   - Otherwise, `True`/`True_` are converted to `None`, and `Cosmology.from_format(cosmo, format=format)` is called.\n\n3. **Equality Check**: After parsing, the actual `cosmology_equal` body runs:\n   - **Without `allow_equivalent`** (default): Uses `cosmo1 == cosmo2`, which calls `Cosmology.__eq__`. This checks: (a) same class (`other.__class__ is self.__class__`), (b) `self.name == other.name`, (c) `self._parameters_all == other._parameters_all` (a `frozenset` of all parameter names), and (d) element-wise equality of each parameter value using `np.all(getattr(self, k) == getattr(other, k))`. Returns `NotImplemented` for different classes.\n   - **With `allow_equivalent=True`**: Uses `cosmo1.__equiv__(cosmo2)`. The base `Cosmology.__equiv__` returns `NotImplemented` if classes differ, or checks parameter-set equality and parameter-value equality (but does NOT check `name`). If the first call returns `NotImplemented`, it tries `cosmo2.__equiv__(cosmo1)`. If both return `NotImplemented`, the result is `False`.\n\n4. **Flat Cosmology Equivalence (`FlatCosmologyMixin.__equiv__`)**: This overrides `__equiv__` to handle cross-class equivalence. If `other` is also a `FlatCosmologyMixin`, it delegates to `Cosmology.__equiv__` via `super()`. If `other` is an instance of `self.__nonflatclass__`, it checks: (a) `_parameters_all` match, (b) all parameter values match, AND (c) `other.is_flat` is `True`. This enables `cosmology_equal` with `allow_equivalent=True` to return `True` for a flat cosmology compared against its non-flat counterpart when the non-flat instance is geometrically flat.\n\n5. **Circular dependency with `is_equivalent`**: The `Cosmology.is_equivalent()` method imports `cosmology_equal` from `astropy.cosmology._src.funcs` (lazy import) and calls it with `format=(None, format)` and `allow_equivalent=True`, wrapped in a try/except that returns `False` on any exception.\n\n6. **`_cosmology_not_equal`**: Also decorated with `@_comparison_decorator`, it simply negates `cosmology_equal` \u2014 `return not cosmology_equal(cosmo1, cosmo2, allow_equivalent=allow_equivalent)`.\n\n7. **Key difference between `__eq__` and `__equiv__`**: `__eq__` checks the cosmology `name` field in addition to parameters and class, while `__equiv__` does NOT check `name` \u2014 it only checks parameter sets and values.",
    "rubric": [
      "Explains that `cosmology_equal` is wrapped by `_comparison_decorator`, which uses `inspect.signature` to count positional-only parameters (kind == 0) to determine `nin`, and raises TypeError if too many positional arguments are passed",
      "Describes that `_parse_formats` broadcasts the `format` argument as a numpy object array to match cosmology count, wraps objects in `_CANT_BROADCAST` (table.Row, table.Table) with a frozen `_CosmologyWrapper` dataclass having `__slots__ = ('wrapped',)`",
      "Explains that `_parse_format` is a numpy ufunc (via `np.frompyfunc`), which unwraps `_CosmologyWrapper`, validates format is in `_COSMO_AOK` for existing Cosmology objects, raises TypeError if format is False_ for non-Cosmology inputs, or calls `Cosmology.from_format`",
      "Describes that without `allow_equivalent`, `cosmo1 == cosmo2` invokes `__eq__` which checks same class, same name, same `_parameters_all` frozenset, and element-wise parameter equality",
      "Describes that with `allow_equivalent=True`, `cosmo1.__equiv__(cosmo2)` is tried first, and if it returns NotImplemented, `cosmo2.__equiv__(cosmo1)` is tried; if both return NotImplemented, result is False",
      "Explains that `__equiv__` (base Cosmology) does NOT check name \u2014 only checks class identity, parameter sets, and parameter values \u2014 unlike `__eq__` which does check name",
      "Explains that `FlatCosmologyMixin.__equiv__` handles cross-class equivalence by checking if `other` is an instance of `self.__nonflatclass__`, comparing parameters, and verifying `other.is_flat`",
      "Mentions that `Cosmology.is_equivalent()` imports and calls `cosmology_equal` with `format=(None, format)` and `allow_equivalent=True`, wrapped in try/except returning False on exception"
    ],
    "key_files": [
      "astropy/cosmology/_src/funcs/comparison.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/funcs/__init__.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/funcs/comparison.py] astropy.cosmology._src.funcs.comparison.cosmology_equal\nReturn element-wise equality check on the cosmologies.\n\n    .. note::\n\n        Cosmologies are currently scalar in their parameters.\n\n    Parameters\n    ----------\n    cosmo1, cosmo2 : |Cosmology|-like\n        The objects to compare. Must be convertible to |Cosmology|, as specified\n        by ``format``.\n\n    format : bool or None or str or tuple thereof, optional keyword-only\n        Whether to allow the arguments to be converted to a |Cosmology|. This\n        allows, e.g. a |Table| to be given instead a |Cosmology|. `False`\n        (default) will not allow conversion. `True` or `None` will, and will use\n        the auto-identification to try to infer the correct format. A `str` is\n        assumed to be the correct format to use when converting. Note ``format``\n        is broadcast as an object array to match the shape of ``cosmos`` so\n        ``format`` cannot determine the output shape.\n\n    allow_equivalent : bool, optional keyword-only\n        Whether to allow cosmologies to be equal even if not of the same class.\n        For example, an instance of |LambdaCDM| might have :math:`\\Omega_0=1`\n        and :math:`\\Omega_k=0` and therefore be flat, like |FlatLambdaCDM|.\n\n    Examples\n    --------\n    Assuming the following imports\n\n        >>> import astropy.units as u\n        >>> from astropy.cosmology import FlatLambdaCDM\n\n    Two identical cosmologies are equal.\n\n        >>> cosmo1 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3)\n        >>> cosmo2 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3)\n        >>> cosmology_equal(cosmo1, cosmo2)\n        True\n\n    And cosmologies with different parameters are not.\n\n        >>> cosmo3 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.4)\n        >>> cosmology_equal(cosmo1, cosmo3)\n        False\n\n    Two cosmologies may be equivalent even if not of the same class. In these\n    examples the |LambdaCDM| has :attr:`~astropy.cosmology.LambdaCDM.Ode0` set\n    to the same value calculated in |FlatLambdaCDM|.\n\n        >>> from astropy.cosmology import LambdaCDM\n        >>> cosmo3 = LambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3, 0.7)\n        >>> cosmology_equal(cosmo1, cosmo3)\n        False\n        >>> cosmology_equal(cosmo1, cosmo3, allow_equivalent=True)\n        True\n\n    While in this example, the cosmologies are not equivalent.\n\n        >>> cosmo4 = FlatLambdaCDM(70 * (u.km/u.s/u.Mpc), 0.3, Tcmb0=3 * u.K)\n        >>> cosmology_equal(cosmo3, cosmo4, allow_equivalent=True)\n        False\n\n    Also, using the keyword argument, the notion of equality is extended to any\n    Python object that can be converted to a |Cosmology|.\n\n        >>> mapping = cosmo2.to_format(\"mapping\")\n        >>> cosmology_equal(cosmo1, mapping, format=True)\n        True\n\n    Either (or both) arguments can be |Cosmology|-like.\n\n        >>> cosmology_equal(mapping, cosmo2, format=True)\n        True\n\n    The list of valid formats, e.g. the |Table| in this example, may be checked\n    with ``Cosmology.from_format.list_formats()``.\n\n    As can be seen in the list of formats, not all formats can be\n    auto-identified by ``Cosmology.from_format.registry``. Objects of these\n    kinds can still be checked for equality, but the correct format string must\n    be used.\n\n        >>> yml = cosmo2.to_format(\"yaml\")\n        >>> cosmology_equal(cosmo1, yml, format=(None, \"yaml\"))\n        True\n\n    This also works with an array of ``format`` matching the number of\n    cosmologies.\n\n        >>> cosmology_equal(mapping, yml, format=[True, \"yaml\"])\n        True",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are directly verifiable from stripped code; only a TODO comment reference and minor rationale phrases needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_17",
    "repo": "astropy",
    "question": "What does the identity conversion system for Cosmology objects do, and how are its functions registered and invoked?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `from_cosmology` function is an identity/passthrough format converter registered in astropy's cosmology I/O system under the format name `\"astropy.cosmology\"`. It is part of a trio of functions (`from_cosmology`, `to_cosmology`, `cosmology_identify`) in `astropy/cosmology/_src/io/builtin/cosmology.py` that together provide the identity conversion for cosmology objects.\n\n**What `from_cosmology` does:**\n- When `cosmology` is `None`, it immediately returns `cosmo` unchanged.\n- When `cosmology` is a string, it resolves the string to a class via `_COSMOLOGY_CLASSES`, a module-level dictionary (`dict[str, type[Cosmology]]`) defined in `core.py` that maps qualified class names to actual cosmology classes.\n- It then performs an `isinstance` check of `cosmo` against the resolved class, raising `TypeError` with the message `f\"cosmology {cosmo} is not an {cosmo_cls} instance.\"` if the check fails.\n- The companion `to_cosmology` is even simpler \u2014 it just returns `cosmo` with no validation at all (it ignores `*args`).\n\n**Registration and integration:**\nAll three functions are registered at module level with `convert_registry`, which is a `UnifiedIORegistry` instance (from `astropy.io.registry`) created in `connect.py`. Specifically:\n- `from_cosmology` is registered as a \"reader\" via `convert_registry.register_reader(\"astropy.cosmology\", Cosmology, from_cosmology)`\n- `to_cosmology` is registered as a \"writer\" via `convert_registry.register_writer(...)`\n- `cosmology_identify` is registered as an identifier\n\n**How users access it:**\nThe `Cosmology` class (a frozen `@dataclass` with `ABCMeta` metaclass) exposes `from_format` and `to_format` as `ClassVar` attributes wrapping `CosmologyFromFormat` and `CosmologyToFormat` via `UnifiedReadWriteMethod`. When `Cosmology.from_format(obj)` is called, `CosmologyFromFormat.__call__` delegates to `self.registry.read(self._cls, obj, ...)`, which is `convert_registry.read(...)`. If the calling class is not the base `Cosmology`, it automatically injects `cosmology=self._cls` as a keyword argument, adding a class-level type constraint.\n\n**Auto-identification:**\nThe `cosmology_identify` function only triggers on `origin == \"read\"` and checks whether `args[1]` is a `Cosmology` instance AND the format is either `None` or `\"astropy.cosmology\"`. This enables auto-detection when calling `from_format` without specifying a format, since the identify function matches when format is `None`.\n\n**`_COSMOLOGY_CLASSES` population:**\nThe dictionary is populated automatically in `Cosmology.__init_subclass__` \u2014 every non-abstract subclass calls `cls._register_cls()`, which sets `_COSMOLOGY_CLASSES[cls.__qualname__] = cls` and also registers the class with the YAML serializer.\n\nThe module's `__all__` is an empty list, indicating these functions are not part of the public API.",
    "rubric": [
      "Identifies from_cosmology, to_cosmology, and cosmology_identify as the trio of functions forming the identity conversion, registered under the format name 'astropy.cosmology'",
      "Explains that from_cosmology returns cosmo unchanged when cosmology is None, and validates the type when cosmology is a string (resolved via _COSMOLOGY_CLASSES) or a class, raising TypeError on mismatch",
      "Explains that to_cosmology simply returns cosmo with no validation, ignoring extra args",
      "Describes registration with convert_registry (a UnifiedIORegistry instance): from_cosmology as reader, to_cosmology as writer, cosmology_identify as identifier",
      "Explains that cosmology_identify only triggers on origin=='read' and checks that args[1] is a Cosmology instance and format is None or 'astropy.cosmology'",
      "Describes how CosmologyFromFormat.__call__ delegates to convert_registry.read and auto-injects cosmology=self._cls when called from a subclass (not the base Cosmology class)",
      "Mentions that _COSMOLOGY_CLASSES is populated by Cosmology.__init_subclass__ via _register_cls for non-abstract subclasses, mapping qualname to class"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/cosmology.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/typing.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/cosmology.py] astropy.cosmology._src.io.builtin.cosmology.from_cosmology\nReturn the |Cosmology| unchanged.\n\n    Parameters\n    ----------\n    cosmo : `~astropy.cosmology.Cosmology`\n        The cosmology to return.\n    cosmology : type[`~astropy.cosmology.Cosmology`] | str | None, optional\n        The |Cosmology| class to check against. If not `None`, ``cosmo`` is checked\n        for correctness.\n    **kwargs\n        This argument is required for compatibility with the standard set of\n        keyword arguments in format |Cosmology.from_format|.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n        Just ``cosmo`` passed through.\n\n    Raises\n    ------\n    TypeError\n        If the |Cosmology| object is not an instance of ``cosmo`` (and\n        ``cosmology`` is not `None`).\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Cosmology, Planck18\n    >>> print(Cosmology.from_format(Planck18))\n    FlatLambdaCDM(name=\"Planck18\", H0=67.66 km / (Mpc s), Om0=0.30966,\n                  Tcmb0=2.7255 K, Neff=3.046, m_nu=[0. 0. 0.06] eV, Ob0=0.04897)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about the TypeError message format (`f\"cosmology {cosmo} is not an {cosmo_cls} instance.\"`) cannot be verified from the documentation alone - it may be accurate code-level detail but could also be a hallucination of the exact string format.",
      "The claim that `to_cosmology` 'just returns cosmo with no validation at all' cannot be verified from the provided documentation, which only covers `from_cosmology`.",
      "Specific line numbers (line 467 of core.py, line 156 of connect.py) cannot be verified from the documentation and could be hallucinated.",
      "The documentation says the TypeError is raised 'If the |Cosmology| object is not an instance of ``cosmo``' but the answer says it checks `cosmo` against the resolved class - the doc actually says 'not an instance of ``cosmology``' (the parameter), which is consistent with the answer's description but uses different wording.",
      "The claim about `Cosmology` being a 'frozen @dataclass with ABCMeta metaclass' cannot be verified from this documentation.",
      "The claim about `_COSMOLOGY_CLASSES` being at 'line 467 of core.py' is a very specific implementation detail that could easily be wrong."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: 'hot path' terminology, specific line numbers, and rationale about 'completeness and internal testing' from module docstring. All core behavioral claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_18",
    "repo": "astropy",
    "question": "Why does the cosmology I/O system register an identity converter that just passes a Cosmology through unchanged?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `to_cosmology` and `from_cosmology` functions in `astropy/cosmology/_src/io/builtin/cosmology.py` exist as identity/passthrough converters. Here's how they fit into the system:\n\n1. **Uniform format system**: The cosmology I/O system registers six formats in the `convert_registry` (\"astropy.cosmology\", \"mapping\", \"astropy.model\", \"astropy.row\", \"astropy.table\", \"yaml\"). The identity format ensures every `Cosmology` object can round-trip through the same `to_format`/`from_format` API as other formats.\n\n2. **Enabling auto-identification for `is_equivalent` and `cosmology_equal`**: The `cosmology_identify` function allows the `Cosmology.from_format` auto-detection to recognize that a `Cosmology` instance can be \"converted\" to itself when `format=None` or `format=\"astropy.cosmology\"`. This is crucial for `Cosmology.is_equivalent()`, which calls `cosmology_equal(self, other, format=(None, format), allow_equivalent=True)`. When `other` is already a Cosmology and `format` is `None` or `True`, the `_parse_format` function in `comparison.py` short-circuits (via the `isinstance(cosmo, Cosmology)` check) before even reaching the identity converter, but the identifier registration ensures format auto-detection works correctly in the registry layer.\n\n3. **Supporting the `_comparison_decorator` framework**: In `astropy/cosmology/_src/funcs/comparison.py`, the `_parse_format` ufunc handles `Cosmology` instances specially (returning them immediately), but for non-Cosmology objects it calls `Cosmology.from_format(cosmo, format=format)`. The identity converter ensures this path works even when someone passes a `Cosmology` to `from_format` directly.\n\n4. **Type checking in `from_cosmology`**: While `to_cosmology` is truly trivial (just `return cosmo`), the companion `from_cosmology` has an early return when `cosmology is None` (`return cosmo`) and a type-checking branch that validates the cosmology instance against a specified class using `_COSMOLOGY_CLASSES` lookup, raising a `TypeError` if the types don't match.\n\n5. **The `cosmology_identify` function only triggers on `origin == \"read\"`**: It checks if the second argument (`args[1]`) is a `Cosmology` instance and that `format` is either `None` or `\"astropy.cosmology\"`. This asymmetry (only \"read\", not \"write\") is because `to_format` always receives an explicit format string (required positional parameter), while `from_format` has format as an optional keyword with default `None`, needing auto-detection.",
    "rubric": [
      "Explains that it provides completeness/uniformity in the format system so Cosmology objects can round-trip through the same to_format/from_format API as other formats",
      "Mentions that the identity format enables auto-identification (via cosmology_identify) so that from_format can recognize a Cosmology instance when format is None",
      "Explains how this supports comparison functions (cosmology_equal/is_equivalent) that use from_format to convert inputs, allowing Cosmology instances to pass through gracefully",
      "Notes that from_cosmology includes type-checking logic (validating against a specified cosmology class via _COSMOLOGY_CLASSES) beyond just returning the input",
      "Mentions that cosmology_identify only triggers on origin=='read' because to_format receives an explicit format string while from_format needs auto-detection with format=None"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/cosmology.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/funcs/comparison.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/cosmology.py] astropy.cosmology._src.io.builtin.cosmology.to_cosmology\nReturn the |Cosmology| unchanged.\n\n    Parameters\n    ----------\n    cosmo : `~astropy.cosmology.Cosmology`\n        The cosmology to return.\n    *args : object\n        Not used.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n        Just ``cosmo`` passed through.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Planck18\n    >>> Planck18.to_format(\"astropy.cosmology\") is Planck18\n    True",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer makes many specific claims about line numbers, internal implementation details, and other functions (cosmology_identify, from_cosmology, _parse_format, etc.) that cannot be verified from the provided documentation chunk alone.",
      "The documentation only covers the `to_cosmology` function's docstring, which simply states it returns the Cosmology unchanged. The answer goes far beyond this into implementation details that may or may not be accurate.",
      "The claim about the module docstring stating 'You probably won't need to use these functions as they are present mainly for completeness and testing' cannot be verified from the provided documentation chunk.",
      "The answer's core claim about `to_cosmology` returning the cosmology unchanged is consistent with the documentation's description and example showing `Planck18.to_format('astropy.cosmology') is Planck18` returns True."
    ],
    "strip_verify_leakage": 0.13,
    "strip_verify_summary": "Two claims relied on stripped docstrings (module-level rationale statement and its quotation); remaining claims are verifiable from code structure and logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_20",
    "repo": "astropy",
    "question": "How does the ECSV write function serialize a Cosmology object to an ECSV file, including the intermediate table conversion steps?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `write_ecsv` function in `astropy/cosmology/_src/io/builtin/ecsv.py` implements a 3-step algorithm to serialize a Cosmology to an ECSV file:\n\n1. **Convert Cosmology to Table via `to_table`** (from `table.py`): The function calls `to_table(cosmology, cls=cls, cosmology_in_meta=cosmology_in_meta, rename=rename)`. Inside `to_table`:\n   - First validates that `cls` is a subclass of `Table` (raising `TypeError` if not)\n   - Calls `to_mapping(cosmology)` to get a dict representation containing the cosmology class, name, parameter values, and metadata\n   - Converts the cosmology class from a type to its `__qualname__` string\n   - Pops the `meta` dict from data; if `cosmology_in_meta=True`, moves the 'cosmology' string into the meta dict\n   - Iterates over all data items: for keys in `cosmology.parameters`, calls `convert_parameter_to_column()` which creates a `Column` with shape `(1,) + np.shape(value)`, using the parameter's `__doc__` as the description; for other keys (like 'name' or 'cosmology'), wraps in `Column([v])`\n   - Constructs the table with `cls(data, meta=meta)`\n   - Applies renames by iterating over `tbl.colnames` and calling `tbl.rename_column(name, renames.get(name, name))`\n   - Adds a unique index on the (possibly renamed) 'name' column via `tbl.add_index(renames.get('name', 'name'), unique=True)`\n\n2. **Set format in kwargs**: Sets `kwargs['format'] = 'ascii.ecsv'`\n\n3. **Write table to file**: Calls `table.write(file, overwrite=overwrite, **kwargs)`\n\nThe `to_mapping` function (in `mapping.py`) builds the dict by: setting 'cosmology' to `cosmology.__class__` (or its qualname if `cosmology_as_str=True`), setting 'name' to `cosmology.name`, deep-copying `cosmology.meta`, then calling `m.update(cosmology.parameters)` which uses the `ParametersAttribute` descriptor that returns a `MappingProxyType({n: getattr(instance, n) for n in getattr(instance, self.attr_name)})` \u2014 i.e., a mapping of parameter names to their current values on the instance.\n\nThe file format is identified by the `ecsv_identify` function which simply checks `filepath is not None and filepath.endswith('.ecsv')`.\n\nFor reading (`read_ecsv`), the function uses `u.add_enabled_units(cu)` as a context manager (where `cu` is `astropy.cosmology.units`) during the `QTable.read()` call to enable cosmological units.\n\nThe functions are registered with `readwrite_registry` under the format name `'ascii.ecsv'` for both reading and writing.",
    "rubric": [
      "Explains that write_ecsv first calls to_table() to convert the Cosmology to a Table",
      "Mentions that to_table calls to_mapping() to get a dict representation of the cosmology",
      "Describes that to_mapping builds a dict with 'cosmology' (the class), 'name', parameters (via cosmology.parameters), and deep-copied metadata",
      "Explains that to_table converts the cosmology class from a type to its __qualname__ string",
      "Describes the cosmology_in_meta logic: if True, moves 'cosmology' from data into the meta dict",
      "Explains that parameters are converted to Columns via convert_parameter_to_column (shape (1,)+np.shape(value), description from parameter.__doc__), while other values are wrapped in Column([v])",
      "Mentions that to_table validates cls is a subclass of Table (raises TypeError otherwise)",
      "Describes that column renames are applied by iterating colnames and calling rename_column",
      "Mentions that a unique index is added on the (possibly renamed) 'name' column",
      "Explains that write_ecsv sets kwargs['format'] = 'ascii.ecsv' and calls table.write(file, overwrite=overwrite, **kwargs)",
      "Mentions that read_ecsv uses u.add_enabled_units(cu) context manager to enable cosmological units during QTable.read()",
      "Notes the functions are registered with readwrite_registry under the format name 'ascii.ecsv'"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/ecsv.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/parameter/descriptors.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/ecsv.py] astropy.cosmology._src.io.builtin.ecsv.write_ecsv\nSerialize the cosmology into a ECSV.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology`\n        The cosmology instance to convert to a mapping.\n    file : path-like or file-like\n        Location to save the serialized cosmology.\n\n    overwrite : bool (optional, keyword-only)\n        Whether to overwrite the file, if it exists.\n    cls : type (optional, keyword-only)\n        Astropy :class:`~astropy.table.Table` (sub)class to use when writing. Default is\n        :class:`~astropy.table.QTable`.\n    cosmology_in_meta : bool (optional, keyword-only)\n        Whether to put the cosmology class in the Table metadata (if `True`, default) or\n        as the first column (if `False`).\n    rename : Mapping[str, str] or None (optional keyword-only)\n        A mapping of field names on the `~astropy.cosmology.Cosmology` to column names\n        of the table.\n\n    **kwargs\n        Passed to ``cls.write``\n\n    Raises\n    ------\n    TypeError\n        If kwarg (optional) 'cls' is not a subclass of `astropy.table.Table`\n\n    Examples\n    --------\n    We assume the following setup:\n\n        >>> from pathlib import Path\n        >>> from tempfile import TemporaryDirectory\n        >>> temp_dir = TemporaryDirectory()\n\n    A Cosmology can be written to an ECSV file:\n\n        >>> from astropy.cosmology import Cosmology, Planck18\n        >>> file = Path(temp_dir.name) / \"file.ecsv\"\n        >>> Planck18.write(file)\n\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: name, datatype: string}\n        ...\n        # meta: !!omap\n        # - {Oc0: 0.2607}\n        ...\n        # schema: astropy-2.0\n        name H0 Om0 Tcmb0 Neff m_nu Ob0\n        Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n        <BLANKLINE>\n\n    If a file already exists, attempting to write will raise an error unless\n    ``overwrite=True``.\n\n        >>> Planck18.write(file, overwrite=True)\n\n    By default :class:`~astropy.cosmology.Cosmology` instances are written using\n    `~astropy.table.QTable` as an intermediate representation (for details see\n    |Cosmology.to_format|, with ``format=\"astropy.table\"``). The `~astropy.table.Table`\n    type can be changed using the ``cls`` keyword argument.\n\n        >>> from astropy.table import Table\n        >>> file = Path(temp_dir.name) / \"file2.ecsv\"\n        >>> Planck18.write(file, cls=Table)\n\n    For most use cases, the default ``cls`` of :class:`~astropy.table.QTable` is\n    recommended and will be largely indistinguishable from other table types, as the\n    ECSV format is agnostic to the table type. An example of a difference that might\n    necessitate using a different table type is if a different ECSV schema is desired.\n\n    By default the cosmology class is written to the Table metadata. This can be changed\n    to a column of the table using the ``cosmology_in_meta`` keyword argument.\n\n        >>> file = Path(temp_dir.name) / \"file3.ecsv\"\n        >>> Planck18.write(file, cosmology_in_meta=False)\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: cosmology, datatype: string}\n        # - {name: name, datatype: string}\n        ...\n        # meta: !!omap\n        # - {Oc0: 0.2607}\n        ...\n        # schema: astropy-2.0\n        cosmology name H0 Om0 Tcmb0 Neff m_nu Ob0\n        FlatLambdaCDM Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n        <BLANKLINE>\n\n    Fields of the Cosmology can be renamed to when writing to an ECSV file using the\n    ``rename`` argument.\n\n        >>> file = Path(temp_dir.name) / \"file4.ecsv\"\n        >>> Planck18.write(file, rename={\"H0\": \"Hubble\"})\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: name, datatype: string}\n        ...\n        # meta:\n        ...\n        # schema: astropy-2.0\n        name Hubble Om0 Tcmb0 Neff m_nu Ob0\n        Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n        <BLANKLINE>\n\n    Additional keyword arguments are passed to :attr:`astropy.table.QTable.write`.\n\n    .. testcleanup::\n\n        >>> temp_dir.cleanup()",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer describes internal implementation details (to_mapping, convert_parameter_to_column, ParametersAttribute, MappingProxyType, etc.) that are not present in the documentation. These are plausible code-level extrapolations but cannot be verified from the provided doc.",
      "The answer mentions 'ecsv_identify' function and its implementation, which is not documented here.",
      "The answer describes 'read_ecsv' behavior with 'u.add_enabled_units(cu)' context manager, which is not in the provided documentation.",
      "The answer states functions are registered with 'readwrite_registry' under format name 'ascii.ecsv', which is not explicitly stated in the documentation but is consistent with kwargs['format'] = 'ascii.ecsv' mentioned.",
      "The answer correctly identifies the key parameters and their behaviors (overwrite, cls, cosmology_in_meta, rename) which are supported by the documentation.",
      "The answer correctly notes that cls defaults to QTable and TypeError is raised if cls is not a Table subclass, which matches the documentation.",
      "The answer's claim about the 3-step algorithm (convert to table, set format, write) is plausible but the internal details go well beyond what's documented."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments needed for unverifiable details about specific unit names and registry class type.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_19",
    "repo": "astropy",
    "question": "Where does the data flow when calling `Cosmology.read()` on an ECSV file, and what functions does it pass through to construct the final instance?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `read_ecsv` function in `astropy/cosmology/_src/io/builtin/ecsv.py` is registered with `readwrite_registry` under the format name `\"ascii.ecsv\"` and is invoked through `Cosmology.read()`. The full data flow chain is:\n\n1. `Cosmology.read(file)` \u2192 `CosmologyRead.__call__()` (in `connect.py`) \u2192 `readwrite_registry.read()` \u2192 `read_ecsv()`\n2. `read_ecsv()` sets `kwargs[\"format\"] = \"ascii.ecsv\"` and reads the file using `QTable.read(filename, **kwargs)` inside a `u.add_enabled_units(cu)` context manager (where `cu` is `astropy.cosmology._src.units`).\n3. The resulting QTable is passed to `from_table()` (in `table.py`), which handles row selection via the `index` parameter.\n4. `from_table()` delegates to `from_row()` (in `row.py`) after selecting the appropriate row.\n5. `from_row()` extracts column data and metadata from the row, builds a mapping dict, and delegates to `from_mapping()` (in `mapping.py`).\n6. `from_mapping()` uses `_rename_map()` to apply renames, `_get_cosmology_class()` to resolve the cosmology class from `_COSMOLOGY_CLASSES` dict (using `__qualname__` as key), then uses `inspect.signature()` with `bind_partial()` and `apply_defaults()` to fill in default parameter values and construct the Cosmology instance.\n\nThe `ecsv_identify` function identifies ECSV files by checking if `filepath` is not None and ends with `\".ecsv\"`. The `CosmologyRead.__call__` adds an additional `add_enabled_units(cu)` context around the registry read call and, when called from a subclass (e.g., `FlatLambdaCDM.read`), automatically sets `kwargs[\"cosmology\"]` to that subclass. The `to_table()` function adds a unique index on the (possibly renamed) \"name\" column via `tbl.add_index(renames.get(\"name\", \"name\"), unique=True)`. In `from_table()`, when `index` is a string and the table has no indices, it uses `np.where(table[nc] == index)[0]` to find matching rows, defaulting `nc` to `\"name\"` but checking the inverse of `rename` for an alternative column name.",
    "rubric": [
      "Mentions that `Cosmology.read()` goes through `CosmologyRead.__call__()` which calls `readwrite_registry.read()` dispatching to `read_ecsv()`",
      "Explains that `read_ecsv()` sets format to 'ascii.ecsv' and reads via `QTable.read()` inside a `u.add_enabled_units(cu)` context manager",
      "Mentions the result is passed to `from_table()` which selects the appropriate row using the `index` parameter",
      "Explains that `from_table()` delegates to `from_row()` after row selection",
      "Explains that `from_row()` builds a mapping dict and delegates to `from_mapping()`",
      "Explains that `from_mapping()` resolves the cosmology class (via `_get_cosmology_class` using `_COSMOLOGY_CLASSES`) and uses `inspect.signature`/`bind_partial`/`apply_defaults` to construct the instance",
      "Mentions that `CosmologyRead.__call__` also wraps the registry read in `add_enabled_units(cu)` and sets `kwargs['cosmology']` when called from a subclass",
      "Mentions that `ecsv_identify` checks if filepath ends with '.ecsv'"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/ecsv.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/row.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/connect.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/ecsv.py] astropy.cosmology._src.io.builtin.ecsv.read_ecsv\nRead a `~astropy.cosmology.Cosmology` from an ECSV file.\n\n    Parameters\n    ----------\n    filename : path-like or file-like\n        From where to read the Cosmology.\n    index : int, str, or None, optional\n        Needed to select the row in tables with multiple rows. ``index`` can be an\n        integer for the row number or, if the table is indexed by a column, the value of\n        that column. If the table is not indexed and ``index`` is a string, the \"name\"\n        column is used as the indexing column.\n\n    move_to_meta : bool (optional, keyword-only)\n        Whether to move keyword arguments that are not in the Cosmology class' signature\n        to the Cosmology's metadata. This will only be applied if the Cosmology does NOT\n        have a keyword-only argument (e.g. ``**kwargs``). Arguments moved to the\n        metadata will be merged with existing metadata, preferring specified metadata in\n        the case of a merge conflict (e.g. for ``Cosmology(meta={'key':10}, key=42)``,\n        the ``Cosmology.meta`` will be ``{'key': 10}``).\n\n    cosmology : str or type or None (optional, keyword-only)\n        The cosmology class (or string name thereof) to use when constructing the\n        cosmology instance. The class also provides default parameter values, filling in\n        any non-mandatory arguments missing in 'table'.\n\n    rename : dict or None (optional, keyword-only)\n        A dictionary mapping column names in 'table' to fields of the\n        `~astropy.cosmology.Cosmology` class.\n\n    **kwargs\n        Passed to ``QTable.read``\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n\n    Examples\n    --------\n    We assume the following setup:\n\n        >>> from pathlib import Path\n        >>> from tempfile import TemporaryDirectory\n        >>> temp_dir = TemporaryDirectory()\n\n    To see reading a Cosmology from an ECSV file, we first write a Cosmology to an ECSV\n    file:\n\n        >>> from astropy.cosmology import Cosmology, Planck18\n        >>> file = Path(temp_dir.name) / \"file.ecsv\"\n        >>> Planck18.write(file)\n\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: name, datatype: string}\n        ...\n        # meta: !!omap\n        # - {Oc0: 0.2607}\n        ...\n        # schema: astropy-2.0\n        name H0 Om0 Tcmb0 Neff m_nu Ob0\n        Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n        <BLANKLINE>\n\n    Now we can read the Cosmology from the ECSV file, constructing a new cosmological\n    instance identical to the ``Planck18`` cosmology from which it was generated.\n\n        >>> cosmo = Cosmology.read(file)\n        >>> print(cosmo)\n        FlatLambdaCDM(name=\"Planck18\", H0=67.66 km / (Mpc s), Om0=0.30966,\n                    Tcmb0=2.7255 K, Neff=3.046, m_nu=[0. 0. 0.06] eV, Ob0=0.04897)\n        >>> cosmo == Planck18\n        True\n\n    The ``cosmology`` information (column or metadata) may be omitted if the cosmology\n    class (or its string name) is passed as the ``cosmology`` keyword argument to\n    |Cosmology.read|. Alternatively, specific cosmology classes can be used to parse the\n    data.\n\n        >>> from astropy.cosmology import FlatLambdaCDM\n        >>> print(FlatLambdaCDM.read(file))\n        FlatLambdaCDM(name=\"Planck18\", H0=67.66 km / (Mpc s), Om0=0.30966,\n                      Tcmb0=2.7255 K, Neff=3.046, m_nu=[0. 0. 0.06] eV, Ob0=0.04897)\n\n    When using a specific cosmology class, the class' default parameter values are used\n    to fill in any missing information.\n\n    For files with multiple rows of cosmological parameters, the ``index`` argument is\n    needed to select the correct row. The index can be an integer for the row number or,\n    if the table is indexed by a column, the value of that column. If the table is not\n    indexed and ``index`` is a string, the \"name\" column is used as the indexing column.\n\n    Here is an example where ``index`` is needed and can be either an integer (for the\n    row number) or the name of one of the cosmologies, e.g. 'Planck15'.\n\n        >>> from astropy.cosmology import Planck13, Planck15, Planck18\n        >>> from astropy.table import vstack\n        >>> cts = vstack([c.to_format(\"astropy.table\")\n        ...               for c in (Planck13, Planck15, Planck18)],\n        ...              metadata_conflicts='silent')\n        >>> file = Path(temp_dir.name) / \"file2.ecsv\"\n        >>> cts.write(file)\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: name, datatype: string}\n        ...\n        # meta: !!omap\n        # - {Oc0: 0.2607}\n        ...\n        # schema: astropy-2.0\n        name H0 Om0 Tcmb0 Neff m_nu Ob0\n        Planck13 67.77 0.30712 2.7255 3.046 [0.0,0.0,0.06] 0.048252\n        Planck15 67.74 0.3075 2.7255 3.046 [0.0,0.0,0.06] 0.0486\n        Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n\n        >>> cosmo = Cosmology.read(file, index=\"Planck15\", format=\"ascii.ecsv\")\n        >>> cosmo == Planck15\n        True\n\n    Fields of the table in the file can be renamed to match the\n    `~astropy.cosmology.Cosmology` class' signature using the ``rename`` argument. This\n    is useful when the files's column names do not match the class' parameter names.\n    For this example we need to make a new file with renamed columns:\n\n        >>> file = Path(temp_dir.name) / \"file3.ecsv\"\n        >>> renamed_table = Planck18.to_format(\"astropy.table\", rename={\"H0\": \"Hubble\"})\n        >>> renamed_table.write(file)\n        >>> with open(file) as f: print(f.read())\n        # %ECSV 1.0\n        # ---\n        # datatype:\n        # - {name: name, datatype: string}\n        ...\n        # meta: !!omap\n        # - {Oc0: 0.2607}\n        ...\n        # schema: astropy-2.0\n        name Hubble Om0 Tcmb0 Neff m_nu Ob0\n        Planck18 67.66 0.30966 2.7255 3.046 [0.0,0.0,0.06] 0.04897\n\n    Now we can read the Cosmology from the ECSV file, with the required renaming:\n\n        >>> cosmo = Cosmology.read(file, rename={\"Hubble\": \"H0\"})\n        >>> cosmo == Planck18\n        True\n\n    Additional keyword arguments are passed to ``QTable.read``.\n\n    .. testcleanup::\n\n        >>> temp_dir.cleanup()",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many claims about internal implementation details (e.g., `_rename_map()`, `_get_cosmology_class()`, `_COSMOLOGY_CLASSES`, `inspect.signature()`, `bind_partial()`, `apply_defaults()`, `tbl.add_index()`, `np.where()`) go beyond what the documentation states, but these are plausible code-level details that don't contradict the documentation.",
      "The claim about `ecsv_identify` checking if filepath ends with '.ecsv' is not mentioned in the documentation but is a reasonable implementation detail.",
      "The claim about `CosmologyRead.__call__` adding `add_enabled_units(cu)` context is not in the documentation but is plausible code-level detail.",
      "The description of `from_table()`, `from_row()`, `from_mapping()` delegation chain is not in the documentation but is plausible internal implementation."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the specific unit names ('redshift' and 'littleh') required documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_22",
    "repo": "astropy",
    "question": "Why does the cosmology LaTeX writer use `<<=` for unit handling while the HTML writer strips units with `.value`, and why is `cosmology_in_meta` hardcoded to `False` in both?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `write_latex` function in `astropy/cosmology/_src/io/builtin/latex.py` uses a specific design pattern for unit handling that differs from the comparable `write_html_table` and `write_ecsv` functions.\n\n**Why `write_latex` uses `<<=` instead of stripping units entirely (like HTML):**\n\nIn `write_latex`, the unit conversion uses `table[name] <<= param.unit`, which is the in-place left-shift operator on Quantity columns. This calls `Quantity.__ilshift__`, which converts the column's values to the specified unit and updates the column's unit attribute \u2014 essentially ensuring the column is in the canonical unit.\n\nIn contrast, `write_html_table` uses `table.replace_column(name, (col << param.unit).value, copy=False)`, which converts the column to the correct unit and then extracts just the numeric `.value`, discarding the unit entirely.\n\nThe ECSV writer (`write_ecsv`) does no unit conversion at all \u2014 it directly passes the QTable through.\n\n**Why `cosmology_in_meta=False` is hardcoded:**\n\nBoth `write_latex` and `write_html_table` call `to_table(cosmology, cls=cls, cosmology_in_meta=False)` with `cosmology_in_meta=False` hardcoded. By forcing `cosmology_in_meta=False`, the cosmology class name is placed as a visible table column (first column, 'cosmology') rather than in metadata. This can be verified by examining the `to_table` function in `table.py`, where `cosmology_in_meta=True` causes the cosmology name to go into `meta` and be popped from `data`, while `cosmology_in_meta=False` keeps it in the data columns.\n\n**Why `_FORMAT_TABLE` is a module-level dict:**\n\nThe LaTeX name mappings are stored in a module-level `_FORMAT_TABLE` dictionary, mapping parameter names like 'H0' to LaTeX strings like '$H_0$'. The HTML module has an equivalent `_FORMAT_TABLE` but with MathJax-style formatting (e.g., '$$H_{0}$$').\n\n**Why there is no `read_latex` registered:**\n\nOnly `register_writer` and `register_identifier` are called for `ascii.latex`. No `register_reader` exists. This means latex is write-only. By contrast, the HTML format registers a reader (`read_html_table`) that includes reverse-mapping logic for the `_FORMAT_TABLE` names.\n\n**The `latex_identify` function only checks file extension:**\n\nThe identifier function simply checks if the filepath ends with '.tex'. It returns `True` if `filepath is not None and filepath.endswith(\".tex\")`, which enables format detection for `.tex` files when registered via `register_identifier`.",
    "rubric": [
      "Explains that `<<=` (in-place left-shift / `__ilshift__`) converts the column to the canonical unit while keeping the unit attached to the column",
      "Explains that the HTML writer uses `(col << param.unit).value` to convert to the correct unit and then strips the unit entirely, replacing the column with a unitless version",
      "Notes that the ECSV writer performs no unit conversion at all, passing the QTable through directly",
      "Explains that `cosmology_in_meta=False` forces the cosmology class name into a visible table column rather than into table metadata",
      "References how `to_table` handles `cosmology_in_meta`: when True, the cosmology string goes into `meta` and is popped from `data`; when False, it remains as a data column",
      "Notes that the LaTeX format is write-only (no registered reader), unlike HTML which has both a reader and writer",
      "Mentions the `_FORMAT_TABLE` dict maps parameter names to LaTeX-formatted strings (e.g., 'H0' \u2192 '$H_0$') and the HTML equivalent uses MathJax-style formatting (e.g., '$$H_{0}$$')"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/latex.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/html.py",
      "astropy/cosmology/_src/io/builtin/ecsv.py",
      "astropy/units/quantity.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/latex.py] astropy.cosmology._src.io.builtin.latex.write_latex\nSerialize the |Cosmology| into a LaTeX.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology` subclass instance\n        The cosmology to serialize.\n    file : path-like or file-like\n        Location to save the serialized cosmology.\n\n    overwrite : bool\n        Whether to overwrite the file, if it exists.\n    cls : type, optional keyword-only\n        Astropy :class:`~astropy.table.Table` (sub)class to use when writing. Default is\n        :class:`~astropy.table.QTable`.\n    latex_names : bool, optional keyword-only\n        Whether to use LaTeX names for the parameters. Default is `True`.\n    **kwargs\n        Passed to ``cls.write``\n\n    Raises\n    ------\n    TypeError\n        If kwarg (optional) 'cls' is not a subclass of `astropy.table.Table`\n\n    Examples\n    --------\n    We assume the following setup:\n\n        >>> from pathlib import Path\n        >>> from tempfile import TemporaryDirectory\n        >>> temp_dir = TemporaryDirectory()\n\n    Writing a cosmology to a LaTeX file will produce a table with the cosmology's type,\n    name, and parameters as columns.\n\n        >>> from astropy.cosmology import Planck18\n        >>> file = Path(temp_dir.name) / \"file.tex\"\n\n        >>> Planck18.write(file, format=\"ascii.latex\")\n        >>> with open(file) as f: print(f.read())\n        \\begin{table}\n        \\begin{tabular}{cccccccc}\n        cosmology & name & $H_0$ & $\\Omega_{m,0}$ & $T_{0}$ & $N_{eff}$ & $m_{nu}$ & $\\Omega_{b,0}$ \\\\\n        &  & $\\mathrm{km\\,Mpc^{-1}\\,s^{-1}}$ &  & $\\mathrm{K}$ &  & $\\mathrm{eV}$ &  \\\\\n        FlatLambdaCDM & Planck18 & 67.66 & 0.30966 & 2.7255 & 3.046 & 0.0 .. 0.06 & 0.04897 \\\\\n        \\end{tabular}\n        \\end{table}\n        <BLANKLINE>\n\n    The cosmology's metadata is not included in the table.\n\n    To save the cosmology in an existing file, use ``overwrite=True``; otherwise, an\n    error will be raised.\n\n        >>> Planck18.write(file, format=\"ascii.latex\", overwrite=True)\n\n    To use a different table class as the underlying writer, use the ``cls`` kwarg. For\n    more information on the available table classes, see the documentation on Astropy's\n    table classes and on ``Cosmology.to_format(\"astropy.table\")``.\n\n    By default the parameter names are converted to LaTeX format. To disable this, set\n    ``latex_names=False``.\n\n        >>> file = Path(temp_dir.name) / \"file2.tex\"\n        >>> Planck18.write(file, format=\"ascii.latex\", latex_names=False)\n        >>> with open(file) as f: print(f.read())\n        \\begin{table}\n        \\begin{tabular}{cccccccc}\n        cosmology & name & H0 & Om0 & Tcmb0 & Neff & m_nu & Ob0 \\\\\n        &  & $\\mathrm{km\\,Mpc^{-1}\\,s^{-1}}$ &  & $\\mathrm{K}$ &  & $\\mathrm{eV}$ &  \\\\\n        FlatLambdaCDM & Planck18 & 67.66 & 0.30966 & 2.7255 & 3.046 & 0.0 .. 0.06 & 0.04897 \\\\\n        \\end{tabular}\n        \\end{table}\n        <BLANKLINE>\n\n    .. testcleanup::\n\n        >>> temp_dir.cleanup()",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer discusses internal implementation details (line numbers, `<<=` operator usage, `_FORMAT_TABLE` dict, `cosmology_in_meta=False`, etc.) that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the given docstring alone.",
      "The answer claims `write_html_table` uses `table.replace_column(name, (col << param.unit).value, copy=False)` \u2014 this is not mentioned in the documentation and cannot be verified.",
      "The answer claims ECSV writer does no unit conversion \u2014 not verifiable from the provided documentation.",
      "The answer claims `_FORMAT_TABLE` is at line 70-83 and `html.py` has a TODO comment at line 106 \u2014 these are specific code claims not verifiable from the docstring.",
      "The answer claims `latex_identify` checks for `.tex` extension at lines 198-207 \u2014 plausible but not in the documentation.",
      "The answer claims no `read_latex` is registered \u2014 plausible but not stated in the documentation.",
      "The core documented behavior is accurately reflected: the function serializes cosmology to LaTeX, supports `overwrite`, `cls`, `latex_names` parameters, default `latex_names=True`, default `cls` is QTable, and the output format matches the examples.",
      "The answer correctly reflects that metadata is not included in the table, consistent with the docstring statement 'The cosmology's metadata is not included in the table.'"
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims relied on stripped comments (TODO note, design rationale comment) or documentation-only reasoning about format capabilities; code-verifiable structural claims were preserved.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_21",
    "repo": "astropy",
    "question": "What are the key steps and internal dependencies involved in the Cosmology HTML table write pipeline, and how does it differ from the LaTeX writer?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `write_html_table` function in `astropy/cosmology/_src/io/builtin/html.py` serializes a Cosmology into an HTML table through a multi-step pipeline that involves several internal modules. Here are the key implementation details:\n\n1. **Registration with I/O system**: The function is registered at module level with `readwrite_registry.register_writer(\"ascii.html\", Cosmology, write_html_table)`, alongside a reader (`read_html_table`) and an identifier (`html_identify`). The `readwrite_registry` is a separate `io_registry.UnifiedIORegistry()` instance (defined in `connect.py`), distinct from the `convert_registry` used for format conversions.\n\n2. **Write pipeline (internal steps)**:\n   - First calls `to_table(cosmology, cls=cls, cosmology_in_meta=False)` \u2014 notably always sets `cosmology_in_meta=False`. In `to_table`, when `cosmology_in_meta` is `False`, the cosmology class name remains in the data dict as a column rather than being moved to table metadata.\n   - `to_table` internally calls `to_mapping(cosmology)` (from `mapping.py`) which creates a dict with keys: `cosmology` (the class itself), `name`, all parameter values, and `meta`. Then `to_table` converts the class to its `__qualname__` string, pops `meta` from the data, and converts each parameter value to a `Column` using `convert_parameter_to_column()` from `utils.py`.\n   - `convert_parameter_to_column` (in `utils.py`) creates a `Column` with shape `(1,) + np.shape(value)`, uses `parameter.__doc__` as the description, and infers dtype from the data (`dtype=None`).\n   - After getting the table, `write_html_table` strips units from columns: it iterates over columns, checks if the corresponding `Parameter.unit` is not `None` and not `u.one`, and replaces the column with the unitless value using `(col << param.unit).value`.\n   - Only then, if `latex_names=True`, it renames columns using `_FORMAT_TABLE` lookup.\n\n3. **_FORMAT_TABLE differences between HTML and LaTeX**: The HTML module's `_FORMAT_TABLE` uses `$$...$$` delimiters (e.g., `\"$$H_{0}$$\"`), while the LaTeX module's `_FORMAT_TABLE` uses single `$...$` delimiters (e.g., `\"$H_0$\"`). Both tables map the same 12 parameter names: H0, Om0, Ode0, Tcmb0, Neff, m_nu, Ob0, w0, wa, wz, wp, zp.\n\n4. **Asymmetric defaults for latex_names**: `write_html_table` defaults `latex_names=False`, while `read_html_table` defaults `latex_names=True`. This asymmetry means that by default, writing produces plain parameter names and reading assumes LaTeX names might be present and tries to reverse-map them. In contrast, `write_latex` defaults `latex_names=True`.\n\n5. **html_identify function**: The identifier function only checks `isinstance(filepath, str) and filepath.endswith(\".html\")` \u2014 it does NOT accept `pathlib.Path` objects for auto-identification, only raw strings.\n\n6. **Read path specifics**: `read_html_table` uses `QTable.read(filename, format=\"ascii.html\")` wrapped in `u.add_enabled_units(cu)` to enable cosmology units during reading. When `latex_names=True`, it reverses the `_FORMAT_TABLE` mapping by checking if any latex column names exist in the table and renaming them back. It then delegates to `from_table()` with `rename=None`, which selects a row and delegates to `from_row()`.\n\n7. **ParametersAttribute descriptor**: The `cosmology.parameters` used in `write_html_table` is a `ParametersAttribute` descriptor. When accessed from an instance, its `__get__` returns a `MappingProxyType` of `{n: getattr(instance, n) for n in getattr(instance, self.attr_name)}`. When accessed from the class (i.e., `instance is None`), it returns `getattr(owner, self.attr_name)`, which provides the `Parameter` objects themselves.",
    "rubric": [
      "Mentions registration with readwrite_registry (a UnifiedIORegistry instance separate from convert_registry) under format 'ascii.html'",
      "Explains that write_html_table calls to_table with cosmology_in_meta=False, which keeps the cosmology class name as a column rather than in metadata",
      "Describes that to_table internally calls to_mapping to get a dict with 'cosmology' (the class), 'name', parameters, and 'meta', then converts the class to __qualname__ string",
      "Explains that convert_parameter_to_column creates a Column with shape (1,) + np.shape(value), uses parameter.__doc__ as description, and dtype=None",
      "Describes the unit-stripping step: iterates columns, checks if Parameter.unit is not None and not u.one, then replaces with (col << param.unit).value",
      "Notes the _FORMAT_TABLE difference: HTML uses $$...$$ delimiters while LaTeX uses single $...$ delimiters for the same 12 parameter names",
      "Identifies the asymmetric latex_names defaults: write_html_table defaults to False, read_html_table defaults to True, and write_latex defaults to True",
      "Explains that html_identify only checks isinstance(filepath, str) and filepath.endswith('.html'), not accepting pathlib.Path objects"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/html.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/parameter/descriptors.py",
      "astropy/cosmology/_src/io/builtin/latex.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/html.py] astropy.cosmology._src.io.builtin.html.write_html_table\nSerialize the |Cosmology| into a HTML table.\n\n    Parameters\n    ----------\n    cosmology : |Cosmology| subclass instance\n        The cosmology to serialize.\n    file : path-like or file-like\n        Where to write the html table.\n\n    overwrite : bool, optional keyword-only\n        Whether to overwrite the file, if it exists.\n    cls : |Table| class, optional keyword-only\n        Astropy |Table| (sub)class to use when writing. Default is |QTable| class.\n    latex_names : bool, optional keyword-only\n        Whether to format the parameters (column) names to latex -- e.g. 'H0' to\n        $$H_{0}$$.\n    **kwargs : Any\n        Passed to ``cls.write``.\n\n    Raises\n    ------\n    TypeError\n        If the optional keyword-argument 'cls' is not a subclass of |Table|.\n    ValueError\n        If the keyword argument 'format' is given and is not \"ascii.html\".\n\n    Examples\n    --------\n    We assume the following setup:\n\n        >>> from pathlib import Path\n        >>> from tempfile import TemporaryDirectory\n        >>> temp_dir = TemporaryDirectory()\n\n    Writing a cosmology to a html file will produce a table with the cosmology's type,\n    name, and parameters as columns.\n\n        >>> from astropy.cosmology import Planck18\n        >>> file = Path(temp_dir.name) / \"file.html\"\n        >>> Planck18.write(file, overwrite=True)\n        >>> with open(file) as f: print(f.read())\n        <html>\n        <head>\n        <meta charset=\"utf-8\"/>\n        <meta content=\"text/html;charset=UTF-8\" http-equiv=\"Content-type\"/>\n        </head>\n        <body>\n        <table>\n        <thead>\n        <tr>\n            <th>cosmology</th> <th>name</th> <th>H0</th> <th>Om0</th> <th>Tcmb0</th>\n            <th>Neff</th> <th colspan=\"3\">m_nu</th> <th>Ob0</th>\n        </tr>\n        </thead>\n        <tr>\n            <td>FlatLambdaCDM</td> <td>Planck18</td> <td>67.66</td> <td>0.30966</td>\n            <td>2.7255</td> <td>3.046</td> <td>0.0</td> <td>0.0</td> <td>0.06</td>\n            <td>0.04897</td>\n        </tr>\n        </table>\n        </body>\n        </html>\n        <BLANKLINE>\n        <BLANKLINE>\n\n    The cosmology's metadata is not included in the file.\n\n    To save the cosmology in an existing file, use ``overwrite=True``; otherwise, an\n    error will be raised.\n\n        >>> Planck18.write(file, overwrite=True)\n\n    To use a different table class as the underlying writer, use the ``cls`` kwarg. For\n    more information on the available table classes, see the documentation on Astropy's\n    table classes and on ``Cosmology.to_format(\"astropy.table\")``.\n\n    By default the parameter names are not converted to LaTeX / MathJax format. To\n    enable this, set ``latex_names=True``.\n\n        >>> file = Path(temp_dir.name) / \"file2.html\"\n        >>> Planck18.write(file, latex_names=True)\n        >>> with open(file) as f: print(f.read())\n        <html>\n        ...\n        <thead>\n            <tr>\n            <th>cosmology</th>\n            <th>name</th>\n            <th>$$H_{0}$$</th>\n            <th>$$\\Omega_{m,0}$$</th>\n            <th>$$T_{0}$$</th>\n            <th>$$N_{eff}$$</th>\n            <th colspan=\"3\">$$m_{nu}$$</th>\n            <th>$$\\Omega_{b,0}$$</th>\n            </tr>\n        ...\n\n    .. testcleanup::\n\n        >>> temp_dir.cleanup()\n\n    Notes\n    -----\n    A HTML file containing a Cosmology HTML table should have scripts enabling MathJax.\n\n    .. code-block:: html\n\n        <script type=\"text/javascript\" id=\"MathJax-script\" async\n            src=\"https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml.js\">\n        </script>",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; minor rewrites needed for interpretive characterizations and unverifiable delegation chains.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_24",
    "repo": "astropy",
    "question": "How does the cosmology-to-mapping conversion work end-to-end, from calling `to_format('mapping')` through the registry to building and returning the dict?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `to_mapping` function in `astropy/cosmology/_src/io/builtin/mapping.py` converts a Cosmology instance into a dictionary-like mapping. Here's how it works in detail:\n\n**Registration mechanism:** The function is registered with `convert_registry` (a `UnifiedIORegistry` instance from `astropy/cosmology/_src/io/connect.py`) as a writer for the format name `\"mapping\"`. When a user calls `Planck18.to_format('mapping', ...)`, it goes through `CosmologyToFormat.__call__` which calls `self.registry.write(self._instance, None, *args, format=format, **kwargs)`. The registry's `write` method (in `astropy/io/registry/core.py`) finds the registered writer and calls `writer(data, *args, **kwargs)` \u2014 this means `to_mapping` receives the cosmology as its first argument and `None` as part of `*args`.\n\n**Implementation steps:**\n1. It first validates that `cls` is a subclass of `dict` or `Mapping`, raising `TypeError` otherwise.\n2. It instantiates `cls()` to create an empty mapping `m`.\n3. When `cosmology_as_str=True`, it uses `cosmology.__class__.__qualname__` (not `__name__`) as the string representation.\n4. The metadata is deep-copied with `copy.deepcopy(cosmology.meta)` before any processing.\n5. When `move_from_meta=True`, it pops both 'cosmology' and 'name' keys from meta before updating `m`, then adds parameters on top \u2014 so parameters always override metadata for conflicting keys.\n6. The `cosmology.parameters` property (via the `ParametersAttribute` descriptor in `astropy/cosmology/_src/parameter/descriptors.py`) returns a `MappingProxyType` that lazily constructs a dict of `{param_name: getattr(instance, param_name)}` for all non-derived parameters.\n7. When `rename` is not None, `_rename_map` is called, which validates that rename target values don't collide with existing map keys (raises `ValueError` if they do), then returns a **plain `dict`** regardless of what `cls` was \u2014 meaning the custom mapping type is lost if rename is used.\n\n**Key design details:**\n- `_rename_map` uses a walrus operator with `set(renames.values()).intersection(map)` to detect conflicts.\n- The `_get_cosmology_class` helper resolves string class names via `_COSMOLOGY_CLASSES`, a module-level dict in `core.py` populated automatically by `__init_subclass__` for non-abstract Cosmology subclasses (via `_register_cls` which checks `inspect.isabstract(cls)`).\n- In `from_mapping`, remaining params after matching the constructor signature are handled by checking if the last parameter's `kind == 4` (which is `inspect.Parameter.VAR_KEYWORD`), corresponding to a `**kwargs` argument.\n- The `mapping_identify` function only identifies for `origin == \"read\"` (not write), checking if the second positional arg (`args[1]`) is a `Mapping` instance.\n- The module's `__all__` is an empty list \u2014 nothing is publicly scoped.",
    "rubric": [
      "Explains that `CosmologyToFormat.__call__` calls `self.registry.write(self._instance, None, *args, format=format, **kwargs)`, passing `None` as a positional arg",
      "Mentions that `to_mapping` is registered with `convert_registry` (a `UnifiedIORegistry`) as a writer for format 'mapping'",
      "Describes that `to_mapping` validates `cls` is a subclass of `dict` or `Mapping`, raising `TypeError` otherwise",
      "Notes that `cosmology_as_str=True` uses `__qualname__` (not `__name__`) for the class string representation",
      "Explains that metadata is deep-copied with `copy.deepcopy(cosmology.meta)`",
      "Describes `move_from_meta=True` behavior: pops 'cosmology' and 'name' from meta, updates map with meta, then parameters are added on top (overriding meta for conflicts)",
      "Explains that `cosmology.parameters` (via `ParametersAttribute` descriptor) returns a `MappingProxyType` constructing `{param_name: getattr(instance, param_name)}` for parameters",
      "Notes that `_rename_map` returns a plain `dict` regardless of original `cls`, so the custom mapping type is lost when rename is used",
      "Mentions that `_rename_map` checks for conflicts between rename target values and existing map keys using set intersection (walrus operator)"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/parameter/descriptors.py",
      "astropy/io/registry/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/mapping.py] astropy.cosmology._src.io.builtin.mapping.to_mapping\nReturn the cosmology class, parameters, and metadata as a `dict`.\n\n    Parameters\n    ----------\n    cosmology : :class:`~astropy.cosmology.Cosmology`\n        The cosmology instance to convert to a mapping.\n    *args : object\n        Not used. Needed for compatibility with\n        `~astropy.io.registry.UnifiedReadWriteMethod`\n    cls : type (optional, keyword-only)\n        `dict` or `collections.Mapping` subclass.\n        The mapping type to return. Default is `dict`.\n    cosmology_as_str : bool (optional, keyword-only)\n        Whether the cosmology value is the class (if `False`, default) or\n        the semi-qualified name (if `True`).\n    move_from_meta : bool (optional, keyword-only)\n        Whether to add the Cosmology's metadata as an item to the mapping (if\n        `False`, default) or to merge with the rest of the mapping, preferring\n        the original values (if `True`)\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of field names of the :class:`~astropy.cosmology.Cosmology` to keys in\n        the map.\n\n    Returns\n    -------\n    MutableMapping[str, Any]\n        A mapping of type ``cls``, by default a `dict`.\n        Has key-values for the cosmology parameters and also:\n        - 'cosmology' : the class\n        - 'meta' : the contents of the cosmology's metadata attribute.\n                   If ``move_from_meta`` is `True`, this key is missing and the\n                   contained metadata are added to the main `dict`.\n\n    Examples\n    --------\n    A Cosmology as a mapping will have the cosmology's name and\n    parameters as items, and the metadata as a nested dictionary.\n\n        >>> from astropy.cosmology import Planck18\n        >>> Planck18.to_format('mapping')\n        {'cosmology': <class 'astropy.cosmology...FlatLambdaCDM'>,\n         'name': 'Planck18', 'H0': <Quantity 67.66 km / (Mpc s)>, 'Om0': 0.30966,\n         'Tcmb0': <Quantity 2.7255 K>, 'Neff': 3.046,\n         'm_nu': <Quantity [0.  , 0.  , 0.06] eV>, 'Ob0': 0.04897,\n         'meta': ...\n\n    The dictionary type may be changed with the ``cls`` keyword argument:\n\n        >>> from collections import OrderedDict\n        >>> Planck18.to_format('mapping', cls=OrderedDict)\n        OrderedDict({'cosmology': <class 'astropy.cosmology...FlatLambdaCDM'>, 'name': 'Planck18', 'H0': <Quantity 67.66 km / (Mpc s)>, 'Om0': 0.30966, 'Tcmb0': <Quantity 2.7255 K>, 'Neff': 3.046, 'm_nu': <Quantity [0.  , 0.  , 0.06] eV>, 'Ob0': 0.04897, 'meta': {...}})\n\n    Sometimes it is more useful to have the name of the cosmology class, not\n    the type itself. The keyword argument ``cosmology_as_str`` may be used:\n\n        >>> Planck18.to_format('mapping', cosmology_as_str=True)\n        {'cosmology': 'FlatLambdaCDM', ...\n\n    The metadata is normally included as a nested mapping. To move the metadata\n    into the main mapping, use the keyword argument ``move_from_meta``. This\n    kwarg inverts ``move_to_meta`` in\n    ``Cosmology.to_format(\"mapping\", move_to_meta=...)`` where extra items\n    are moved to the metadata (if the cosmology constructor does not have a\n    variable keyword-only argument -- ``**kwargs``).\n\n        >>> from astropy.cosmology import Planck18\n        >>> Planck18.to_format('mapping', move_from_meta=True)\n        {'cosmology': <class 'astropy.cosmology...FlatLambdaCDM'>,\n         'name': 'Planck18', 'Oc0': 0.2607, 'n': 0.9665, 'sigma8': 0.8102, ...\n\n    Lastly, the keys in the mapping may be renamed with the ``rename`` keyword.\n\n        >>> rename = {'cosmology': 'cosmo_cls', 'name': 'cosmo_name'}\n        >>> Planck18.to_format('mapping', rename=rename)\n        {'cosmo_cls': <class 'astropy.cosmology...FlatLambdaCDM'>,\n         'cosmo_name': 'Planck18', ...",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `cosmology_as_str=True` uses `__qualname__` rather than `__name__` is not verifiable from the documentation. The doc example shows 'FlatLambdaCDM' which would be the same for both `__qualname__` and `__name__` in this case, so this is unverifiable but plausible.",
      "The claim that `_rename_map` returns a plain `dict` regardless of `cls`, losing the custom mapping type, is not supported by the documentation and could be a hallucination about implementation details.",
      "The claim about `_rename_map` using a walrus operator is an implementation detail not verifiable from the documentation.",
      "The claim about `_get_cosmology_class` using `_COSMOLOGY_CLASSES` and `__init_subclass__` is not in the provided documentation.",
      "The claim about `from_mapping` checking parameter `kind == 4` for `VAR_KEYWORD` is not in the provided documentation.",
      "The claim about `mapping_identify` only identifying for `origin == 'read'` is not in the provided documentation.",
      "The claim about `MetaData` descriptor using `OrderedDict` as default factory is not in the provided documentation.",
      "The claim about `__all__` being an empty list is not in the provided documentation.",
      "The description of `move_from_meta=True` says it 'pops both cosmology and name keys from meta before updating m, then adds parameters on top' - this is plausible but not explicitly described in the doc. The doc says metadata are 'merged with the rest of the mapping, preferring the original values'.",
      "The answer says cosmology_as_str default is the class (False) which matches the doc, and move_from_meta default is False which matches the doc - these are correct."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from line number references and an external module (MetaData/OrderedDict) claim not verifiable from provided stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_23",
    "repo": "astropy",
    "question": "Where is `from_mapping` registered and how does a `Cosmology.from_format()` call ultimately reach it, including format auto-identification?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `from_mapping` function is located in `astropy/cosmology/_src/io/builtin/mapping.py` and is registered with `convert_registry` (an instance of `io_registry.UnifiedIORegistry` defined in `astropy/cosmology/_src/io/connect.py`) as a reader for the format name \"mapping\".\n\nWhen invoked through `Cosmology.from_format()`, the call chain is:\n1. `CosmologyFromFormat.__call__` (in `connect.py`) \u2014 which auto-sets `kwargs['cosmology']` to `self._cls` when called from a subclass, wraps the call in `add_enabled_units(cu)` to enable cosmological units, and delegates to `self.registry.read()`.\n2. `UnifiedInputRegistry.read()` (inherited by `UnifiedIORegistry` in `astropy/io/registry/core.py`) \u2014 which handles format auto-identification via identifiers when `format=None`.\n3. The registered reader `from_mapping()` is called with the mapping object and keyword arguments.\n\nThe `mapping_identify` function performs auto-identification. Its second positional parameter is named `format` but actually receives the `path` variable from the registry's `identify_format` call (`identifier(origin, path, fileobj, *args, **kwargs)`). It checks `args[1]` (not `args[0]`) because `args[0]` is `fileobj` (None for non-file objects), and `args[1]` is the actual user-provided object (the mapping).\n\nInternally, `from_mapping` uses two helper functions:\n- `_rename_map`: raises `ValueError` if any rename target value already exists as a key in the map (checks `set(renames.values()).intersection(map)`).\n- `_get_cosmology_class`: resolves the cosmology class from either the `cosmology` argument or from `params['cosmology']`, using the global `_COSMOLOGY_CLASSES` dict (defined in `core.py`) when the value is a string.\n\nThe `_COSMOLOGY_CLASSES` registry is populated by `Cosmology.__init_subclass__` \u2192 `_register_cls()`, which registers every non-abstract Cosmology subclass using its `__qualname__` as the key.\n\nIn `from_mapping`, the check `lastp.kind == 4` tests for `inspect.Parameter.VAR_KEYWORD` to determine if the cosmology class accepts `**kwargs`. The `Cosmology` base class has a `_init_has_kwargs` classproperty that performs the same check using `inspect.Parameter.VAR_KEYWORD`.\n\nThe `to_mapping` function uses `cosmology.__class__.__qualname__` (not `__name__`) when `cosmology_as_str=True`, which is consistent with how `_COSMOLOGY_CLASSES` stores classes by their qualified name.",
    "rubric": [
      "States that `from_mapping` is registered with `convert_registry` (a `UnifiedIORegistry` instance) as a reader for format name 'mapping'",
      "Identifies that `convert_registry` is defined in the connect module (astropy/cosmology/_src/io/connect.py)",
      "Explains that `CosmologyFromFormat.__call__` auto-sets `kwargs['cosmology']` to `self._cls` for subclasses and wraps the call in `add_enabled_units(cu)`",
      "Explains that `CosmologyFromFormat.__call__` delegates to `self.registry.read()` (the UnifiedIORegistry's read method)",
      "Describes that `mapping_identify` handles auto-identification and checks `args[1]` (not `args[0]`) because args[0] is fileobj and args[1] is the user-provided mapping object",
      "Explains how `_get_cosmology_class` resolves the cosmology class from either the `cosmology` argument or `params['cosmology']`, using `_COSMOLOGY_CLASSES` dict when the value is a string",
      "Notes that `_COSMOLOGY_CLASSES` is populated by `Cosmology.__init_subclass__` \u2192 `_register_cls()` using `__qualname__` as the key",
      "Mentions the `lastp.kind == 4` check for `VAR_KEYWORD` to detect if the cosmology class accepts `**kwargs`"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py",
      "astropy/io/registry/core.py",
      "astropy/io/registry/base.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/mapping.py] astropy.cosmology._src.io.builtin.mapping.from_mapping\nLoad `~astropy.cosmology.Cosmology` from mapping object.\n\n    Parameters\n    ----------\n    mapping : Mapping\n        Arguments into the class -- like \"name\" or \"meta\". If 'cosmology' is None, must\n        have field \"cosmology\" which can be either the string name of the cosmology\n        class (e.g. \"FlatLambdaCDM\") or the class itself.\n\n    move_to_meta : bool (optional, keyword-only)\n        Whether to move keyword arguments that are not in the Cosmology class' signature\n        to the Cosmology's metadata. This will only be applied if the Cosmology does NOT\n        have a keyword-only argument (e.g. ``**kwargs``). Arguments moved to the\n        metadata will be merged with existing metadata, preferring specified metadata in\n        the case of a merge conflict (e.g. for ``Cosmology(meta={'key':10}, key=42)``,\n        the ``Cosmology.meta`` will be ``{'key': 10}``).\n\n    cosmology : str, |Cosmology| class, or None (optional, keyword-only)\n        The cosmology class (or string name thereof) to use when constructing the\n        cosmology instance. The class also provides default parameter values, filling in\n        any non-mandatory arguments missing in 'map'.\n\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of keys in ``map`` to fields of the `~astropy.cosmology.Cosmology`.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n\n    Examples\n    --------\n    To see loading a `~astropy.cosmology.Cosmology` from a dictionary with\n    ``from_mapping``, we will first make a mapping using\n    :meth:`~astropy.cosmology.Cosmology.to_format`.\n\n        >>> from astropy.cosmology import Cosmology, Planck18\n        >>> cm = Planck18.to_format('mapping')\n        >>> cm\n        {'cosmology': <class 'astropy.cosmology...FlatLambdaCDM'>,\n         'name': 'Planck18', 'H0': <Quantity 67.66 km / (Mpc s)>, 'Om0': 0.30966,\n         'Tcmb0': <Quantity 2.7255 K>, 'Neff': 3.046,\n         'm_nu': <Quantity [0. , 0. , 0.06] eV>, 'Ob0': 0.04897,\n         'meta': ...\n\n    Now this dict can be used to load a new cosmological instance identical to the\n    |Planck18| cosmology from which it was generated.\n\n        >>> cosmo = Cosmology.from_format(cm, format=\"mapping\")\n        >>> cosmo\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    The ``cosmology`` field can be omitted if the cosmology class (or its string name)\n    is passed as the ``cosmology`` keyword argument to |Cosmology.from_format|.\n\n        >>> del cm[\"cosmology\"]  # remove cosmology class\n        >>> Cosmology.from_format(cm, cosmology=\"FlatLambdaCDM\")\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    Alternatively, specific cosmology classes can be used to parse the data.\n\n        >>> from astropy.cosmology import FlatLambdaCDM\n        >>> FlatLambdaCDM.from_format(cm)\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    When using a specific cosmology class, the class' default parameter values are used\n    to fill in any missing information.\n\n        >>> del cm[\"Tcmb0\"]  # show FlatLambdaCDM provides default\n        >>> FlatLambdaCDM.from_format(cm)\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 0. K>, Neff=3.046, m_nu=None, Ob0=0.04897)\n\n    The ``move_to_meta`` keyword argument can be used to move fields that are not in the\n    Cosmology constructor to the Cosmology's metadata. This is useful when the\n    dictionary contains extra information that is not part of the Cosmology.\n\n        >>> cm2 = cm | {\"extra\": 42, \"cosmology\": \"FlatLambdaCDM\"}\n        >>> cosmo = Cosmology.from_format(cm2, move_to_meta=True)\n        >>> cosmo.meta\n        {'extra': 42, ...}\n\n    The ``rename`` keyword argument can be used to rename keys in the mapping to fields\n    of the |Cosmology|. This is crucial when the mapping has keys that are not valid\n    arguments to the |Cosmology| constructor.\n\n        >>> cm3 = dict(cm)  # copy\n        >>> cm3[\"cosmo_cls\"] = \"FlatLambdaCDM\"\n        >>> cm3[\"cosmo_name\"] = cm3.pop(\"name\")\n\n        >>> rename = {'cosmo_cls': 'cosmology', 'cosmo_name': 'name'}\n        >>> Cosmology.from_format(cm3, rename=rename)\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 0. K>, Neff=3.046, m_nu=None, Ob0=0.04897)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, internal function names, registry mechanisms) that cannot be verified from the provided documentation alone. These are plausible but unverifiable from the given doc chunk.",
      "The claim about `mapping_identify` checking `args[1]` vs `args[0]` cannot be verified from the documentation provided.",
      "The claim about `_rename_map` raising ValueError if rename target values already exist as keys cannot be verified from the documentation.",
      "The claim about `lastp.kind == 4` at line 287 cannot be verified from the documentation.",
      "The claim about `__qualname__` vs `__name__` usage cannot be verified from the documentation.",
      "The documentation states that in a merge conflict 'preferring specified metadata' (e.g., meta={'key':10} wins over key=42), but the answer does not discuss this behavior, which is fine since it's not contradicted."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only specific line numbers and a reference to unseen files needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_25",
    "repo": "astropy",
    "question": "What is the architecture of the Cosmology-to-Model conversion system, including how models are dynamically created and how the reverse conversion reconstructs a Cosmology?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `from_model` function in `astropy/cosmology/_src/io/builtin/model.py` is part of a bidirectional conversion system between `Cosmology` objects and `astropy.modeling.Model` objects, registered under the format name `\"astropy.model\"` in a `convert_registry` (an instance of `io_registry.UnifiedIORegistry`).\n\n**Architecture overview:**\n\n1. **`_CosmologyModel` class** (line 51): Inherits from both `FittableModel` (from `astropy.modeling`) and `Generic[_CosmoT]`. It has `n_outputs = 1` always set by `to_model`. It uses `@abc.abstractmethod` for `_cosmology_class` and `_method_name`, which are actually set as concrete class attributes in dynamically-created subclasses.\n\n2. **`from_model` function** (line 167): Converts back from a `_CosmologyModel` instance to a `Cosmology` instance. It:\n   - Calls `model.cosmology` property, which constructs a Cosmology instance using the model's parameter values\n   - Deep-copies `model.meta`\n   - Iterates over `model.param_names` and for each modeling Parameter, collects all non-private, non-callable attributes into the metadata dict\n   - Uses `dataclasses.replace(cosmo, meta=meta)` to create a new Cosmology with updated metadata (since Cosmology is a frozen dataclass)\n\n3. **`to_model` function** (line 201): Converts a Cosmology to a Model by dynamically creating a new class using `type()`. The class name follows the pattern `{QualName}Cosmology{MethodTitleCase}Model` (e.g., `FlatLambdaCDMCosmologyLookbackTimeModel`). Parameters with `None` values are filtered out. The `n_inputs` is determined by counting positional parameters (kind 0=POSITIONAL_ONLY or 1=POSITIONAL_OR_KEYWORD) from the method signature.\n\n4. **`convert_parameter_to_model_parameter`** (in `utils.py`, line 35): Converts a cosmology `Parameter` to a `ModelParameter` using the parameter's `__doc__` as description, its value as `default`, and extracts `unit` from the value. It also looks for metadata keys: `'getter'`, `'setter'`, `'fixed'`, `'tied'`, `'min'`, `'max'`, `'bounds'`, `'prior'`, `'posterior'`.\n\n5. **`model_identify`** (line 278): The identifier function only returns `True` for `origin == \"read\"` (not \"write\"), checking if `args[1]` is a `Model` instance and format is `None` or `\"astropy.model\"`.\n\n6. **Registration** (lines 297-299): Three registrations are made on `convert_registry`: reader (`from_model`), writer (`to_model`), and identifier (`model_identify`).\n\n7. **`_CosmologyModel.cosmology` property** (line 96): Reconstructs the cosmology by iterating over `param_names`, using walrus operator (`:=`) to get each parameter and checking if it has a unit - if so uses `.quantity`, otherwise `.value`.\n\n8. **`_CosmologyModel.evaluate`** has special handling for `m_nu` parameter: when unvectorizing parameters, it skips `m_nu` if it's 1-dimensional.\n\n9. **`_cosmology_class_sig`** is a `@classproperty(lazy=True)` which caches the `inspect.signature` of the cosmology class after first access.\n\n10. The `Cosmology` base class itself is a frozen dataclass (`dataclass(frozen=True, repr=True, eq=False, init=True)`) with a precomputed `__signature__`.",
    "rubric": [
      "Mentions _CosmologyModel as a base class inheriting from FittableModel (and Generic[_CosmoT]) with abstract _cosmology_class and _method_name attributes",
      "Explains that to_model dynamically creates a subclass of _CosmologyModel using type() with a naming pattern like {QualName}Cosmology{MethodTitleCase}Model",
      "Describes how n_inputs is determined by counting positional (kind 0 or 1) parameters from the method signature",
      "Explains from_model reconstructs the Cosmology via model.cosmology property, collects parameter metadata, and uses dataclasses.replace to attach metadata",
      "Mentions the cosmology property uses walrus operator to check if parameters have units (uses .quantity) or not (uses .value)",
      "Describes convert_parameter_to_model_parameter extracting description from __doc__, default from value, unit from value, and optional metadata keys (getter, setter, fixed, tied, min, max, bounds, prior, posterior)",
      "Notes the registration with convert_registry (UnifiedIORegistry) under format name 'astropy.model' for reader, writer, and identifier",
      "Mentions model_identify only returns True for origin=='read' and checks if args[1] is a Model instance",
      "Notes that parameters with None values are filtered out during to_model conversion",
      "Mentions the evaluate method's special handling of m_nu (skips unvectorization if 1-dimensional)"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/model.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/model.py] astropy.cosmology._src.io.builtin.model.from_model\nLoad |Cosmology| from `~astropy.modeling.Model` object.\n\n    Parameters\n    ----------\n    model : `_CosmologyModel` subclass instance\n        See ``Cosmology.to_format.help(\"astropy.model\") for details.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Cosmology, Planck18\n    >>> model = Planck18.to_format(\"astropy.model\", method=\"lookback_time\")\n    >>> print(Cosmology.from_format(model))\n    FlatLambdaCDM(name=\"Planck18\", H0=67.66 km / (Mpc s), Om0=0.30966,\n                  Tcmb0=2.7255 K, Neff=3.046, m_nu=[0. 0. 0.06] eV, Ob0=0.04897)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides the docstring for `from_model`, showing its parameters, return type, and an example. The generated answer contains extensive implementation details (line numbers, class hierarchies, internal method behaviors, walrus operators, lazy classproperties, etc.) that cannot be verified from the provided documentation alone.",
      "Claim about `from_model` using `dataclasses.replace(cosmo, meta=meta)` cannot be verified from the documentation - this is an implementation detail not mentioned in the docstring.",
      "The claim that `from_model` calls `model.cosmology` property and deep-copies `model.meta` is not verifiable from the provided documentation.",
      "The detailed description of `to_model`, `_CosmologyModel`, `convert_parameter_to_model_parameter`, `model_identify`, and other functions/classes goes far beyond what the documentation excerpt covers.",
      "The claim about Cosmology being a 'frozen dataclass' with `dataclass(frozen=True, repr=True, eq=False, init=True)` and '~20x speedup' cannot be verified from this documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rationale descriptions (neutrino masses being array-valued, ~20x speedup) come from documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_26",
    "repo": "astropy",
    "question": "Why does `to_model` use `type()` for dynamic class creation instead of a static class hierarchy when converting a Cosmology to a Model?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `to_model` function in `astropy/cosmology/_src/io/builtin/model.py` uses dynamic class creation via `type()` rather than a static class hierarchy for several interconnected design reasons:\n\n1. **Dynamic class creation to match arbitrary cosmology+method combinations**: The function uses `type(clsname, (_CosmologyModel,), {**attrs, **params})` to create a new class at runtime because each cosmology class has different parameters and each method has different numbers of inputs. The class name is dynamically constructed by combining the cosmology's qualified name, the string \"Cosmology\", the title-cased method name, and \"Model\". This allows creating unique model classes like `FlatLambdaCDMCosmologyLookbackTimeModel` on the fly.\n\n2. **Filtering out None parameters**: Parameters with `None` values are excluded from the model (`if v is not None`) when building the params dict. The `convert_parameter_to_model_parameter` function passes the value as the `default` argument to `ModelParameter`, which expects actual numeric defaults.\n\n3. **n_outputs is hardcoded to 1** in the attrs dictionary.\n\n4. **The evaluate method re-creates a Cosmology instance on every call** (`cosmo = self._cosmology_class(**ba.arguments)` inside evaluate) rather than caching it. The unvectorization logic takes `[0]` from array-shaped arguments to convert them back to scalars. The `m_nu` parameter gets special-cased (`if k == 'm_nu' and len(ba.arguments[k].shape) == 1: continue`) to skip this unvectorization when it has only one dimension, as it is inherently array-shaped.\n\n5. **Inherits from `FittableModel`** rather than plain `Model`, as seen in `class _CosmologyModel(FittableModel, Generic[_CosmoT])`, which enables the cosmology model to work with astropy's fitting framework.\n\n6. **The `_cosmology_class_sig` uses `@classproperty(lazy=True)`** to cache the result of `inspect.signature(cls._cosmology_class)`, since the cosmology class doesn't change after the model class is created.\n\n7. **The `model_identify` function only identifies on \"read\" origin** (`if origin == \"read\"`) and checks `isinstance(args[1], Model)` \u2014 it does not identify on \"write\".\n\n8. **The `from_model` function uses `replace(cosmo, meta=meta)`** (imported from `dataclasses`) to reconstruct the cosmology with enriched metadata. The metadata includes Model Parameter attributes (the `convert_parameter_to_model_parameter` in utils.py shows attributes like bounds, fixed, tied, etc. are tracked), collecting all non-private, non-callable attributes of each parameter.",
    "rubric": [
      "Explains that each cosmology class has different parameters and each method has different numbers of inputs, making static definitions impractical",
      "Mentions the class name is dynamically constructed by combining cosmology qualname, 'Cosmology', title-cased method name, and 'Model'",
      "Notes that parameters with None values are filtered out (if v is not None) because they cannot serve as valid defaults for ModelParameter",
      "Explains that n_inputs is dynamically determined by inspecting the method signature for positional parameters",
      "Mentions the created class inherits from _CosmologyModel which extends FittableModel, enabling use with astropy's fitting framework",
      "Notes that evaluate re-creates a Cosmology instance on each call and includes special-casing for m_nu's array shape"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/model.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/modeling/core.py",
      "astropy/cosmology/_src/io/connect.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/model.py] astropy.cosmology._src.io.builtin.model.to_model\nConvert a `~astropy.cosmology.Cosmology` to a `~astropy.modeling.Model`.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology` subclass instance\n    method : str, keyword-only\n        The name of the method on the ``cosmology``.\n\n    Returns\n    -------\n    `_CosmologyModel` subclass instance\n        The Model wraps the |Cosmology| method, converting each non-`None`\n        :class:`~astropy.cosmology.Parameter` to a\n        :class:`astropy.modeling.Model` :class:`~astropy.modeling.Parameter`\n        and the method to the model's ``__call__ / evaluate``.\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Planck18\n    >>> model = Planck18.to_format(\"astropy.model\", method=\"lookback_time\")\n    >>> model\n    <FlatLambdaCDMCosmologyLookbackTimeModel(H0=67.66 km / (Mpc s), Om0=0.30966,\n        Tcmb0=2.7255 K, Neff=3.046, m_nu=[0.  , 0.  , 0.06] eV, Ob0=0.04897,\n        name='Planck18')>",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims relied on stripped comments/docstrings for design rationale; code-visible behavior was preserved while documentation-only explanations were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_27",
    "repo": "astropy",
    "question": "Where does data flow when a Cosmology is deserialized from an astropy Row, and how does the table-based deserialization connect to it?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `from_row` function in `astropy/cosmology/_src/io/builtin/row.py` follows this data flow:\n\n1. **Entry point**: `from_row` is registered with `convert_registry` (an instance of `io_registry.UnifiedIORegistry`) as the reader for format \"astropy.row\" via `convert_registry.register_reader(\"astropy.row\", Cosmology, from_row)`.\n\n2. **Invocation path**: When a user calls `Cosmology.from_format(row, format=\"astropy.row\")`, it goes through `CosmologyFromFormat.__call__` (in `connect.py`), which wraps the call with `add_enabled_units(cu)` and then calls `self.registry.read(self._cls, obj, ...)`. If called from a subclass (e.g., `FlatLambdaCDM.from_format`), the `cosmology` kwarg is automatically set to the calling class.\n\n3. **Auto-detection**: `row_identify` checks `isinstance(args[1], Row)` and `format in (None, \"astropy.row\")` to auto-detect the format when `format=None`.\n\n4. **Inside `from_row`**: \n   - The `rename` parameter is inverted (`{v: k for k, v in rename.items()}`) to find the original column names for special keys (\"name\", \"meta\", \"cosmology\").\n   - Metadata is built as a `defaultdict(dict)` initialized with `copy.deepcopy(row.meta)`, then columnar metadata is gathered by iterating `row._table.itercols()` and checking each column's `col.info.meta`.\n   - The cosmology class info is retrieved via `mapping.setdefault(kcosmo, meta.pop(kcosmo, None))` \u2014 meaning it can come from either a column or from table-level metadata.\n   - Finally delegates to `from_mapping(mapping, move_to_meta=move_to_meta, cosmology=cosmology, rename=rename)`.\n\n5. **Inside `from_mapping`** (in `mapping.py`):\n   - Applies `_rename_map` which raises `ValueError` if rename values overlap with existing keys.\n   - Calls `_get_cosmology_class` which resolves the class from `_COSMOLOGY_CLASSES` dict (keyed by `__qualname__`) if it's a string.\n   - Uses `inspect.signature(cosmology)` then `sig.bind_partial()` + `ba.apply_defaults()` to create a blank argument binding with default values.\n   - Transfers matching params from the mapping into `ba.arguments`.\n   - Handles remaining params: if the last parameter's kind is 4 (VAR_KEYWORD / `**kwargs`), passes extra params there; if `move_to_meta=True`, merges remaining params into meta (with existing meta taking precedence via `{**params, **meta}`); otherwise raises `TypeError`.\n   - Returns `cosmology(*ba.args, **ba.kwargs)`.\n\n6. **`to_row` output path**: `to_row` delegates to `to_table` (from `table.py`) which first calls `to_mapping(cosmology)` to get a dict, converts the cosmology class to its `__qualname__` string, converts parameters to `Column` objects via `convert_parameter_to_column`, creates the table, applies column renames, and adds a unique index on the \"name\" column. Then `to_row` returns `table[0]`.\n\n7. **`from_table` \u2192 `from_row` chain**: `from_table` (in `table.py`) eventually calls `from_row` after selecting the appropriate row from the table, making `from_row` the common leaf function for both table and row deserialization.\n\n8. **Default for `cosmology_in_meta`**: In `to_row`, `cosmology_in_meta` defaults to `False` (cosmology as a column), while in `to_table`, `cosmology_in_meta` defaults to `True` (cosmology in metadata).",
    "rubric": [
      "Explains that `from_row` is registered with `convert_registry` as the reader for format 'astropy.row' and is invoked via `CosmologyFromFormat.__call__` which wraps the call with `add_enabled_units` and delegates to `self.registry.read`",
      "Describes that `row_identify` auto-detects the format by checking `isinstance(args[1], Row)` and `format in (None, 'astropy.row')`",
      "Explains that `from_row` inverts the `rename` dict to find original column names for special keys ('name', 'meta', 'cosmology'), builds metadata as a `defaultdict(dict)` from `row.meta` plus columnar metadata from `row._table.itercols()`, and uses `mapping.setdefault(kcosmo, meta.pop(kcosmo, None))` to resolve the cosmology class from either a column or table-level metadata",
      "Explains that `from_row` delegates to `from_mapping`, which applies `_rename_map`, resolves the cosmology class via `_get_cosmology_class` (using `_COSMOLOGY_CLASSES` dict if string), uses `inspect.signature` with `bind_partial` and `apply_defaults` to build arguments, and handles extra params via VAR_KEYWORD, `move_to_meta`, or raises TypeError",
      "Notes that `from_table` selects a row from the table (handling integer/string index) and then calls `from_row`, making `from_row` the common leaf function for both table and row deserialization",
      "Notes that `to_row` delegates to `to_table` (which calls `to_mapping` internally) and returns `table[0]`, and that `cosmology_in_meta` defaults to `False` in `to_row` vs `True` in `to_table`",
      "Mentions that when `from_format` is called from a subclass (not base `Cosmology`), `CosmologyFromFormat.__call__` automatically sets the `cosmology` kwarg to the calling class"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/row.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/row.py] astropy.cosmology._src.io.builtin.row.from_row\nInstantiate a `~astropy.cosmology.Cosmology` from a `~astropy.table.Row`.\n\n    Parameters\n    ----------\n    row : `~astropy.table.Row`\n        The object containing the Cosmology information.\n    move_to_meta : bool (optional, keyword-only)\n        Whether to move keyword arguments that are not in the Cosmology class'\n        signature to the Cosmology's metadata. This will only be applied if the\n        Cosmology does NOT have a keyword-only argument (e.g. ``**kwargs``).\n        Arguments moved to the metadata will be merged with existing metadata,\n        preferring specified metadata in the case of a merge conflict\n        (e.g. for ``Cosmology(meta={'key':10}, key=42)``, the ``Cosmology.meta``\n        will be ``{'key': 10}``).\n\n    cosmology : str, type, or None (optional, keyword-only)\n        The cosmology class (or string name thereof) to use when constructing\n        the cosmology instance. The class also provides default parameter values,\n        filling in any non-mandatory arguments missing in 'table'.\n\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of column names in the row to field names of the |Cosmology|.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology`\n\n    Examples\n    --------\n    To see loading a `~astropy.cosmology.Cosmology` from a Row with\n    ``from_row``, we will first make a `~astropy.table.Row` using\n    :func:`~astropy.cosmology.Cosmology.to_format`.\n\n        >>> from astropy.cosmology import Cosmology, Planck18\n        >>> cr = Planck18.to_format(\"astropy.row\")\n        >>> cr\n        <Row index=0>\n          cosmology     name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                               km / (Mpc s)            K                 eV\n            str13       str8     float64    float64 float64 float64  float64[3] float64\n        ------------- -------- ------------ ------- ------- ------- ----------- -------\n        FlatLambdaCDM Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    Now this row can be used to load a new cosmological instance identical\n    to the ``Planck18`` cosmology from which it was generated.\n\n        >>> cosmo = Cosmology.from_format(cr, format=\"astropy.row\")\n        >>> cosmo\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    The ``cosmology`` information (column or metadata) may be omitted if the cosmology\n    class (or its string name) is passed as the ``cosmology`` keyword argument to\n    |Cosmology.from_format|.\n\n        >>> del cr.columns[\"cosmology\"]  # remove cosmology from metadata\n        >>> Cosmology.from_format(cr, cosmology=\"FlatLambdaCDM\")\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    Alternatively, specific cosmology classes can be used to parse the data.\n\n        >>> from astropy.cosmology import FlatLambdaCDM\n        >>> FlatLambdaCDM.from_format(cr)\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    When using a specific cosmology class, the class' default parameter values are used\n    to fill in any missing information.\n\n        >>> del cr.columns[\"Tcmb0\"]  # show FlatLambdaCDM provides default\n        >>> FlatLambdaCDM.from_format(cr)\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 0. K>, Neff=3.046, m_nu=None, Ob0=0.04897)\n\n    If a `~astropy.table.Row` object has columns that do not match the fields of the\n    `~astropy.cosmology.Cosmology` class, they can be mapped using the ``rename``\n    keyword argument.\n\n        >>> renamed = Planck18.to_format(\"astropy.row\", rename={\"H0\": \"Hubble\"})\n        >>> renamed\n        <Row index=0>\n          cosmology     name      Hubble      Om0    Tcmb0    Neff      m_nu      Ob0\n                               km / (Mpc s)            K                 eV\n            str13       str8     float64    float64 float64 float64  float64[3] float64\n        ------------- -------- ------------ ------- ------- ------- ----------- -------\n        FlatLambdaCDM Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n        >>> cosmo = Cosmology.from_format(renamed, format=\"astropy.row\",\n        ...                               rename={\"Hubble\": \"H0\"})\n        >>> cosmo == Planck18\n        True",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many implementation details (line numbers, internal function names, registry mechanisms, etc.) that go far beyond what the provided documentation covers. These cannot be verified from the documentation alone, but they are plausible code-level details.",
      "Claim about `from_table` calling `from_row` as a 'leaf function' is not verifiable from the documentation, but is plausible.",
      "Claim about `cosmology_in_meta` defaults (`False` for `to_row`, `True` for `to_table`) cannot be verified from the provided documentation.",
      "The merge conflict example in the answer's point 5 about `{**params, **meta}` giving existing meta precedence is consistent with the documentation's description that 'preferring specified metadata in the case of a merge conflict'.",
      "The documentation states that for `Cosmology(meta={'key':10}, key=42)`, the meta will be `{'key': 10}` \u2014 the answer's claim about `{**params, **meta}` is consistent with this behavior (meta takes precedence over params)."
    ],
    "strip_verify_leakage": 0.06,
    "strip_verify_summary": "Minor leakage from line number references and one module-content assumption; all substantive claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_29",
    "repo": "astropy",
    "question": "What is the full deserialization call chain when converting an astropy Table into a Cosmology instance, and how does each layer handle renamed fields and missing parameters?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `from_table` function in `astropy/cosmology/_src/io/builtin/table.py` is a deserialization function that converts an `astropy.table.Table` into a `Cosmology` instance. It is part of a layered I/O architecture where `from_table` delegates to `from_row` (from `row.py`), which in turn delegates to `from_mapping` (from `mapping.py`), forming a call chain: `from_table` \u2192 `from_row` \u2192 `from_mapping`.\n\n**Key implementation details:**\n\n1. **Row selection logic**: When `index` is a string and the table has no indices (`not table.indices`), the function defaults to using a column named \"name\" to locate the row. If a `rename` mapping is provided, it iterates through the rename dict to find which key maps to \"name\" and uses that key as the column name (`nc`) instead. This uses `np.where(table[nc] == index)[0]` for matching.\n\n2. **Delegation chain**: `from_table` extracts a single row (via `table[index]`) and passes it to `from_row`, which converts the row to a dict-like mapping. `from_row` creates an inverse rename dict (`inv_rename = {v: k for k, v in rename.items()}`) to look up special keys like \"name\", \"meta\", and \"cosmology\" under their potentially-renamed column names. It also collects column-level metadata from `row._table.itercols()` via `col.info.meta`.\n\n3. **Default value filling in `from_mapping`**: The function uses `inspect.signature(cosmology)` on the resolved cosmology class, creates a partial binding with `sig.bind_partial()`, calls `ba.apply_defaults()` to fill in default parameter values, then overlays any values found in the input mapping. This is how missing parameters get their defaults from the cosmology class.\n\n4. **Cosmology class resolution**: `_get_cosmology_class` resolves the class from a global registry `_COSMOLOGY_CLASSES` (a `dict[str, type[Cosmology]]` defined in `core.py`). Classes are registered automatically in `__init_subclass__` via `_register_cls()` using `cls.__qualname__` as the key.\n\n5. **Extra parameter handling**: In `from_mapping`, if the last parameter in the cosmology class signature has `kind == 4` (VAR_KEYWORD, i.e., `**kwargs`), remaining params go there. If `move_to_meta=True`, extra params are merged into metadata with `{**params, **meta}` (preferring existing meta values). Otherwise, a `TypeError` is raised for unused parameters.\n\n6. **Registration**: All three functions (`from_table`, `to_table`, `table_identify`) are registered with `convert_registry` (an instance of `io_registry.UnifiedIORegistry`) under the format name \"astropy.table\". The `table_identify` function only returns `True` when `origin == \"read\"` and the second positional arg is a `Table` instance.\n\n7. **`to_table` output structure**: `to_table` uses `to_mapping(cosmology)` to get the data, converts the cosmology class to its `__qualname__` string, uses `convert_parameter_to_column` for parameters (which creates a `Column` with `shape=(1,) + np.shape(value)`, `description=parameter.__doc__`, and `dtype=None`), and adds a unique index on the \"name\" column via `tbl.add_index(renames.get(\"name\", \"name\"), unique=True)`.\n\n8. **Unit context**: When called via `Cosmology.from_format`, the `CosmologyFromFormat.__call__` wraps the read call with `add_enabled_units(cu)` to enable cosmology-specific units (from `astropy.cosmology._src.units`).",
    "rubric": [
      "Identifies the delegation chain: from_table \u2192 from_row \u2192 from_mapping",
      "Explains that from_table uses string index matching via np.where on a 'name' column (or renamed equivalent found by iterating the rename dict for value=='name')",
      "Explains that from_row creates an inverse rename dict (inv_rename = {v: k for k, v in rename.items()}) to look up special keys like 'name', 'meta', and 'cosmology'",
      "Explains that from_row collects column-level metadata from row._table.itercols() via col.info.meta",
      "Explains that from_mapping uses inspect.signature, bind_partial, and apply_defaults to fill in default parameter values from the cosmology class",
      "Explains that _get_cosmology_class resolves the class from _COSMOLOGY_CLASSES dict using __qualname__ as the key, populated via __init_subclass__ / _register_cls",
      "Explains the extra parameter handling in from_mapping: VAR_KEYWORD (kind==4) absorbs extras, move_to_meta merges with {**params, **meta}, otherwise raises TypeError"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/row.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/io/connect.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/table.py] astropy.cosmology._src.io.builtin.table.from_table\nInstantiate a `~astropy.cosmology.Cosmology` from a |QTable|.\n\n    Parameters\n    ----------\n    table : `~astropy.table.Table`\n        The object to parse into a |Cosmology|.\n    index : int, str, or None, optional\n        Needed to select the row in tables with multiple rows. ``index`` can be an\n        integer for the row number or, if the table is indexed by a column, the value of\n        that column. If the table is not indexed and ``index`` is a string, the \"name\"\n        column is used as the indexing column.\n\n    move_to_meta : bool (optional, keyword-only)\n        Whether to move keyword arguments that are not in the Cosmology class' signature\n        to the Cosmology's metadata. This will only be applied if the Cosmology does NOT\n        have a keyword-only argument (e.g. ``**kwargs``). Arguments moved to the\n        metadata will be merged with existing metadata, preferring specified metadata in\n        the case of a merge conflict (e.g. for ``Cosmology(meta={'key':10}, key=42)``,\n        the ``Cosmology.meta`` will be ``{'key': 10}``).\n\n    cosmology : str or type or None (optional, keyword-only)\n        The cosmology class (or string name thereof) to use when constructing the\n        cosmology instance. The class also provides default parameter values, filling in\n        any non-mandatory arguments missing in 'table'.\n\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of column names in 'table' to field names of the |Cosmology| class.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology`\n\n    Examples\n    --------\n    To see loading a `~astropy.cosmology.Cosmology` from a Table with ``from_table``, we\n    will first make a |QTable| using :func:`~astropy.cosmology.Cosmology.to_format`.\n\n        >>> from astropy.cosmology import Cosmology, Planck18\n        >>> ct = Planck18.to_format(\"astropy.table\")\n        >>> ct\n        <QTable length=1>\n          name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                 km / (Mpc s)            K                 eV\n          str8     float64    float64 float64 float64  float64[3] float64\n        -------- ------------ ------- ------- ------- ----------- -------\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    Now this table can be used to load a new cosmological instance identical to the\n    ``Planck18`` cosmology from which it was generated.\n\n        >>> cosmo = Cosmology.from_format(ct, format=\"astropy.table\")\n        >>> cosmo\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    The ``cosmology`` information (column or metadata) may be omitted if the cosmology\n    class (or its string name) is passed as the ``cosmology`` keyword argument to\n    |Cosmology.from_format|.\n\n        >>> del ct.meta[\"cosmology\"]  # remove cosmology from metadata\n        >>> Cosmology.from_format(ct, cosmology=\"FlatLambdaCDM\")\n        FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    Alternatively, specific cosmology classes can be used to parse the data.\n\n        >>> from astropy.cosmology import FlatLambdaCDM\n        >>> FlatLambdaCDM.from_format(ct)\n         FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 2.7255 K>, Neff=3.046, m_nu=<Quantity [0.  , 0.  , 0.06] eV>, Ob0=0.04897)\n\n    When using a specific cosmology class, the class' default parameter values are used\n    to fill in any missing information.\n\n        >>> del ct[\"Tcmb0\"]  # show FlatLambdaCDM provides default\n        >>> FlatLambdaCDM.from_format(ct)\n         FlatLambdaCDM(name='Planck18', H0=<Quantity 67.66 km / (Mpc s)>, Om0=0.30966, Tcmb0=<Quantity 0. K>, Neff=3.046, m_nu=None, Ob0=0.04897)\n\n    For tables with multiple rows of cosmological parameters, the ``index`` argument is\n    needed to select the correct row. The index can be an integer for the row number or,\n    if the table is indexed by a column, the value of that column. If the table is not\n    indexed and ``index`` is a string, the \"name\" column is used as the indexing column.\n\n    Here is an example where ``index`` is needed and can be either an integer (for the\n    row number) or the name of one of the cosmologies, e.g. 'Planck15'.\n\n        >>> from astropy.cosmology import Planck13, Planck15, Planck18\n        >>> from astropy.table import vstack\n        >>> cts = vstack([c.to_format(\"astropy.table\")\n        ...               for c in (Planck13, Planck15, Planck18)],\n        ...              metadata_conflicts='silent')\n        >>> cts\n        <QTable length=3>\n          name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                 km / (Mpc s)            K                 eV\n          str8     float64    float64 float64 float64  float64[3]  float64\n        -------- ------------ ------- ------- ------- ----------- --------\n        Planck13        67.77 0.30712  2.7255   3.046 0.0 .. 0.06 0.048252\n        Planck15        67.74  0.3075  2.7255   3.046 0.0 .. 0.06   0.0486\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06  0.04897\n\n        >>> cosmo = Cosmology.from_format(cts, index=\"Planck15\", format=\"astropy.table\")\n        >>> cosmo == Planck15\n        True\n\n    Fields in the table can be renamed to match the `~astropy.cosmology.Cosmology`\n    class' signature using the ``rename`` argument. This is useful when the table's\n    column names do not match the class' parameter names.\n\n        >>> renamed_table = Planck18.to_format(\"astropy.table\", rename={\"H0\": \"Hubble\"})\n        >>> renamed_table\n        <QTable length=1>\n          name      Hubble      Om0    Tcmb0    Neff      m_nu      Ob0\n                 km / (Mpc s)            K                 eV\n          str8     float64    float64 float64 float64  float64[3] float64\n        -------- ------------ ------- ------- ------- ----------- -------\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n        >>> cosmo = Cosmology.from_format(renamed_table, format=\"astropy.table\",\n        ...                               rename={\"Hubble\": \"H0\"})\n        >>> cosmo == Planck18\n        True\n\n    For further examples, see :doc:`astropy:cosmology/io`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states that for merge conflicts with move_to_meta, 'preferring specified metadata in the case of a merge conflict (e.g. for Cosmology(meta={\"key\":10}, key=42), the Cosmology.meta will be {\"key\": 10})'. The answer states 'extra params are merged into metadata with {**params, **meta} (preferring existing meta values)', which is consistent with the documentation.",
      "Many implementation details (delegation chain from_table \u2192 from_row \u2192 from_mapping, inspect.signature usage, _COSMOLOGY_CLASSES registry, convert_parameter_to_column, CosmologyFromFormat.__call__ with add_enabled_units, etc.) are not verifiable from the provided documentation alone. These appear to be plausible code-level details but cannot be confirmed or denied from this docstring.",
      "The claim about np.where(table[nc] == index)[0] for row matching is an implementation detail not in the documentation, but is plausible.",
      "The claim about inv_rename = {v: k for k, v in rename.items()} is an implementation detail not verifiable from the doc.",
      "The claim about table_identify only returning True when origin == 'read' is not in the documentation."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone; no documentation leakage detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_30",
    "repo": "astropy",
    "question": "Why does the cosmology Table serialization layer use a chain of delegations (table\u2192row\u2192mapping) and what design choices drive its `to_table` implementation?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `to_table` function in `astropy/cosmology/_src/io/builtin/table.py` is designed as part of a layered serialization architecture where each format builds on top of the `mapping` format. Here's why specific design choices were made:\n\n1. **Why `to_table` calls `to_mapping` without `cosmology_as_str=True` and then manually converts**: `to_table` calls `to_mapping(cosmology)` (which returns the actual class object for `'cosmology'`) and then manually does `data[\"cosmology\"] = data[\"cosmology\"].__qualname__`. The `to_mapping` function also uses `__qualname__` when `cosmology_as_str=True`. By calling `to_mapping` in its default mode (which includes `move_from_meta=False` and no rename), `to_table` gets a clean dict with the class object, name, all parameters, and metadata nested under 'meta'. The class is then converted to a string for storage in the Table.\n\n2. **Why a unique index is added on the 'name' column**: `tbl.add_index(renames.get(\"name\", \"name\"), unique=True)` enables `from_table` to use string-based index lookups via `table.loc_indices[index]` when the table has multiple cosmology rows. Without this index, string lookups in `from_table` fall back to a `np.where(table[nc] == index)` search on the 'name' column.\n\n3. **Why the type check uses `issubclass` not `isinstance`**: The check `if not issubclass(cls, Table)` validates that `cls` is a *class* that is or inherits from `Table`, because `cls` is used as a constructor (`tbl = cls(data, meta=meta)`). This enables returning QTable, Table, or any custom Table subclass.\n\n4. **Why `convert_parameter_to_column` exists separately**: This utility (in `utils.py`) reshapes parameter values to have a leading dimension of 1 using `np.reshape(value, (1,) + np.shape(value))`, ensuring scalar parameters become 1-element arrays and array parameters (like `m_nu` with shape (3,)) become shape (1, 3). It also sets the Column's `description` from `parameter.__doc__` (which is set in `Parameter.__post_init__` via `object.__setattr__(self, \"__doc__\", self.doc)`).\n\n5. **Why `from_table` delegates to `from_row` which delegates to `from_mapping`**: This creates a chain where table \u2192 row \u2192 mapping \u2192 cosmology. The `from_row` function inverts the rename mapping, extracts column-level metadata from `row._table.itercols()` (merging per-column `col.info.meta` back into the metadata dict), and reassembles a mapping that includes the cosmology class from either the row or the table metadata. Then `from_mapping` uses `inspect.signature` to `bind_partial` and `apply_defaults` on the cosmology class, enabling default values to fill in missing parameters.\n\n6. **Why `table_identify` only activates on 'read' origin**: The identifier returns `True` only when `origin == \"read\"` and the input is a `Table` instance. This is consistent with how `CosmologyToFormat.__call__` always passes an explicit format string when writing, so identification is not needed for the write path.\n\n7. **Why `Cosmology.__astropy_table__` exists**: In `core.py`, this method delegates to `self.to_format(\"astropy.table\", cls=cls, **kwargs)`. This allows direct construction of a Table from a Cosmology instance (e.g., `QTable(cosmo)`) by implementing the `__astropy_table__` protocol method.",
    "rubric": [
      "Explains that `to_table` calls `to_mapping(cosmology)` without `cosmology_as_str=True`, getting the class object, then manually converts it to `__qualname__` string",
      "Explains why a unique index is added on the 'name' column: enables `loc_indices` string-based lookups in `from_table` for multi-row tables",
      "Explains the `issubclass(cls, Table)` check validates `cls` is a class (not instance) because it's used as a constructor for the output table",
      "Explains `convert_parameter_to_column` reshapes values to have a leading dimension of 1 using `np.reshape(value, (1,) + np.shape(value))` and sets `description` from `parameter.__doc__`",
      "Explains the delegation chain: `from_table` extracts a row, `from_row` merges column metadata and constructs a mapping, `from_mapping` uses `inspect.signature`/`bind_partial`/`apply_defaults` to fill defaults",
      "Explains that `table_identify` only returns True on `origin == 'read'` because the write path uses an explicit format string",
      "Explains that `Cosmology.__astropy_table__` delegates to `self.to_format('astropy.table', cls=cls, **kwargs)` to support direct Table construction from a Cosmology"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/io/builtin/row.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/parameter/core.py",
      "astropy/cosmology/_src/io/connect.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/table.py] astropy.cosmology._src.io.builtin.table.to_table\nSerialize the cosmology into a `~astropy.table.QTable`.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology`\n        The cosmology instance to convert to a table.\n    *args : object\n        Not used. Needed for compatibility with\n        `~astropy.io.registry.UnifiedReadWriteMethod`\n    cls : type (optional, keyword-only)\n        Astropy :class:`~astropy.table.Table` class or subclass type to return.\n        Default is :class:`~astropy.table.QTable`.\n    cosmology_in_meta : bool (optional, keyword-only)\n        Whether to put the cosmology class in the Table metadata (if `True`,\n        default) or as the first column (if `False`).\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of field names of the |Cosmology| class to column names in the\n        |Table|.\n\n    Returns\n    -------\n    `~astropy.table.QTable`\n        With columns for the cosmology parameters, and metadata and\n        cosmology class name in the Table's ``meta`` attribute\n\n    Raises\n    ------\n    TypeError\n        If kwarg (optional) 'cls' is not a subclass of `astropy.table.Table`\n\n    Examples\n    --------\n    A Cosmology as a `~astropy.table.QTable` will have the cosmology's name and\n    parameters as columns.\n\n        >>> from astropy.cosmology import Planck18\n        >>> ct = Planck18.to_format(\"astropy.table\")\n        >>> ct\n        <QTable length=1>\n          name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                 km / (Mpc s)            K                 eV\n          str8     float64    float64 float64 float64  float64[3] float64\n        -------- ------------ ------- ------- ------- ----------- -------\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    The cosmological class and other metadata, e.g. a paper reference, are in\n    the Table's metadata.\n\n        >>> ct.meta\n        {..., 'cosmology': 'FlatLambdaCDM'}\n\n    To move the cosmology class from the metadata to a Table column, set the\n    ``cosmology_in_meta`` argument to `False`:\n\n        >>> Planck18.to_format(\"astropy.table\", cosmology_in_meta=False)\n        <QTable length=1>\n          cosmology     name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                               km / (Mpc s)            K                 eV\n            str13       str8     float64    float64 float64 float64  float64[3] float64\n        ------------- -------- ------------ ------- ------- ------- ----------- -------\n        FlatLambdaCDM Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    Astropy recommends `~astropy.table.QTable` for tables with |Quantity| columns.\n    However the returned type may be overridden using the ``cls`` argument:\n\n        >>> from astropy.table import Table\n        >>> Planck18.to_format(\"astropy.table\", cls=Table)\n        <Table length=1>\n        ...\n\n    Fields of the cosmology may be renamed using the ``rename`` argument.\n\n        >>> Planck18.to_format(\"astropy.table\", rename={\"H0\": \"Hubble\"})\n        <QTable length=1>\n          name      Hubble      Om0    Tcmb0    Neff      m_nu      Ob0\n                 km / (Mpc s)            K                 eV\n          str8     float64    float64 float64 float64  float64[3] float64\n        -------- ------------ ------- ------- ------- ----------- -------\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer discusses internal implementation details (line numbers, internal function calls like `to_mapping`, `from_table`, `from_row`, `convert_parameter_to_column`, etc.) that are not present in the provided documentation. These cannot be verified against the doc chunk.",
      "The answer mentions `cosmology_as_str=True` parameter for `to_mapping` which is not documented in this chunk.",
      "The answer references `table_identify` function behavior which is not in the provided documentation.",
      "The answer mentions `Cosmology.__astropy_table__` method which is not in the provided documentation.",
      "While the answer doesn't directly contradict the documentation, it makes many specific claims about implementation details (line numbers, internal logic) that go far beyond what the documentation covers and cannot be verified from it."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor design rationale claims about why certain patterns were chosen needed slight rewording.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_28",
    "repo": "astropy",
    "question": "How does the cosmology-to-row serialization pipeline work, from `to_row` through `to_table` down to `from_mapping`, and back via `from_row`?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `to_row` function in `astropy/cosmology/_src/io/builtin/row.py` works by delegating to `to_table` and extracting the first row. Here's how the full pipeline works:\n\n1. **`to_row` delegates to `to_table`**: The function imports `to_table` from `.table` (a lazy import inside the function body), then calls it with `cls=table_cls`, `cosmology_in_meta=cosmology_in_meta`, and `rename=rename`. It then returns `table[0]` to extract the single row.\n\n2. **Key parameter difference**: `to_row` defaults `cosmology_in_meta=False`, while `to_table` defaults `cosmology_in_meta=True`. When `cosmology_in_meta=False`, the cosmology class remains as a column in the data; when `True`, it is moved into the table metadata. This means the \"astropy.row\" format will have a \"cosmology\" column by default, while \"astropy.table\" format will not.\n\n3. **`to_table` pipeline**: \n   - First calls `to_mapping(cosmology)` which creates a dict with keys: 'cosmology' (the class itself), 'name', all parameter values from `cosmology.parameters`, and 'meta' (a deep copy of cosmology.meta).\n   - Converts the cosmology class to its `__qualname__` string.\n   - Pops 'meta' to use as table metadata; if `cosmology_in_meta=True`, moves 'cosmology' from data to meta.\n   - Iterates over remaining data items: parameters are converted via `convert_parameter_to_column()` (which creates a Column with shape `(1,) + np.shape(value)`, `dtype=None`, and `parameter.__doc__` as `description`). Non-parameter items (like 'cosmology' and 'name') are wrapped in `Column([v])`.\n   - Constructs the table with `cls(data, meta=meta)`.\n   - Applies renames using `tbl.rename_column()`.\n   - Adds a unique index on the (possibly renamed) 'name' column via `tbl.add_index(renames.get('name', 'name'), unique=True)`.\n\n4. **`from_row` reverse pipeline**: Uses an inverted rename dict to find the keys for 'name', 'meta', and 'cosmology'. Accesses column metadata from `row._table.itercols()`. Constructs a mapping dict and delegates to `from_mapping()`, which uses `inspect.signature(cosmology)` with `bind_partial()` and `apply_defaults()` to set up parameter defaults before populating with provided values.\n\n5. **`from_mapping` handles extra parameters**: After binding known parameters, if the last parameter in the signature has `kind == 4` (VAR_KEYWORD / **kwargs), remaining params go there. If `move_to_meta=True`, they merge into the 'meta' argument (preferring explicitly set meta values). Otherwise raises TypeError.\n\n6. **Registration**: All three functions (`from_row`, `to_row`, `row_identify`) are registered with `convert_registry` under the format name \"astropy.row\". The `row_identify` function only returns True for `origin == 'read'` when the second positional arg is a `Row` instance and format is None or \"astropy.row\".\n\n7. **`_rename_map` validation**: When renaming keys in `from_mapping`, the helper `_rename_map` validates that rename target values are disjoint from existing map keys (raises ValueError if there's overlap).\n\n8. **Parameter vs non-parameter columns**: In `__init_subclass__`, parameters are filtered using `v.derived` \u2014 only non-derived parameters appear in `cosmology.parameters` (via `_parameters`), and thus become table columns. Derived parameters are stored separately in `_derived_parameters`.",
    "rubric": [
      "Explains that to_row delegates to to_table and extracts the first row with table[0]",
      "Notes the key default difference: to_row defaults cosmology_in_meta=False while to_table defaults cosmology_in_meta=True",
      "Describes to_table calling to_mapping to get a dict with cosmology class, name, parameters, and meta",
      "Explains that to_table converts the cosmology class to its __qualname__ string",
      "Describes how cosmology_in_meta controls whether cosmology stays as a column or moves to table metadata",
      "Explains that parameters are converted via convert_parameter_to_column (shape (1,)+np.shape(value), dtype=None, description from parameter.__doc__) while non-parameters use Column([v])",
      "Mentions that to_table applies renames via rename_column and adds a unique index on the name column",
      "Describes from_row extracting data from the row and column metadata, building a mapping dict, then delegating to from_mapping",
      "Explains from_mapping uses inspect.signature with bind_partial and apply_defaults to handle parameters and defaults",
      "Describes how from_mapping handles extra parameters: VAR_KEYWORD absorbs them, move_to_meta merges them into meta, otherwise raises TypeError",
      "Mentions the registration of functions with convert_registry under format name 'astropy.row'",
      "Notes that _rename_map validates that rename target values are disjoint from existing map keys"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/row.py",
      "astropy/cosmology/_src/io/builtin/table.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/io/builtin/utils.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/row.py] astropy.cosmology._src.io.builtin.row.to_row\nSerialize the cosmology into a `~astropy.table.Row`.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology`\n        The cosmology instance to convert to a mapping.\n    *args\n        Not used. Needed for compatibility with\n        `~astropy.io.registry.UnifiedReadWriteMethod`\n    table_cls : type (optional, keyword-only)\n        Astropy :class:`~astropy.table.Table` class or subclass type to use. Default is\n        :class:`~astropy.table.QTable`.\n    cosmology_in_meta : bool\n        Whether to put the cosmology class in the Table metadata (if `True`) or as the\n        first column (if `False`, default).\n    rename : Mapping[str, str] or None (optional, keyword-only)\n        A mapping of field names of the |Cosmology| to column names in the row.\n\n    Returns\n    -------\n    `~astropy.table.Row`\n        With columns for the cosmology parameters, and metadata in the Table's ``meta``\n        attribute. The cosmology class name will either be a column or in ``meta``,\n        depending on 'cosmology_in_meta'.\n\n    Examples\n    --------\n    A `~astropy.cosmology.Cosmology` as a `~astropy.table.Row` will have the cosmology's\n    name and parameters as columns.\n\n        >>> from astropy.cosmology import Planck18\n        >>> cr = Planck18.to_format(\"astropy.row\")\n        >>> cr\n        <Row index=0>\n          cosmology     name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                               km / (Mpc s)            K                 eV\n            str13       str8     float64    float64 float64 float64  float64[3] float64\n        ------------- -------- ------------ ------- ------- ------- ----------- -------\n        FlatLambdaCDM Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    The cosmological class and other metadata, e.g. a paper reference, are in the\n    Table's metadata.\n\n        >>> cr.meta\n        {'Oc0': 0.2607, 'n': 0.9665, ...}\n\n    To move the cosmology class from a column to the Table's metadata, set the\n    ``cosmology_in_meta`` argument to `True`:\n\n        >>> Planck18.to_format(\"astropy.table\", cosmology_in_meta=True)\n        <QTable length=1>\n        name        H0        Om0    Tcmb0    Neff      m_nu      Ob0\n                km / (Mpc s)            K                 eV\n        str8     float64    float64 float64 float64  float64[3] float64\n        -------- ------------ ------- ------- ------- ----------- -------\n        Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897\n\n    In Astropy, Row objects are always part of a Table. :class:`~astropy.table.QTable`\n    is recommended for tables with |Quantity| columns. However the returned type may be\n    overridden using the ``cls`` argument:\n\n        >>> from astropy.table import Table\n        >>> Planck18.to_format(\"astropy.table\", cls=Table)\n        <Table length=1>\n        ...\n\n    The columns can be renamed using the ``rename`` keyword argument.\n\n        >>> renamed = Planck18.to_format(\"astropy.row\", rename={\"H0\": \"Hubble\"})\n        >>> renamed\n        <Row index=0>\n          cosmology     name      Hubble      Om0    Tcmb0    Neff      m_nu      Ob0\n                               km / (Mpc s)            K                 eV\n            str13       str8     float64    float64 float64 float64  float64[3] float64\n        ------------- -------- ------------ ------- ------- ------- ----------- -------\n        FlatLambdaCDM Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation shows `table_cls` as the keyword parameter name for specifying the table class in `to_row`, but the answer refers to it correctly. However, in the examples section, the documentation uses `cls=Table` (not `table_cls=Table`) when calling `to_format('astropy.table', cls=Table)`, which creates some ambiguity about the actual parameter name.",
      "The claim about `to_table` defaulting `cosmology_in_meta=True` is not verifiable from the provided documentation, which only documents `to_row`. This is a plausible extrapolation from the example showing `Planck18.to_format('astropy.table', cosmology_in_meta=True)` where the cosmology column disappears, but the default for `to_table` is not stated in this doc.",
      "Many implementation details (e.g., `convert_parameter_to_column()`, `from_mapping` using `inspect.signature`, `_rename_map` validation, `ParametersAttribute` descriptor, `bind_partial()`, `apply_defaults()`, VAR_KEYWORD handling, `row_identify`, `convert_registry`) are not present in the provided documentation and appear to be code-level details that cannot be verified from this doc alone.",
      "The documentation example shows `cr.meta` containing `{'Oc0': 0.2607, 'n': 0.9665, ...}` but the answer doesn't mention these specific metadata fields.",
      "Point 3 mentions `tbl.add_index(renames.get('name', 'name'), unique=True)` which is not mentioned in the documentation at all - this is an implementation detail that cannot be verified."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for explanatory parentheticals about parameter semantics.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_31",
    "repo": "astropy",
    "question": "Where is the YAML deserialization for Cosmology objects implemented, and how does the data flow from input to a constructed instance?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `from_yaml` function is located in `astropy/cosmology/_src/io/builtin/yaml.py` and is registered with the `convert_registry` as a reader for the format name `\"yaml\"`: `convert_registry.register_reader(\"yaml\", Cosmology, from_yaml)`. The data flows through the following call chain:\n\n1. **Entry point**: `Cosmology.from_format(yml, format=\"yaml\")` invokes `CosmologyFromFormat.__call__` (in `astropy/cosmology/_src/io/connect.py`), which calls `self.registry.read(self._cls, obj, *args, format=format, **kwargs)` on the `convert_registry`.\n\n2. **`from_yaml` function** (yaml.py): Uses `astropy.units.add_enabled_units(cu)` as a context manager (where `cu` is `astropy.cosmology.units`, imported as `astropy.cosmology._src.units`). Inside this context, it calls `load(yml)` \u2014 the `load` function from `astropy.io.misc.yaml`.\n\n3. **`load` function** (`astropy/io/misc/yaml.py`): Simply wraps `yaml.load(stream, Loader=AstropyLoader)`. `AstropyLoader` extends `yaml.SafeLoader` and has registered constructors for cosmology YAML tags.\n\n4. **YAML tag-based dispatch**: Each concrete Cosmology class is registered at class creation time via `Cosmology.__init_subclass__` \u2192 `_register_cls()`, which calls `register_cosmology_yaml(cls)`. This creates a tag like `\"!astropy.cosmology.FlatLambdaCDM\"` using `FULLQUALNAME_SUBSTITUTIONS` to map internal module paths (e.g., `astropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM`) to public paths. The tag is registered as both a representer on `AstropyDumper` and a constructor on `AstropyLoader`.\n\n5. **`yaml_constructor` closure** (yaml.py): When the YAML loader encounters a cosmology tag, the constructor: (a) builds a mapping via `loader.construct_mapping(node)`, (b) converts `map[\"meta\"]` from a tuple of items back to a dict, (c) extracts the cosmology class name by splitting the tag on `\".\"` and taking the last element (`str(node.tag).split(\".\")[-1]`), and (d) delegates to `from_mapping(map, move_to_meta=False, cosmology=cosmology)`.\n\n6. **`from_mapping`** (mapping.py): Resolves the cosmology class name via `_get_cosmology_class`, which looks up string names in `_COSMOLOGY_CLASSES` dict. It then uses `inspect.signature(cosmology)` with `bind_partial()` and `apply_defaults()` to match mapping keys to constructor parameters, and finally instantiates the cosmology: `return cosmology(*ba.args, **ba.kwargs)`.\n\n7. **Type checking in `from_yaml`** (yaml.py): After the cosmology is loaded, if a `cosmology` argument was passed and is a string, it's resolved via `_COSMOLOGY_CLASSES[cosmology]`. If `cosmology is not None` and the loaded instance is not an instance of that class, a `TypeError` is raised.\n\n8. **`to_yaml` (serialization direction)**: The representer (created by `yaml_representer(tag)`) converts the cosmology to a mapping via `obj.to_format(\"mapping\")`, then pops the `\"cosmology\"` key, converts `meta` to a tuple of items, and calls `dumper.represent_mapping(tag, map)`. The `to_yaml` function itself simply calls `dump(cosmology)` from `astropy.io.misc.yaml`, which uses `AstropyDumper` with `default_flow_style=None`.\n\nRegistration is skipped for abstract Cosmology classes (`if not inspect.isabstract(cls)` in `__init_subclass__`). The `FULLQUALNAME_SUBSTITUTIONS` dict maps internal paths (including paths like `astropy.cosmology.flrw.*`) to their public `astropy.cosmology.*` equivalents.",
    "rubric": [
      "Identifies that `from_yaml` is located in `astropy/cosmology/_src/io/builtin/yaml.py` and registered with `convert_registry` for the 'yaml' format",
      "Explains that `Cosmology.from_format(yml, format='yaml')` triggers the convert_registry which dispatches to `from_yaml`",
      "Mentions that `from_yaml` uses `astropy.units.add_enabled_units` context manager with cosmology units before calling `load(yml)`",
      "Explains that `load` from `astropy.io.misc.yaml` calls `yaml.load` with `AstropyLoader` which has registered constructors for cosmology tags",
      "Describes how cosmology YAML tags (e.g., `!astropy.cosmology.FlatLambdaCDM`) are registered via `__init_subclass__` \u2192 `_register_cls()` \u2192 `register_cosmology_yaml(cls)` using `FULLQUALNAME_SUBSTITUTIONS`",
      "Explains the `yaml_constructor` closure: constructs mapping from node, converts meta tuple back to dict, extracts class name from tag, and delegates to `from_mapping`",
      "Describes that `from_mapping` resolves the cosmology class, uses `inspect.signature` with `bind_partial`/`apply_defaults` to match parameters, and instantiates the cosmology",
      "Mentions the post-load type check in `from_yaml` that raises TypeError if the loaded instance doesn't match the requested cosmology class"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/yaml.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/io/misc/yaml.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/io/connect.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/yaml.py] astropy.cosmology._src.io.builtin.yaml.from_yaml\nLoad `~astropy.cosmology.Cosmology` from :mod:`yaml` object.\n\n    Parameters\n    ----------\n    yml : str\n        :mod:`yaml` representation of |Cosmology| object\n    cosmology : str, |Cosmology| class, or None (optional, keyword-only)\n        The expected cosmology class (or string name thereof). This argument is\n        is only checked for correctness if not `None`.\n\n    Returns\n    -------\n    `~astropy.cosmology.Cosmology` subclass instance\n\n    Raises\n    ------\n    TypeError\n        If the |Cosmology| object loaded from ``yml`` is not an instance of\n        the ``cosmology`` (and ``cosmology`` is not `None`).\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Cosmology, Planck18\n    >>> yml = Planck18.to_format(\"yaml\")\n    >>> print(Cosmology.from_format(yml, format=\"yaml\"))\n    FlatLambdaCDM(name=\"Planck18\", H0=67.66 km / (Mpc s), Om0=0.30966,\n                  Tcmb0=2.7255 K, Neff=3.046, m_nu=[0. 0. 0.06] eV, Ob0=0.04897)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor leakage in line numbers, design rationale phrases, and version-specific documentation references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_32",
    "repo": "astropy",
    "question": "How does the YAML serialization and deserialization of Cosmology objects work end-to-end, from `to_yaml` through the representer and back via the constructor?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `to_yaml` function in `astropy/cosmology/_src/io/builtin/yaml.py` is a thin wrapper that simply calls `dump(cosmology)` from `astropy.io.misc.yaml`. The actual serialization algorithm works through a multi-step process:\n\n1. **`to_yaml` calls `dump(cosmology)`** \u2014 The `dump` function (in `astropy/io/misc/yaml.py`) calls `yaml.dump(data, stream=stream, Dumper=AstropyDumper, default_flow_style=None)`.\n\n2. **The `AstropyDumper` dispatches to a registered representer** \u2014 When yaml encounters a Cosmology object, it calls the representer registered for that specific cosmology class. Registration happens in `register_cosmology_yaml()`, which is called from `Cosmology._register_cls()` (in `core.py`) during `__init_subclass__` for every non-abstract Cosmology subclass.\n\n3. **The representer (created by `yaml_representer(tag)`)** performs these steps:\n   - Converts the cosmology to a mapping (dict) via `obj.to_format(\"mapping\")` \u2014 this calls `to_mapping()` which builds a dict with 'cosmology' (the class), 'name', all parameter values from `cosmology.parameters`, and 'meta' (a deep copy of metadata).\n   - Removes the 'cosmology' key from the mapping.\n   - Converts metadata dict to a tuple of items (`tuple(map[\"meta\"].items())`).\n   - Calls `dumper.represent_mapping(tag, map)` to produce the YAML mapping node.\n\n4. **The YAML tag** is constructed as `\"!\" + fqn` where `fqn` is the fully qualified name of the cosmology class, possibly substituted via `FULLQUALNAME_SUBSTITUTIONS` dictionary (e.g., `\"astropy.cosmology._src.flrw.lambdacdm.FlatLambdaCDM\"` becomes `\"astropy.cosmology.FlatLambdaCDM\"`).\n\n5. **Nested objects** (like `Quantity`, `Unit`, `ndarray`) are handled by other representers registered on `AstropyDumper` (e.g., `_quantity_representer`, `_unit_representer`, `_ndarray_representer`).\n\nFor deserialization (`from_yaml`), the process uses `astropy.io.misc.yaml.load()` with `AstropyLoader` (which extends `yaml.SafeLoader`). The constructor extracts the cosmology class name from the last segment of the YAML tag (`str(node.tag).split(\".\")[-1]`), restores metadata from tuple back to dict, and calls `from_mapping()` which uses `inspect.signature` to bind parameters to the cosmology constructor with defaults filled in.\n\nThe `from_yaml` function also wraps the `load()` call in `u.add_enabled_units(cu)` context manager to enable cosmological units during deserialization.",
    "rubric": [
      "Explains that `to_yaml` calls `dump(cosmology)` which uses `yaml.dump` with `AstropyDumper`",
      "Describes how `register_cosmology_yaml` registers a representer on `AstropyDumper` for each non-abstract Cosmology subclass (called via `_register_cls` / `__init_subclass__`)",
      "Explains that the representer converts the cosmology to a mapping via `obj.to_format('mapping')`, removes the 'cosmology' key, converts metadata to a tuple of items, then calls `dumper.represent_mapping(tag, map)`",
      "Explains how the YAML tag is constructed as '!' + fully qualified name, with possible substitution via FULLQUALNAME_SUBSTITUTIONS",
      "Mentions that nested objects like Quantity, Unit, and ndarray have their own registered representers on AstropyDumper",
      "Explains that deserialization uses `load()` with `AstropyLoader` (extends `yaml.SafeLoader`), and the constructor extracts the class name from the last segment of the tag, restores metadata from tuple to dict, and calls `from_mapping()`",
      "Notes that `from_yaml` wraps the `load()` call in `u.add_enabled_units(cu)` to enable cosmological units during deserialization"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/builtin/yaml.py",
      "astropy/io/misc/yaml.py",
      "astropy/cosmology/_src/io/builtin/mapping.py",
      "astropy/cosmology/_src/core.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/builtin/yaml.py] astropy.cosmology._src.io.builtin.yaml.to_yaml\nReturn the cosmology class, parameters, and metadata as a :mod:`yaml` object.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology` subclass instance\n        The cosmology to serialize.\n    *args : Any\n        Not used. Needed for compatibility with\n        `~astropy.io.registry.UnifiedReadWriteMethod`\n\n    Returns\n    -------\n    str\n        :mod:`yaml` representation of |Cosmology| object\n\n    Examples\n    --------\n    >>> from astropy.cosmology import Planck18\n    >>> Planck18.to_format(\"yaml\")\n    \"!astropy.cosmology...FlatLambdaCDM\\nH0: !astropy.units.Quantity...",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only line numbers, some design rationales, and specific unit names relied on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_33",
    "repo": "astropy",
    "question": "What is the architecture of `CosmologyRead` \u2014 its inheritance, registries, validation logic, and how it connects to the `Cosmology` class?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`CosmologyRead` is a subclass of `io_registry.UnifiedReadWrite` (defined in `astropy/io/registry/interface.py`) that serves as the worker object for the `Cosmology.read` class method, which is attached to the `Cosmology` class via `UnifiedReadWriteMethod(CosmologyRead)` \u2014 a descriptor that subclasses `property`.\n\n**Key architectural details discovered from code:**\n\n1. **Two separate registries**: The module defines two distinct `UnifiedIORegistry` instances at module level:\n   - `readwrite_registry` \u2014 used by `CosmologyRead` and `CosmologyWrite` for file I/O\n   - `convert_registry` \u2014 used by `CosmologyFromFormat` and `CosmologyToFormat` for object interchange (formats like `mapping`, `astropy.table`, `astropy.row`, `astropy.model`, `astropy.cosmology`, `yaml`)\n\n2. **`CosmologyRead.__init__`** passes `\"read\"` as the `method_name` and `readwrite_registry` as the `registry` to the parent `UnifiedReadWrite.__init__`, which stores them as `self._registry`, `self._instance`, `self._cls`, and `self._method_name`.\n\n3. **`CosmologyRead.__call__` validation logic**: When called from a subclass (i.e., `self._cls is not Cosmology`), it sets `kwargs.setdefault(\"cosmology\", self._cls)` and then validates that `kwargs[\"cosmology\"]` is either the class itself or its `__qualname__` string. If a user calls `w0wzCDM.read(..., cosmology=FlatLambdaCDM)`, it raises a `ValueError`.\n\n4. **Cosmological units context manager**: `CosmologyRead.__call__` wraps the actual registry read in `with add_enabled_units(cu)`, where `cu` is `astropy.cosmology._src.units`, which defines two custom units: `redshift` (with physical type \"redshift\") and `littleh` (with latex format `h_{100}`).\n\n5. **`UnifiedReadWriteMethod` is a `property` subclass** (`astropy/io/registry/interface.py`). Its `__get__` method calls `self.fget(instance, owner_cls)`, which constructs a fresh `CosmologyRead` (or similar) instance on each access, passing both the instance and the class.\n\n6. **Registered read formats for `readwrite_registry`**: The `__init__.py` imports `ecsv`, `html`, and `latex` modules, indicating these are the builtin I/O formats.\n\n7. **Registered read formats for `convert_registry`**: Six formats are registered as readers: `astropy.cosmology`, `mapping`, `astropy.model`, `astropy.row`, `astropy.table`, and `yaml`.\n\n8. **The actual read dispatching** happens in `UnifiedInputRegistry.read()` (`astropy/io/registry/core.py`), which handles tilde expansion via `_expand_user_in_args`, auto-format detection via `_get_valid_format`, opens files via `get_readable_fileobj` with `encoding=\"binary\"`, and after the reader returns, checks if the result is an instance of `cls` \u2014 if not, it tries to coerce via `cls(data, copy=False)`.\n\n9. **Cosmology subclass registration**: In `Cosmology.__init_subclass__`, non-abstract subclasses automatically call `cls._register_cls()`, which registers the class in `_COSMOLOGY_CLASSES` dict (keyed by `__qualname__`) and calls `register_cosmology_yaml(cls)` for YAML support.\n\n10. **`CosmologyFromFormat` also uses `\"read\"` as method_name** but points to `convert_registry` instead of `readwrite_registry`, and it has 7 `@overload` signatures for type-checking support covering formats: `astropy.cosmology`, `astropy.model`, `astropy.row`, `astropy.table`, `mapping`, `yaml`, and a generic fallback.",
    "rubric": [
      "Identifies that CosmologyRead is a subclass of io_registry.UnifiedReadWrite",
      "Explains there are two separate UnifiedIORegistry instances: readwrite_registry (for file I/O) and convert_registry (for format interchange)",
      "Describes that CosmologyRead.__init__ passes 'read' as method_name and readwrite_registry to the parent class",
      "Explains the __call__ validation: when called from a subclass (self._cls is not Cosmology), it sets kwargs cosmology default and validates it matches the class or its __qualname__",
      "Notes that __call__ wraps the registry read in an add_enabled_units(cu) context manager that enables custom cosmological units (redshift and littleh)",
      "Explains that CosmologyRead is attached to the Cosmology class via UnifiedReadWriteMethod(CosmologyRead), where UnifiedReadWriteMethod is a property subclass whose __get__ calls self.fget(instance, owner_cls)",
      "Mentions that CosmologyFromFormat also uses 'read' as method_name but targets convert_registry instead of readwrite_registry",
      "Identifies registered readwrite_registry read formats (ascii.ecsv, ascii.html) and convert_registry read formats (astropy.cosmology, mapping, astropy.model, astropy.row, astropy.table, yaml)"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/connect.py",
      "astropy/io/registry/interface.py",
      "astropy/io/registry/core.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/io/builtin/__init__.py",
      "astropy/cosmology/_src/units.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/connect.py] astropy.cosmology._src.io.connect.CosmologyRead\nastropy.cosmology._src.io.connect.CosmologyRead:\n  Read and parse data to a `~astropy.cosmology.Cosmology`.\n\n    This function provides the Cosmology interface to the Astropy unified I/O\n    layer. This allows easily reading a file in supported data formats using\n    syntax such as::\n\n        >>> from astropy.cosmology import Cosmology\n        >>> cosmo1 = Cosmology.read('<file name>')\n\n    When the ``read`` method is called from a subclass the subclass will\n    provide a keyword argument ``cosmology=<class>`` to the registered read\n    method. The method uses this cosmology class, regardless of the class\n    indicated in the file, and sets parameters' default values from the class'\n    signature.\n\n    Get help on the available readers using the ``help()`` method::\n\n      >>> Cosmology.read.help()  # Get help reading and list supported formats\n      >>> Cosmology.read.help(format='<format>')  # Get detailed help on a format\n      >>> Cosmology.read.list_formats()  # Print list of available formats\n\n    See also: https://docs.astropy.org/en/stable/io/unified.html\n\n    Parameters\n    ----------\n    *args\n        Positional arguments passed through to data reader. If supplied the\n        first argument is typically the input filename.\n    format : str (optional, keyword-only)\n        File format specifier.\n    **kwargs\n        Keyword arguments passed through to data reader.\n\n    Returns\n    -------\n    out : `~astropy.cosmology.Cosmology` subclass instance\n        `~astropy.cosmology.Cosmology` corresponding to file contents.\n\n    Notes\n    -----",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: one purpose description about 'ensuring units available during deserialization' and specific line number references; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_34",
    "repo": "astropy",
    "question": "Why does the cosmology I/O system use separate registries and identity checks in its format conversion classes?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `CosmologyFromFormat` class uses several performance-related design patterns:\n\n**1. Two Separate Registries to Avoid Pollution:**\nThe cosmology I/O system uses two separate `UnifiedIORegistry` instances (`readwrite_registry` for file I/O, `convert_registry` for format interchange). This avoids cross-contamination: `CosmologyFromFormat` and `CosmologyToFormat` use `convert_registry`, while `CosmologyRead`/`CosmologyWrite` use `readwrite_registry`. This separation means that when auto-identifying formats, each registry only loops through its own relevant identifiers (e.g., `convert_registry` has formats like 'mapping', 'astropy.table', 'astropy.row', 'astropy.cosmology', 'astropy.model', 'yaml' as seen from the overload signatures), not file-based ones.\n\n**2. Explicit `format` Bypasses Expensive Auto-Detection:**\nIn `UnifiedInputRegistry.read()`, when `format is None`, the code performs expensive operations: checking if args[0] is a path type, potentially opening the file via `get_readable_fileobj` with a context manager, and then calling `_get_valid_format` which loops through ALL registered identifiers via `identify_format`. Each identifier call invokes `_is_best_match`, which reconstructs a set of classes from `format_classes` on every invocation (`classes = {cls for fmt, cls in format_classes}`) and walks the MRO. When `format` is explicitly provided, all of this is skipped entirely \u2014 the code jumps directly to `get_reader(format, cls)`.\n\n**3. Identity Check (`is not`) Instead of Equality:**\nIn `CosmologyFromFormat.__call__`, the check `self._cls is not Cosmology` uses identity comparison (`is not`) rather than equality. This is a fast pointer comparison that avoids any custom `__eq__` methods. This guards the subclass validation logic: when called from the base `Cosmology` class, there's no need to validate or set the `cosmology` keyword argument.\n\n**4. `delay_doc_updates` Performance Optimization in Registry Base:**\nThe `_UnifiedIORegistryBase` has a `_delayed_docs_classes` set used to batch documentation updates. The `delay_doc_updates` context manager batches documentation rebuilds. Additionally, it short-circuits entirely when Python runs in optimized mode (`sys.flags.optimize >= 2`).\n\n**5. Pre-sorted Rows for `get_formats`:**\nThe `get_formats` method sorts the list of tuples before creating the Table (`sorted(rows, key=itemgetter(i_dataclass, i_deprecated, i_format))`).\n\n**6. `add_enabled_units` Context Manager:**\n`CosmologyFromFormat.__call__` and `CosmologyRead.__call__` both wrap their registry calls in `with add_enabled_units(cu)` to temporarily enable cosmological units (`redshift` and `littleh` from `astropy.cosmology._src.units`). Notably, `CosmologyToFormat.__call__` and `CosmologyWrite.__call__` do NOT use this context manager.",
    "rubric": [
      "Explains that two separate UnifiedIORegistry instances exist (readwrite_registry for file I/O and convert_registry for format interchange), preventing cross-contamination of identifiers during auto-detection",
      "Explains that when format is explicitly provided in UnifiedInputRegistry.read(), the expensive auto-detection logic (path checking, file opening, looping through all identifiers) is bypassed entirely",
      "Mentions that _is_best_match reconstructs a set of classes from format_classes on every invocation and walks the MRO, making format auto-identification expensive",
      "Explains that `self._cls is not Cosmology` uses identity comparison (pointer check) rather than equality to avoid invoking custom __eq__ and quickly skip subclass validation when called from the base Cosmology class",
      "Mentions the add_enabled_units(cu) context manager used in CosmologyFromFormat.__call__ and CosmologyRead.__call__ but not in the write/to_format counterparts",
      "Mentions the delay_doc_updates context manager that batches documentation rebuilds and short-circuits in Python optimized mode (sys.flags.optimize >= 2)"
    ],
    "key_files": [
      "astropy/cosmology/_src/io/connect.py",
      "astropy/io/registry/core.py",
      "astropy/io/registry/base.py",
      "astropy/io/registry/interface.py",
      "astropy/cosmology/_src/units.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/io/connect.py] astropy.cosmology._src.io.connect.CosmologyFromFormat\nastropy.cosmology._src.io.connect.CosmologyFromFormat:\n  Transform object to a `~astropy.cosmology.Cosmology`.\n\n    This function provides the Cosmology interface to the Astropy unified I/O\n    layer. This allows easily parsing supported data formats using\n    syntax such as::\n\n      >>> from astropy.cosmology import Cosmology\n      >>> cosmo1 = Cosmology.from_format(cosmo_mapping, format='mapping')\n\n    When the ``from_format`` method is called from a subclass the subclass will\n    provide a keyword argument ``cosmology=<class>`` to the registered parser.\n    The method uses this cosmology class, regardless of the class indicated in\n    the data, and sets parameters' default values from the class' signature.\n\n    Get help on the available readers using the ``help()`` method::\n\n      >>> Cosmology.from_format.help()  # Get help and list supported formats\n      >>> Cosmology.from_format.help('<format>')  # Get detailed help on a format\n      >>> Cosmology.from_format.list_formats()  # Print list of available formats\n\n    See also: https://docs.astropy.org/en/stable/io/unified.html\n\n    Parameters\n    ----------\n    obj : object\n        The object to parse according to 'format'\n    *args\n        Positional arguments passed through to data parser.\n    format : str or None, optional keyword-only\n        Object format specifier. For `None` (default) CosmologyFromFormat tries\n        to identify the correct format.\n    **kwargs\n        Keyword arguments passed through to data parser.\n        Parsers should accept the following keyword arguments:\n\n        - cosmology : the class (or string name thereof) to use / check when\n                      constructing the cosmology instance.\n\n    Returns\n    -------\n    out : `~astropy.cosmology.Cosmology` subclass instance\n        `~astropy.cosmology.Cosmology` corresponding to ``obj`` contents.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses detailed internal implementation patterns (line numbers, registry internals, context managers, sorting optimizations) that are entirely beyond the scope of the provided documentation. The documentation only describes the public interface of CosmologyFromFormat.",
      "Claims about specific line numbers (e.g., 'line 36', 'line 156', 'line 274'), specific registry names ('readwrite_registry', 'convert_registry'), specific format identifiers ('mapping', 'astropy.table', etc.), and internal methods ('_get_valid_format', '_is_best_match') cannot be verified from the provided documentation alone.",
      "The claim about `self._cls is not Cosmology` identity check on line 274 is plausible but unverifiable from the docstring alone.",
      "The documentation does mention `format='mapping'` as an example and describes the `format` parameter behavior, which is consistent with claim #2 about explicit format bypassing auto-detection, though the internal mechanism details are unverifiable.",
      "The documentation does mention that subclass calls provide `cosmology=<class>` keyword, which is broadly consistent with claim #3 about subclass validation logic, but the specific implementation details are unverifiable.",
      "The answer appears to be derived from actual source code inspection rather than from the provided documentation chunk, making most claims plausible extrapolations but not directly supported."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in quoted comments about performance rationale and specific issue numbers; most claims are verifiable from stripped code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_35",
    "repo": "astropy",
    "question": "Where is the `Parameter` descriptor defined, how does it integrate with `Cosmology`, and where are its validator functions registered?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Parameter` class is defined in `astropy/cosmology/_src/parameter/core.py` as a frozen dataclass. It serves as a descriptor that is used on `Cosmology` subclasses (defined in `astropy/cosmology/_src/core.py`). The validator functions that Parameter uses are defined in a separate module `astropy/cosmology/_src/parameter/converter.py`, where they are stored in a module-level dictionary `_REGISTRY_FVALIDATORS`. The registered validators include 'default' (maps to `validate_with_unit`), 'float' (maps to `validate_to_float`), 'scalar' (maps to `validate_to_scalar`), and 'non-negative' (maps to `validate_non_negative`).\n\nThe `Parameter` descriptor integrates with the `Cosmology` class through:\n1. `Cosmology.__init_subclass__` which calls `all_parameters(cls)` (from `utils.py`) to discover all Parameters on the class MRO, splitting them into `_parameters` (non-derived) and `_parameters_derived` (derived) as `MappingProxyType` dictionaries.\n2. The `ParametersAttribute` descriptor (in `descriptors.py`) which returns Parameter objects when accessed from the class, but parameter values when accessed from an instance.\n3. The `dataclass_decorator` in `core.py` which transforms Cosmology classes into frozen dataclasses with `frozen=True, repr=True, eq=False, init=True`.\n\nKey implementation details:\n- `Parameter` uses two custom descriptor fields: `_UnitField` (converts values to `astropy.units.Unit`) and `_FValidateField` (resolves string keys to callables from the registry).\n- In `__set__`, if the value being set is the Parameter descriptor itself (`value is self`), it's replaced with the default value.\n- The value is deep-copied before validation (`copy.deepcopy(value)`).\n- After validation, if the value has a `setflags` method (ndarray-like), it's made read-only via `value.setflags(write=False)`.\n- The `clone` method uses `dataclasses.replace` and sets `fvalidate` default to `self._fvalidate_in` (the original input, not the processed callable).\n- The `__repr__` method omits the `default` field when it equals `MISSING` (a `Sentinel` enum value).\n- `MISSING` is defined as `Sentinel.MISSING` where `Sentinel` is an `Enum` class with `auto()` value, and its `__repr__` returns `<MISSING>`.\n- In `FlatFLRWMixin`, `Ode0` is redefined as a derived parameter.",
    "rubric": [
      "Identifies that `Parameter` is defined as a frozen dataclass in the parameter subpackage's core module",
      "Explains that validator functions are stored in a module-level `_REGISTRY_FVALIDATORS` dictionary in the converter module",
      "Lists the registered validators: 'default' -> validate_with_unit, 'float' -> validate_to_float, 'scalar' -> validate_to_scalar, 'non-negative' -> validate_non_negative",
      "Describes that `Cosmology.__init_subclass__` calls `all_parameters(cls)` to discover Parameters on the MRO and splits them into `_parameters` (non-derived) and `_parameters_derived` (derived) as MappingProxyType",
      "Mentions the `ParametersAttribute` descriptor that returns Parameter objects from the class but parameter values from an instance",
      "Mentions that `dataclass_decorator` transforms Cosmology classes into frozen dataclasses with frozen=True, repr=True, eq=False, init=True",
      "Describes the `_UnitField` and `_FValidateField` custom descriptor fields used by Parameter",
      "Explains that `__set__` replaces `value is self` with the default, deep-copies the value, validates it, and optionally makes ndarray-like values read-only via setflags",
      "Notes that `clone` uses `dataclasses.replace` and defaults fvalidate to `self._fvalidate_in` (the original input)",
      "Mentions that MISSING is a Sentinel enum value whose __repr__ returns '<MISSING>', and that __repr__ omits the default field when it equals MISSING"
    ],
    "key_files": [
      "astropy/cosmology/_src/parameter/core.py",
      "astropy/cosmology/_src/parameter/converter.py",
      "astropy/cosmology/_src/core.py",
      "astropy/cosmology/_src/parameter/descriptors.py",
      "astropy/cosmology/_src/parameter/utils.py",
      "astropy/cosmology/_src/flrw/base.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/parameter/core.py] astropy.cosmology._src.parameter.core.Parameter\nastropy.cosmology._src.parameter.core.Parameter:\n  Cosmological parameter (descriptor).\n\n    Should only be used with a :class:`~astropy.cosmology.Cosmology` subclass.\n\n    Parameters\n    ----------\n    default : Any (optional, keyword-only)\n        Default value of the Parameter. If not given the\n        Parameter must be set when initializing the cosmology.\n    derived : bool (optional, keyword-only)\n        Whether the Parameter is 'derived', default `False`.\n        Derived parameters behave similarly to normal parameters, but are not\n        sorted by the |Cosmology| signature (probably not there) and are not\n        included in all methods. For reference, see ``Ode0`` in\n        ``FlatFLRWMixin``, which removes :math:`\\Omega_{de,0}`` as an\n        independent parameter (:math:`\\Omega_{de,0} \\equiv 1 - \\Omega_{tot}`).\n    unit : unit-like or None (optional, keyword-only)\n        The `~astropy.units.Unit` for the Parameter. If None (default) no\n        unit as assumed.\n    equivalencies : `~astropy.units.Equivalency` or sequence thereof\n        Unit equivalencies for this Parameter.\n    fvalidate : callable[[object, object, Any], Any] or str (optional, keyword-only)\n        Function to validate the Parameter value from instances of the\n        cosmology class. If \"default\", uses default validator to assign units\n        (with equivalencies), if Parameter has units.\n        For other valid string options, see ``Parameter._registry_validators``.\n        'fvalidate' can also be set through a decorator with\n        :meth:`~astropy.cosmology.Parameter.validator`.\n    doc : str or None (optional, keyword-only)\n        Parameter description.\n\n    Examples\n    --------\n    For worked examples see :class:`~astropy.cosmology.FLRW`.\n\nastropy.cosmology._src.parameter.core.Parameter.__set__:\n  Allows attribute setting once.\n\n        Raises AttributeError subsequently.\n\nastropy.cosmology._src.parameter.core.Parameter.validator:\n  Make new Parameter with custom ``fvalidate``.\n\n        Note: ``Parameter.fvalidator`` must be the top-most descriptor decorator.\n\n        Parameters\n        ----------\n        fvalidate : callable[[type, type, Any], Any]\n\n        Returns\n        -------\n        `~astropy.cosmology.Parameter`\n            Copy of this Parameter but with custom ``fvalidate``.\n\nastropy.cosmology._src.parameter.core.Parameter.validate:\n  Run the validator on this Parameter.\n\n        Parameters\n        ----------\n        cosmology : `~astropy.cosmology.Cosmology` instance\n        value : Any\n            The object to validate.\n\n        Returns\n        -------\n        Any\n            The output of calling ``fvalidate(cosmology, self, value)``\n            (yes, that parameter order).\n\nastropy.cosmology._src.parameter.core.Parameter.register_validator:\n  Decorator to register a new kind of validator function.\n\n        Parameters\n        ----------\n        key : str\n        fvalidate : callable[[object, object, Any], Any] or None, optional\n            Value validation function.\n\n        Returns\n        -------\n        ``validator`` or callable[``validator``]\n            if validator is None returns a function that takes and registers a\n            validator. This allows ``register_validator`` to be used as a\n            decorator.\n\nastropy.cosmology._src.parameter.core.Parameter.clone:\n  Clone this `Parameter`, changing any constructor argument.\n\n        Parameters\n        ----------\n        **kw\n            Passed to constructor. The current values, eg. ``fvalidate`` are\n            used as the default values, so an empty ``**kw`` is an exact copy.\n\n        Examples\n        --------\n        >>> p = Parameter()\n        >>> p\n        Parameter(derived=False, unit=None, equivalencies=[],\n                  fvalidate='default', doc=None)\n\n        >>> p.clone(unit=\"km\")\n        Parameter(derived=False, unit=Unit(\"km\"), equivalencies=[],\n                  fvalidate='default', doc=None)\n\nastropy.cosmology._src.parameter.core.Parameter.__repr__:\n  Return repr(self).",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states that derived parameters are 'not sorted by the Cosmology signature (probably not there) and are not included in all methods', but the answer claims they are split into '_parameters_derived' as MappingProxyType dictionaries - this is plausible but not documented here.",
      "The answer claims specific validator names ('float', 'scalar', 'non-negative') and their corresponding function names (validate_to_float, validate_to_scalar, validate_non_negative) - these are not mentioned in the documentation and could be hallucinated.",
      "The answer claims Parameter is a 'frozen dataclass' - the documentation does not state this explicitly, though it's plausible implementation detail.",
      "The answer claims __set__ replaces 'value is self' with default value for backwards compatibility with dataclass construction - not documented here.",
      "The answer claims values are deep-copied before validation and made read-only via setflags - not documented here.",
      "The answer's claim about FlatFLRWMixin computing Ode0 as '1.0 - (self.Om0 + self.Ogamma0 + self.Onu0 + self.Ok0)' differs slightly from the documentation which states '\u03a9_{de,0} \u2261 1 - \u03a9_{tot}' - this is a plausible expansion but the exact formula could be wrong.",
      "The answer mentions 'ParametersAttribute' descriptor in 'descriptors.py', 'dataclass_decorator', '_UnitField', '_FValidateField' - none of these are mentioned in the documentation.",
      "The answer claims MISSING is 'Sentinel.MISSING' where Sentinel is an Enum - not documented here."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: one claim about backwards compatibility rationale and one about truncated FlatFLRWMixin details; rest verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_37",
    "repo": "astropy",
    "question": "What is the architecture of the CurvatureComponent trait, and how does it integrate into the FLRW cosmology class hierarchy?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `CurvatureComponent` class in `astropy/cosmology/_src/traits/curvature.py` is a trait/mixin class that provides curvature-related attributes and methods to cosmology classes. Here is its architecture and how it connects to the broader system:\n\n**Class Structure:**\n- `CurvatureComponent` defines two abstract properties (`Ok0` and `is_flat`) decorated with both `@property` and `@abc.abstractmethod`, and one concrete method `Ok(z)`.\n- The `Ok(z)` method uses positional-only parameter syntax (`z: Quantity | ArrayLike, /`) and computes curvature density as: `self.Ok0 * (z + 1.0) ** 2 * self.inv_efunc(z) ** 2`.\n- It has an early-exit optimization: when `self.Ok0 == 0`, it returns `np.zeros(getattr(z, 'shape', ()))` without calling `inv_efunc`.\n\n**Integration into FLRW:**\n- `CurvatureComponent` is one of 12 traits that `FLRW` inherits from (in `astropy/cosmology/_src/flrw/base.py`). The full MRO for `FLRW` includes: `Cosmology`, `BaryonComponent`, `TotalComponent`, `CriticalDensity`, `MatterComponent`, `CurvatureComponent`, `DarkEnergyComponent`, `HubbleParameter`, `PhotonComponent`, `NeutrinoComponent`, `ScaleFactor`, `DarkMatterComponent`, `TemperatureCMB`.\n- In `FLRW`, `Ok0` is implemented as a `@cached_property` (not just `@property`) and is computed as: `1.0 - self.Om0 - self.Ode0 - self.Ogamma0 - self.Onu0`.\n- The `is_flat` property in `FLRW` checks both conditions: `(self.Ok0 == 0.0) and (self.Otot0 == 1.0)`.\n\n**FlatFLRWMixin behavior:**\n- In `FlatFLRWMixin.__post_init__`, `Ok0` is explicitly set to `0.0` in `self.__dict__` (bypassing the cached_property computation), and then `Ode0` is derived as `1.0 - (self.Om0 + self.Ogamma0 + self.Onu0 + self.Ok0)`.\n\n**The `aszarr` utility:**\n- Used by `Ok()` to convert the redshift argument. It handles: scalars (via `np.asarray`), `Quantity` objects (via `.to_value(cu.redshift)[...]`), plain `np.ndarray` (returned as-is), and everything else by constructing a `Quantity(z, cu.redshift, copy=None, subok=True).view(np.ndarray)`.\n- `SCALAR_TYPES` is defined as `(float, int, np.generic, Number)`.\n\n**Pattern similarity with other traits:**\n- `MatterComponent.Om(z)` uses the same formula pattern: `self.Om0 * (z + 1.0) ** 3 * self.inv_efunc(z) ** 2`.\n- `DarkEnergyComponent.Ode(z)` similarly has the `== 0` optimization and uses: `self.Ode0 * self.de_density_scale(z) * self.inv_efunc(z) ** 2`.\n\n**The `inv_efunc` dependency:**\n- `CurvatureComponent` implicitly depends on `inv_efunc` (not declared as abstract in the trait). The `HubbleParameter` trait declares `inv_efunc: Callable[[Any], FArray | float]` as a class-level annotation.\n- In `FLRW.inv_efunc`, the formula is: `(zp1**2 * ((Or * zp1 + self.Om0) * zp1 + self.Ok0) + self.Ode0 * self.de_density_scale(z)) ** (-0.5)`.",
    "rubric": [
      "Identifies CurvatureComponent as a trait/mixin class with two abstract properties (Ok0, is_flat) and one concrete method (Ok(z))",
      "Explains the Ok(z) formula: self.Ok0 * (z + 1.0)**2 * self.inv_efunc(z)**2, with an early-exit optimization returning zeros when Ok0 == 0",
      "Lists CurvatureComponent as one of 12 traits in FLRW's inheritance (along with BaryonComponent, TotalComponent, CriticalDensity, MatterComponent, DarkEnergyComponent, HubbleParameter, PhotonComponent, NeutrinoComponent, ScaleFactor, DarkMatterComponent, TemperatureCMB)",
      "Describes that FLRW implements Ok0 as a cached_property computing 1.0 - self.Om0 - self.Ode0 - self.Ogamma0 - self.Onu0",
      "Describes that FLRW's is_flat checks (self.Ok0 == 0.0) and (self.Otot0 == 1.0)",
      "Explains FlatFLRWMixin's __post_init__ sets Ok0 to 0.0 directly in __dict__ and derives Ode0 from 1.0 - (Om0 + Ogamma0 + Onu0 + Ok0)",
      "Notes that CurvatureComponent implicitly depends on inv_efunc, which is declared as a type annotation in HubbleParameter (or other traits) and implemented in FLRW",
      "Mentions the aszarr utility used for redshift conversion, handling scalars, Quantities, ndarrays, and other array-likes"
    ],
    "key_files": [
      "astropy/cosmology/_src/traits/curvature.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/traits/__init__.py",
      "astropy/cosmology/_src/traits/darkenergy.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/traits/curvature.py] astropy.cosmology._src.traits.curvature.CurvatureComponent\nastropy.cosmology._src.traits.curvature.CurvatureComponent:\n  The object has attributes and methods related to the global curvature.\n\n    This is a trait class; it is not meant to be instantiated directly, but\n    instead to be used as a mixin to other classes.\n\nastropy.cosmology._src.traits.curvature.CurvatureComponent.Ok0:\n  Omega curvature; the effective curvature density/critical density at z=0.\n\nastropy.cosmology._src.traits.curvature.CurvatureComponent.is_flat:\n  Return `bool`; `True` if the cosmology is globally flat.\n\nastropy.cosmology._src.traits.curvature.CurvatureComponent.Ok:\n  Return the equivalent density parameter for curvature at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        Ok : ndarray\n            The equivalent density parameter for curvature at each redshift.\n\n            .. versionchanged:: 7.2\n                Always returns a numpy object, never a `float`.\n\n        Examples\n        --------\n        >>> import numpy as np\n        >>> from astropy.cosmology import Planck18, units as cu\n\n        >>> Planck18.Ok(2)\n        array(0.)\n\n        >>> Planck18.Ok([1, 2])\n        array([0., 0.])\n\n        >>> Planck18.Ok(np.array([2]))\n        array([0.])\n\n        >>> Planck18.Ok(2 * cu.redshift)\n        array(0.)\n\n        >>> cosmo = Planck18.clone(Ode0=0.71, to_nonflat=True)\n\n        >>> cosmo.Ok0\n        np.float64(-0.021153694455455927)\n\n        >>> cosmo.Ok(100)\n        np.float64(-0.0006557825253017665)",
    "verification_verdict": "pass",
    "verification_issues": [
      "Most of the detailed implementation claims (e.g., exact MRO order, cached_property usage, aszarr internals, inv_efunc formula, FlatFLRWMixin behavior, SCALAR_TYPES definition) go far beyond what the documentation provides and cannot be verified from it alone \u2014 but they are plausible code-level details that don't contradict the documentation.",
      "The claim that Ok0 and is_flat are 'decorated with both @property and @abc.abstractmethod' cannot be verified from the documentation, but is a plausible implementation detail.",
      "The Ok(z) formula 'self.Ok0 * (z + 1.0) ** 2 * self.inv_efunc(z) ** 2' is consistent with the documented examples (e.g., Planck18.Ok(2) returning 0. when Ok0=0).",
      "The documented behavior of Ok() \u2014 parameter changes in versions 7.0 and 8.0, return type changes in 7.2, and examples \u2014 is not contradicted by any claims in the answer."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only 'ordered for speed' comment about SCALAR_TYPES was documentation-derived.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_36",
    "repo": "astropy",
    "question": "How does the FITS diff framework use `_BaseDiff` to propagate options between diff levels and determine if two objects are identical?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `_BaseDiff` class in `astropy/io/fits/diff.py` serves as the abstract base class for the FITS diff framework. Here is how the framework is implemented:\n\n## Class Hierarchy\n`_BaseDiff` has 6 concrete subclasses:\n- `FITSDiff(_BaseDiff)` \u2014 compares two FITS files/HDULists\n- `HDUDiff(_BaseDiff)` \u2014 compares two HDU objects\n- `HeaderDiff(_BaseDiff)` \u2014 compares two Header objects\n- `ImageDataDiff(_BaseDiff)` \u2014 compares two image data arrays\n- `RawDataDiff(ImageDataDiff)` \u2014 special case of ImageDataDiff treating data as bytes\n- `TableDataDiff(_BaseDiff)` \u2014 compares two table data arrays\n\n## Core Mechanism: `__init__` triggers `_diff()`\nThe `_BaseDiff.__init__` method stores `self.a` and `self.b`, initializes `self._fileobj = None` and `self._indent = 0`, and then immediately calls `self._diff()`. Subclasses must set all their attributes (e.g., `self.ignore_keywords`, `self.diff_*` attributes) **before** calling `super().__init__(a, b)`, because `_diff()` is invoked inside the parent constructor.\n\n## `identical` Property Implementation\nThe `identical` property uses introspection: it iterates over `self.__dict__`, finds all attributes starting with `\"diff_\"`, and returns `True` if none of them are truthy: `return not any(getattr(self, attr) for attr in self.__dict__ if attr.startswith(\"diff_\"))`.\n\n## `fromdiff` Classmethod \u2014 Argument Forwarding via `inspect.signature`\n`fromdiff(cls, other, a, b)` uses `inspect.signature(cls.__init__)` to get the parameter names of the target class's `__init__`. It skips the first 3 parameters (self, a, b) and for each remaining parameter name, checks if the source diff object (`other`) has a matching attribute with `hasattr/getattr`. This is how options like `ignore_keywords`, `rtol`, `atol` are automatically forwarded when `FITSDiff._diff` creates `HDUDiff` objects via `HDUDiff.fromdiff(self, ...)`, and when `HDUDiff._diff` creates `HeaderDiff`, `ImageDataDiff`, etc.\n\n## `report()` Method \u2014 Output Routing\nThe `report()` method supports three output modes: (1) if `fileobj` is `None`, it creates a `io.StringIO()` and returns the report as a string; (2) if `fileobj` is a `path_like` (which is defined as `(str, bytes, os.PathLike)` in `astropy/io/fits/util.py`), it opens the file for writing and closes it after; (3) if `fileobj` is a file-like object, it writes directly. Overwrite protection is handled via `NOT_OVERWRITING_MSG` from `astropy.utils.misc`. The `_writeln` helper uses `textwrap.indent(text, self._indent * \"  \")` \u2014 indentation is 2 spaces per indent level, not 4 as the docstring might suggest.\n\n## `FITSDiff.__init__` \u2014 File Lifecycle Management\n`FITSDiff.__init__` opens files passed as strings/PathLike using `fitsopen()` and uses a `try/finally` block around `super().__init__(a, b)` to ensure opened files are closed even if the diff raises an exception. It also separates wildcard HDU patterns (detected via `glob.has_magic()`) from literal HDU names into `self.ignore_hdu_patterns`, filtering HDUs with `fnmatch.filter`.\n\n## `HDUDiff._diff` \u2014 Memory Cleanup Pattern\nAfter creating data diff objects (`ImageDataDiff`, `TableDataDiff`, or `RawDataDiff`), `HDUDiff._diff` immediately sets `self.diff_data.a = None` and `self.diff_data.b = None` to release references to possibly memory-mapped arrays.\n\n## `ImageDataDiff._diff` \u2014 Tolerance Handling\nFor non-inexact dtypes (integers), `rtol` and `atol` are forced to 0 regardless of user settings. The method uses `where_not_allclose()` from `astropy.utils.diff`, which handles INF and NaN as equivalent by masking them with `np.ma.masked_invalid()`. When `atol == 0.0 and rtol == 0.0`, a fast path sets `thresh = 0` instead of computing `atol + rtol * np.abs(b)`. Values invalid in only one array (detected via `a.mask ^ b.mask`) are always reported as different.\n\n## `diff_values()` \u2014 Scalar Comparison\nThe `diff_values()` function in `astropy/utils/diff.py` has special float handling: if both values are floats and `b` is finite, it uses `not abs(a - b) <= atol + rtol * abs(b)`. If both are NaN, they are considered equal (returns `False`). For non-float types, simple `!=` is used.\n\n## `HeaderDiff` \u2014 String Input Support\n`HeaderDiff.__init__` accepts string inputs for headers, converting them via `Header.fromstring(a)` before calling `super().__init__()`. It also initializes `diff_keyword_values` and `diff_keyword_comments` as `defaultdict(list)` to accumulate per-keyword differences. The `diff_keyword_positions` attribute has a `# TODO: Implement this` comment \u2014 it's initialized but never populated.\n\n## Column Attribute Comparison\nTable column attributes compared are defined in `_COL_ATTRS`: unit, null, bscale, bzero, disp (display formats), and dim (dimensions).",
    "rubric": [
      "Identifies the class hierarchy: FITSDiff, HDUDiff, HeaderDiff, ImageDataDiff, RawDataDiff(ImageDataDiff), and TableDataDiff all inherit from _BaseDiff",
      "Explains that `__init__` stores self.a and self.b, then immediately calls self._diff(), so subclasses must set their attributes before calling super().__init__",
      "Explains the `identical` property introspects self.__dict__ for attributes starting with 'diff_' and returns True if none are truthy",
      "Describes the `fromdiff` classmethod: uses inspect.signature to get __init__ parameter names (skipping first 3: self, a, b), then copies matching attributes from the source diff object via hasattr/getattr",
      "Shows how options propagate: FITSDiff._diff creates HDUDiff via HDUDiff.fromdiff(self, ...), and HDUDiff._diff creates HeaderDiff/ImageDataDiff/TableDataDiff/RawDataDiff the same way",
      "Mentions that report() handles three output modes: returns string via StringIO when fileobj is None, opens/closes a file for path-like input, or writes directly to a file-like object",
      "Notes that HDUDiff._diff sets diff_data.a and diff_data.b to None after creation to release references to possibly memory-mapped arrays"
    ],
    "key_files": [
      "astropy/io/fits/diff.py",
      "astropy/utils/diff.py",
      "astropy/io/fits/util.py"
    ],
    "source_doc": "[docstring: astropy/io/fits/diff.py] astropy.io.fits.diff._BaseDiff\nastropy.io.fits.diff._BaseDiff:\n  Base class for all FITS diff objects.\n\n    When instantiating a FITS diff object, the first two arguments are always\n    the two objects to diff (two FITS files, two FITS headers, etc.).\n    Instantiating a ``_BaseDiff`` also causes the diff itself to be executed.\n    The returned ``_BaseDiff`` instance has a number of attribute that describe\n    the results of the diff operation.\n\n    The most basic attribute, present on all ``_BaseDiff`` instances, is\n    ``.identical`` which is `True` if the two objects being compared are\n    identical according to the diff method for objects of that type.\n\nastropy.io.fits.diff._BaseDiff.__init__:\n  The ``_BaseDiff`` class does not implement a ``_diff`` method and\n        should not be instantiated directly. Instead instantiate the\n        appropriate subclass of ``_BaseDiff`` for the objects being compared\n        (for example, use `HeaderDiff` to compare two `Header` objects.\n\nastropy.io.fits.diff._BaseDiff.__bool__:\n  A ``_BaseDiff`` object acts as `True` in a boolean context if the two\n        objects compared are different.  Otherwise it acts as `False`.\n\nastropy.io.fits.diff._BaseDiff.fromdiff:\n  Returns a new Diff object of a specific subclass from an existing diff\n        object, passing on the values for any arguments they share in common\n        (such as ignore_keywords).\n\n        For example::\n\n            >>> from astropy.io import fits\n            >>> hdul1, hdul2 = fits.HDUList(), fits.HDUList()\n            >>> headera, headerb = fits.Header(), fits.Header()\n            >>> fd = fits.FITSDiff(hdul1, hdul2, ignore_keywords=['*'])\n            >>> hd = fits.HeaderDiff.fromdiff(fd, headera, headerb)\n            >>> list(hd.ignore_keywords)\n            ['*']\n\nastropy.io.fits.diff._BaseDiff.identical:\n  `True` if all the ``.diff_*`` attributes on this diff instance are\n        empty, implying that no differences were found.\n\n        Any subclass of ``_BaseDiff`` must have at least one ``.diff_*``\n        attribute, which contains a non-empty value if and only if some\n        difference was found between the two objects being compared.\n\nastropy.io.fits.diff._BaseDiff.report:\n  Generates a text report on the differences (if any) between two\n        objects, and either returns it as a string or writes it to a file-like\n        object.\n\n        Parameters\n        ----------\n        fileobj : file-like, string, or None, optional\n            If `None`, this method returns the report as a string. Otherwise it\n            returns `None` and writes the report to the given file-like object\n            (which must have a ``.write()`` method at a minimum), or to a new\n            file at the path specified.\n\n        indent : int\n            The number of 4 space tabs to indent the report.\n\n        overwrite : bool, optional\n            If ``True``, overwrite the output file if it exists. Raises an\n            ``OSError`` if ``False`` and the output file exists. Default is\n            ``False``.\n\n        Returns\n        -------\n        report : str or None",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states indentation is '2 spaces per indent level, not 4 as the docstring might suggest', but the documentation explicitly says 'The number of 4 space tabs to indent the report.' This is a potential contradiction, though it could reflect actual implementation differing from the docstring.",
      "Many implementation details (e.g., exact introspection logic for `identical`, `fromdiff` using `inspect.signature`, memory cleanup in `HDUDiff._diff`, tolerance handling specifics, `diff_keyword_positions` TODO comment, `_COL_ATTRS` contents) are not verifiable from the provided documentation and could be hallucinations or accurate code-level details.",
      "The claim about 6 concrete subclasses is not verifiable from the provided documentation, though it's plausible.",
      "The claim that `FITSDiff` uses `glob.has_magic()` and `fnmatch.filter` for HDU patterns is not verifiable from the documentation provided."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_38",
    "repo": "astropy",
    "question": "Why does the NeutrinoComponent trait and its FLRW implementation use multiple optimization strategies for handling neutrino density calculations?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `NeutrinoComponent` trait in `astropy/cosmology/_src/traits/neutrino.py` is designed as a mixin class that separates the abstract interface (what subclasses must implement) from concrete implementations (computed from the abstracts). The key design aspects involve performance optimization and separation of concerns:\n\n**1. Performance optimization through early-exit check:**\nIn the `Onu` method, there's an explicit check `if self.Onu0 == 0` that returns `np.zeros_like(z)` immediately. This avoids calling both `self.Ogamma(z)` and `self.nu_relative_density(z)` when there are no neutrinos.\n\n**2. Three-tier scalar optimization in Cython:**\nThe FLRW concrete implementation has a Cython extension (`scalar_inv_efuncs.pyx`) with three separate functions per cosmology model: `_norel` (no relativistic species), `_nomnu` (massless neutrinos only), and the full version (with massive neutrinos). The `_norel` and `_nomnu` variants avoid calling the `nufunc` fitting function. The Cython `nufunc` hardcodes the fitting constants (p=1.83, 1/p=0.54644808743, k=0.3173) directly as literal values rather than using `cdef` variables.\n\n**3. Python list vs NumPy array for `nu_y_list`:**\nThe `NeutrinoInfo` NamedTuple stores both `nu_y` (NumPy array) and `nu_y_list` (Python list). The Cython functions explicitly accept `list nu_y` as their parameter type, indicating Python lists are preferred for the scalar integral routines.\n\n**4. `Tnu0` is a `cached_property` while `Tnu` is a regular method:**\nSince `Tnu0` depends only on the immutable `Tcmb0` parameter, it's computed once and cached. The computation is simply `TEMP_NEUTRINO * self.Tcmb0` where `TEMP_NEUTRINO = 0.7137658555036082`.\n\n**5. `nu_y` represents `m_nu / (kB * Tnu0)` and is divided by `(1+z)` at runtime:**\nIn the massive neutrino case, `curr_nu_y = self._nu_info.nu_y / (1.0 + np.expand_dims(z, axis=-1))` computes the effective mass-to-temperature ratio at each redshift. The `np.expand_dims` on `z` enables broadcasting when there are multiple neutrino species with different masses.\n\n**6. Immutability enforced with `object.__setattr__`:**\nThe `_nu_info` attribute is set using `object.__setattr__(self, \"_nu_info\", nu_info)` in `__post_init__` because FLRW instances are immutable (they use `__setattr__` blocking), requiring the bypass through `object.__setattr__`.\n\n**7. The trait declares type annotations for dependencies:**\nThe `NeutrinoComponent` class declares `Tcmb0: Quantity` and `Ogamma: Callable[[ArrayLike], FArray]` as type annotations (lines 112-113), documenting what the parent class must provide, without enforcing them at runtime.\n\n**8. `Neff` is shared equally among neutrino species:**\nThe `neff_per_nu` is computed as `self.Neff / n_nu` (where `n_nu = floor(Neff)`).",
    "rubric": [
      "Mentions the early-exit check in the Onu method where Onu0==0 returns zeros immediately to avoid unnecessary computation",
      "Explains the three-tier Cython scalar functions (_norel, _nomnu, full) that avoid calling nufunc when neutrinos are absent or massless",
      "Notes that nufunc hardcodes fitting constants (p=1.83, 1/p=0.54644808743, k=0.3173) as literals rather than cdef variables for speed",
      "Explains that nu_y_list is stored as a Python list because small Python lists are more efficient than small NumPy arrays for Cython scalar integrals",
      "Mentions that Tnu0 is a cached_property since it depends only on the immutable Tcmb0",
      "Explains the trait design separating abstract interface (has_massive_nu, Onu0, nu_relative_density) from concrete implementations",
      "Notes the use of object.__setattr__ to bypass immutability enforcement when setting _nu_info",
      "Mentions that NeutrinoComponent declares type annotations for dependencies (Tcmb0, Ogamma) that the parent class must provide"
    ],
    "key_files": [
      "astropy/cosmology/_src/traits/neutrino.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/flrw/scalar_inv_efuncs.pyx",
      "astropy/cosmology/_src/utils.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/traits/neutrino.py] astropy.cosmology._src.traits.neutrino.NeutrinoComponent\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent:\n  The cosmology has attributes and methods for the neutrino density.\n\n    This trait handles both massless neutrinos (relativistic, radiation-like)\n    and massive neutrinos (with complex evolution).\n\n    This is an abstract trait. Subclasses must implement:\n\n    - `has_massive_nu` (property): Whether there are massive neutrinos\n    - `Onu0` (property): Neutrino density parameter at z=0\n    - `nu_relative_density` (method): Neutrino-to-photon density ratio at redshift z\n\n    The parent class must provide ``Tcmb0`` (CMB temperature) and ``Ogamma``\n    (method to compute photon density at redshift z).\n\n    Notes\n    -----\n    The density in neutrinos is given by:\n\n    .. math::\n\n        \\rho_{\\nu} \\left(a\\right) = 0.2271 \\, N_{eff} \\,\n        f\\left(m_{\\nu} a / T_{\\nu 0} \\right) \\,\n        \\rho_{\\\\gamma} \\left( a \\right)\n\n    where\n\n    .. math::\n\n        f \\left(y\\right) = \\frac{120}{7 \\pi^4}\n        \\int_0^{\\\\infty} \\, dx \\frac{x^2 \\\\sqrt{x^2 + y^2}}\n        {e^x + 1}\n\n    assuming that all neutrino species have the same mass.\n\n    If they have different masses, a similar term is calculated for each\n    one.\n\n    Note that ``f`` has the asymptotic behavior :math:`f(0) = 1`. This\n    method returns :math:`0.2271 f` using an analytical fitting formula\n    (Komatsu et al., 2011), ApJS, 192, 18.\n\n    The neutrino density evolution depends on whether neutrinos are massive or massless:\n\n    - **Massless neutrinos**: Behave like radiation with density scaling as (1+z)^4.\n      The density is simply proportional to the photon density with a constant ratio\n      determined by Neff and Fermi-Dirac statistics.\n\n    - **Massive neutrinos**: Have complex evolution that transitions from relativistic\n      (radiation-like) at early times to non-relativistic (matter-like) at late times.\n      The implementation typically uses the Komatsu fitting formula (Komatsu\n      et al., 2011) for computational efficiency.\n\n    References\n    ----------\n    Komatsu et al. (2011), \"Seven-Year Wilkinson Microwave Anisotropy Probe\n    (WMAP) Observations: Cosmological Interpretation\", ApJS, 192, 18.\n\n    Examples\n    --------\n    >>> import numpy as np\n    >>> import astropy.units as u\n    >>> from astropy.cosmology.traits import NeutrinoComponent\n    >>> NEUTRINO_FERMI_DIRAC_CORRECTION = 0.22710731766  # 7/8 (4/11)^4/3\n    >>>\n    >>> class ExampleNeutrinoCosmology(NeutrinoComponent):\n    ...     def __init__(self):\n    ...         self.Tcmb0 = 2.7255 * u.K\n    ...         self.Neff = 3.046\n    ...         self.Ogamma0 = 5e-5\n    ...     @property\n    ...     def has_massive_nu(self):\n    ...         return False\n    ...     @property\n    ...     def Onu0(self):\n    ...         return NEUTRINO_FERMI_DIRAC_CORRECTION * self.Neff * self.Ogamma0\n    ...     def nu_relative_density(self, z):\n    ...         return NEUTRINO_FERMI_DIRAC_CORRECTION * self.Neff * np.ones_like(np.asarray(z))\n    ...     def Ogamma(self, z):\n    ...         return self.Ogamma0 * (np.asarray(z) + 1.0) ** 4\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.has_massive_nu:\n  Does this cosmology have at least one massive neutrino species?\n\n        Returns\n        -------\n        has_massive_nu : bool\n            True if at least one neutrino species has non-zero mass.\n\n        Notes\n        -----\n        Subclasses must implement this property.\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.Onu0:\n  Omega nu; the density/critical density of neutrinos at z=0.\n\n        Returns\n        -------\n        Onu0 : float\n            The density parameter for neutrinos at z=0.\n\n        Notes\n        -----\n        Subclasses must implement this property.\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.nu_relative_density:\n  Neutrino density function relative to the energy density in photons.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n        Returns\n        -------\n        f : array\n            The neutrino density scaling factor relative to the density in\n            photons at each redshift.\n\n        Notes\n        -----\n        Subclasses must implement this method. For massless neutrinos, this\n        should return a constant. For massive neutrinos, this should use an\n        appropriate fitting formula (e.g., Komatsu et al. 2011).\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.Tnu0:\n  Temperature of the neutrino background as |Quantity| at z=0.\n\n        Returns\n        -------\n        Tnu0 : Quantity ['temperature']\n            The neutrino temperature at z=0 in Kelvin.\n\n        Notes\n        -----\n        The neutrino temperature is related to the CMB temperature by:\n\n        .. math::\n\n            T_{\\nu 0} = \\left(\\frac{4}{11}\\right)^{1/3} T_{CMB}\n\n        This comes from the decoupling of neutrinos before electron-positron\n        annihilation. See Weinberg 'Cosmology' p 154 eq (3.1.21).\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.Onu:\n  Return the density parameter for neutrinos at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n        Returns\n        -------\n        Onu : ndarray\n            The energy density of neutrinos relative to the critical density at\n            each redshift. Note that this includes their kinetic energy (if\n            they have mass), so it is not equal to the commonly used\n            :math:`\\sum \\frac{m_{\\nu}}{94 eV}`, which does not include\n            kinetic energy.\n\n        Notes\n        -----\n        The neutrino density parameter evolves with redshift according to:\n\n        .. math::\n\n            \\\\Omega_{\\\\nu}(z) = \\\\Omega_{\\\\gamma}(z) \\\\times f(z)\n\n        where f(z) is the neutrino-to-photon density ratio computed by\n        nu_relative_density(z). For massless neutrinos, f(z) is constant.\n        For massive neutrinos, f(z) evolves as neutrinos transition from\n        relativistic to non-relativistic.\n\nastropy.cosmology._src.traits.neutrino.NeutrinoComponent.Tnu:\n  Return the neutrino temperature at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n        Returns\n        -------\n        Tnu : Quantity ['temperature']\n            The temperature of the cosmic neutrino background in K.\n\n        Notes\n        -----\n        The neutrino temperature scales with redshift as:\n\n        .. math::\n\n            T_{\\nu}(z) = T_{\\nu 0} (1 + z)\n\n        This simple scaling applies to both massless and massive neutrinos,\n        as the temperature depends only on the expansion of the universe.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims reference specific line numbers, variable names, implementation details (e.g., Cython extensions, NeutrinoInfo NamedTuple, nu_y_list, scalar_inv_efuncs.pyx, nufunc constants, object.__setattr__, neff_per_nu) that are not present in the provided documentation. These appear to be plausible implementation details but cannot be verified from the documentation alone.",
      "The claim about 'Tnu0 is a cached_property' is not explicitly stated in the documentation - the doc only says it returns the temperature at z=0 with the (4/11)^(1/3) formula.",
      "The claim about type annotations for dependencies (Tcmb0: Quantity, Ogamma: Callable) at specific line numbers cannot be verified from the documentation.",
      "The claim about Onu method having an early-exit check for Onu0 == 0 is not in the documentation.",
      "The TEMP_NEUTRINO constant value 0.7137658555036082 is plausible as (4/11)^(1/3) \u2248 0.71377, consistent with the documented formula.",
      "The documented relationship Tnu0 = (4/11)^(1/3) * Tcmb is correctly reflected in the answer.",
      "The documentation states the parent class must provide Tcmb0 and Ogamma, which is consistent with claim #7 about dependencies.",
      "The Komatsu et al. 2011 reference is correctly cited and consistent with the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few quoted comments and docstring-based rationale explanations needed to be removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_39",
    "repo": "astropy",
    "question": "Where does the data flow when `TemperatureCMB.Tcmb(z)` is called, and how does redshift conversion handle different input types?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `TemperatureCMB.Tcmb` method's data flows through the following call chain:\n\n1. **Entry point**: `TemperatureCMB.Tcmb(z)` in `astropy/cosmology/_src/traits/tcmb.py` computes `self.Tcmb0 * (aszarr(z) + 1.0)`. The `z` parameter is positional-only (enforced by `/` in the signature).\n\n2. **Redshift conversion via `aszarr`**: The input `z` is converted to a numpy array by `aszarr()` in `astropy/cosmology/_src/utils.py`. This function has four branches:\n   - **Scalars** (checked against `SCALAR_TYPES = (float, int, np.generic, Number)`): converted via `np.asarray(z)`.\n   - **Quantities** (checked before ndarray): converted via `z.to_value(cu.redshift)[...]`, which uses the `cu.redshift` unit. The `[...]` indexing is applied to the result.\n   - **Plain numpy arrays**: returned as-is.\n   - **Fallback** (e.g., Python lists): wrapped as `Quantity(z, cu.redshift, copy=None, subok=True).view(np.ndarray)`.\n\n3. **Dimensionless-to-redshift equivalency**: The reason `Quantity(0.5, '')` (dimensionless) can be converted to `cu.redshift` is that `astropy/cosmology/units.py` globally enables the `dimensionless_redshift()` equivalency via `_add_enabled_equivalencies(dimensionless_redshift())`. The `dimensionless_redshift()` function (in `units_equivalencies.py`) defines `Equivalency([(redshift, None)], \"dimensionless_redshift\")`, making redshift 1-to-1 equivalent to dimensionless.\n\n4. **Where `Tcmb0` comes from in FLRW**: In `astropy/cosmology/_src/flrw/base.py`, `Tcmb0` is defined as a `Parameter` with `default=0.0 * u.K`, `unit=\"Kelvin\"`, and `fvalidate=\"scalar\"`. The `TemperatureCMB` trait declares `Tcmb0: Quantity` as a type annotation in tcmb.py, which the FLRW class satisfies through its `Parameter` descriptor.\n\n5. **Downstream consumers of `Tcmb`**: \n   - `NeutrinoComponent.Tnu0` (a `cached_property` in `neutrino.py`) computes `TEMP_NEUTRINO * self.Tcmb0` where `TEMP_NEUTRINO = 0.7137658555036082`.\n   - `FLRW.Ogamma0` (a `cached_property` in `base.py`) computes photon density from `self.Tcmb0.value**4`.\n   - The `redshift_temperature()` equivalency in `units_equivalencies.py` calls `cosmology.Tcmb(z)` to convert redshifts to temperatures, and uses `z_at_value(cosmology.Tcmb, T)` for the inverse.\n   - The `with_redshift()` equivalency conditionally includes the temperature equivalency via `redshift_temperature()` when `Tcmb=True` (the default).\n\n6. **Class hierarchy**: `FLRW` inherits from both `Cosmology` and 12 trait classes including `TemperatureCMB` (visible in base.py). `TemperatureCMB` is a standalone mixin with no parent class other than `object`. The `Tcmb` method is only defined in the trait (no override in any FLRW subclass). The same `aszarr` function is also used by `NeutrinoComponent.Tnu()` and other methods in the codebase.",
    "rubric": [
      "Explains that Tcmb computes self.Tcmb0 * (aszarr(z) + 1.0) with z as positional-only",
      "Identifies that aszarr in utils.py converts z to a numpy array with branches for scalars, Quantities, ndarrays, and a fallback",
      "Describes the scalar branch using np.asarray for float/int/np.generic/Number types",
      "Describes the Quantity branch using z.to_value(cu.redshift) with [...] indexing",
      "Describes the ndarray branch returning z as-is",
      "Describes the fallback branch wrapping input as Quantity(z, cu.redshift).view(np.ndarray)",
      "Explains that dimensionless quantities can convert to cu.redshift because dimensionless_redshift() equivalency is globally enabled in astropy/cosmology/units.py",
      "Identifies that Tcmb0 is defined as a Parameter with default 0.0*u.K and unit Kelvin in the FLRW base class",
      "Mentions downstream consumers like Tnu0, Ogamma0, or the redshift_temperature equivalency that use Tcmb or Tcmb0",
      "Notes that TemperatureCMB is a mixin trait inherited by FLRW alongside other trait classes"
    ],
    "key_files": [
      "astropy/cosmology/_src/traits/tcmb.py",
      "astropy/cosmology/_src/utils.py",
      "astropy/cosmology/_src/flrw/base.py",
      "astropy/cosmology/_src/traits/neutrino.py",
      "astropy/cosmology/_src/units_equivalencies.py",
      "astropy/cosmology/units.py",
      "astropy/cosmology/_src/units.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/traits/tcmb.py] astropy.cosmology._src.traits.tcmb.TemperatureCMB\nastropy.cosmology._src.traits.tcmb.TemperatureCMB:\n  The trait for computing the cosmological background temperature.\n\nastropy.cosmology._src.traits.tcmb.TemperatureCMB.Tcmb:\n  Compute the CMB temperature at redshift ``z``.\n\n        Parameters\n        ----------\n        z : Quantity-like ['redshift'], array-like\n            Input redshift.\n\n            .. versionchanged:: 7.0\n                Passing z as a keyword argument is deprecated.\n\n            .. versionchanged:: 8.0\n               z must be a positional argument.\n\n        Returns\n        -------\n        Tcmb : Quantity ['temperature']\n            The temperature of the CMB.\n\n        Examples\n        --------\n        >>> import astropy.units as u\n        >>> from astropy.cosmology import Planck18, units as cu\n\n        >>> Planck18.Tcmb(u.Quantity([0.5, 1.0], cu.redshift))\n        <Quantity [4.08825, 5.451  ] K>\n\n        >>> Planck18.Tcmb(u.Quantity(0.5, ''))\n        <Quantity 4.08825 K>\n\n        >>> Planck18.Tcmb(0.5)\n        <Quantity 4.08825 K>\n\n        >>> Planck18.Tcmb([0.5, 1.0])\n        <Quantity [4.08825, 5.451  ] K>",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states that z must be a positional argument (as of version 8.0), and the answer claims it is 'positional-only (enforced by `/` in the signature on line 23)'. While consistent with the documentation's intent, the specific implementation detail (line number, `/` syntax) cannot be verified from the provided documentation alone.",
      "Many implementation details (line numbers, specific file paths, internal function names like `aszarr`, class hierarchy details, downstream consumers) are not verifiable from the provided documentation chunk alone - they are plausible code-level extrapolations but could be hallucinated.",
      "The claim about `aszarr` having four branches with specific behavior, `SCALAR_TYPES` tuple, and line numbers cannot be verified from this documentation.",
      "The claim about `dimensionless_redshift()` equivalency being globally enabled, `TEMP_NEUTRINO` constant value, and other implementation specifics are beyond the scope of verification from this doc chunk."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: a comment about SCALAR_TYPES ordering for speed, explanation that (4/11)^(1/3) = TEMP_NEUTRINO, and rationale for Quantity being checked before ndarray.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_41",
    "repo": "astropy",
    "question": "What does `redshift_hubble` return and how does it enable conversions between redshift, the Hubble parameter, and little-h?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`redshift_hubble` is a unit equivalency function that creates a `astropy.units.equivalencies.Equivalency` object (which is a subclass of `list`) enabling bidirectional conversion between cosmological redshift and both the Hubble parameter H(z) (in km/s/Mpc) and the dimensionless little-h unit.\n\nSpecifically, `redshift_hubble` returns an `Equivalency` containing exactly **two equivalency pairs** (tuples):\n\n1. `(redshift, u.km / u.s / u.Mpc, z_to_hubble, hubble_to_z)` \u2014 converts between the `redshift` unit and the Hubble parameter unit km/s/Mpc.\n2. `(redshift, littleh, z_to_littleh, littleh_to_z)` \u2014 converts between the `redshift` unit and the `littleh` unit.\n\nThe forward conversion `z_to_hubble` calls `cosmology.H(z)`, which is defined in `HubbleParameter.H()` (in `astropy/cosmology/_src/traits/hubble.py`) as `self.H0 * self.efunc(z)`.\n\nThe forward conversion `z_to_littleh` divides the Hubble parameter value (in km/s/Mpc) by 100 and attaches the `littleh` unit: `z_to_hubble(z).to_value(u.km / u.s / u.Mpc) / 100 * littleh`.\n\nThe reverse conversion `littleh_to_z` multiplies the little-h value by 100 (to get km/s/Mpc) and passes it to `hubble_to_z`, which internally calls `z_at_value(cosmology.H, H << (u.km / u.s / u.Mpc), **atzkw)` \u2014 using scipy's `minimize_scalar` with default method 'Brent', default zmin=1e-8, zmax=1000, ztol=1e-8, and maxfun=500.\n\nThe cosmology resolution pattern (shared by `redshift_hubble`, `redshift_distance`, `redshift_temperature`, and `with_redshift`) first falls back to `default_cosmology.get()` if None, then uses `default_cosmology.set(cosmology)` as a context manager to resolve the cosmology, calling `default_cosmology.get()` within the context.\n\nThe `redshift` unit is defined via `u.def_unit(['redshift'], ...)` with physical type 'redshift' and a LaTeX format of an empty string `r\"\"`. The `littleh` unit is defined via `u.def_unit(['littleh'], ...)` with LaTeX format `r'h_{100}'`.\n\nThe `with_redshift` function composes `redshift_hubble`, `redshift_temperature`, and `redshift_distance` into a single combined equivalency by calling `equivs.extend(redshift_hubble(cosmology, **atzkw))`, etc.\n\nWhen the `astropy.cosmology.units` module is imported, `dimensionless_redshift()` is automatically added as an enabled equivalency globally via `_add_enabled_equivalencies(dimensionless_redshift())`. Since `dimensionless_redshift()` returns `Equivalency([(redshift, None)])`, this enables redshift to be treated as interchangeable with dimensionless quantities without explicit equivalency specification.",
    "rubric": [
      "States that redshift_hubble returns an Equivalency object containing two equivalency pairs/tuples",
      "Identifies the first pair converts between the redshift unit and km/s/Mpc (Hubble parameter)",
      "Identifies the second pair converts between the redshift unit and the littleh unit",
      "Explains the forward conversion z_to_hubble calls cosmology.H(z), which computes H0 * efunc(z)",
      "Explains z_to_littleh divides the Hubble parameter value (in km/s/Mpc) by 100 and attaches the littleh unit",
      "Explains the reverse conversion hubble_to_z uses z_at_value to numerically invert cosmology.H",
      "Explains littleh_to_z multiplies by 100 and delegates to hubble_to_z",
      "Mentions the cosmology resolution pattern: falls back to default_cosmology.get() if None is passed"
    ],
    "key_files": [
      "astropy/cosmology/_src/units_equivalencies.py",
      "astropy/cosmology/_src/units.py",
      "astropy/cosmology/_src/traits/hubble.py",
      "astropy/cosmology/_src/funcs/optimize.py",
      "astropy/units/equivalencies.py",
      "astropy/cosmology/units.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/units_equivalencies.py] astropy.cosmology._src.units_equivalencies.redshift_hubble\nConvert quantities between redshift and Hubble parameter and little-h.\n\n    Care should be taken to not misinterpret a relativistic, gravitational, etc\n    redshift as a cosmological one.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology`, str, or None, optional\n        A cosmology realization or built-in cosmology's name (e.g. 'Planck18').\n        If None, will use the default cosmology\n        (controlled by |default_cosmology|).\n    **atzkw\n        keyword arguments for :func:`~astropy.cosmology.z_at_value`\n\n    Returns\n    -------\n    `~astropy.units.equivalencies.Equivalency`\n        Equivalency between redshift and Hubble parameter and little-h unit.\n\n    Examples\n    --------\n    >>> import astropy.units as u\n    >>> import astropy.cosmology.units as cu\n    >>> from astropy.cosmology import WMAP9\n\n    >>> z = 1100 * cu.redshift\n    >>> equivalency = cu.redshift_hubble(WMAP9)  # construct equivalency\n\n    >>> z.to(u.km / u.s / u.Mpc, equivalency)  # doctest: +FLOAT_CMP\n    <Quantity 1565637.40154275 km / (Mpc s)>\n\n    >>> z.to(cu.littleh, equivalency)  # doctest: +FLOAT_CMP\n    <Quantity 15656.37401543 littleh>",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many implementation details (e.g., exact tuple structure of equivalency pairs, z_at_value's default parameters like zmin=1e-8/zmax=1000/ztol=1e-8/maxfun=500, cosmology resolution pattern details, littleh LaTeX format, dimensionless_redshift auto-registration) go far beyond what the documentation states, but these are plausible code-level details rather than contradictions.",
      "The claim about 'minimize_scalar with default method Brent' for z_at_value is a specific implementation detail not mentioned in the doc \u2014 plausible but unverifiable from this doc alone.",
      "The claim that default cosmology is 'Planck18' is not stated in the doc but is a well-known astropy default \u2014 plausible extrapolation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage: default cosmology value 'Planck18', some interpretation of H0/efunc semantics, and details about default_cosmology.validate() fetching from realizations module.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_40",
    "repo": "astropy",
    "question": "How does the `redshift_distance` equivalency convert between redshift and distance, including the cosmology resolution and the inverse lookup mechanism?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `redshift_distance` function in `astropy/cosmology/_src/units_equivalencies.py` works through the following design:\n\n1. **Cosmology Resolution**: It resolves the cosmology parameter using a two-step process. First, if `cosmology` is None, it gets the default via `default_cosmology.get()`. Then it uses `default_cosmology.set(cosmology)` as a context manager (which calls `validate()` on the value, handling string-to-Cosmology lookup via `_get_from_registry`) and calls `default_cosmology.get()` again inside that context. This ensures string names like 'Planck18' are resolved to actual Cosmology instances.\n\n2. **Method lookup**: The distance method is obtained via `getattr(cosmology, kind + \"_distance\")`, concatenating the kind string ('comoving', 'lookback', or 'luminosity') with '_distance'. This maps to methods like `FLRW.comoving_distance`, `FLRW.lookback_distance`, or `FLRW.luminosity_distance`.\n\n3. **Forward conversion (z_to_distance)**: The inner function `z_to_distance(z)` simply calls `method(z)` \u2014 e.g., `cosmology.comoving_distance(z)`.\n\n4. **Reverse conversion (distance_to_z)**: The inner function `distance_to_z(d)` calls `z_at_value(method, d << u.Mpc, **atzkw)`. The `<<` operator creates a Quantity with Mpc units.\n\n5. **z_at_value internals**: The `z_at_value` function (in `astropy/cosmology/_src/funcs/optimize.py`) uses `scipy.optimize.minimize_scalar` to find the redshift where the cosmological function equals the target value. The default method is 'Brent', with default `zmin=1e-8`, `zmax=1000`, `ztol=1e-8`, and `maxfun=500`. It constructs an objective function `f(z) = abs(func(z) - fval)` and minimizes it. For out-of-bounds z values, it returns a penalty of `1.0e300 * (1.0 + offset)`. The function raises `CosmologyError` if the result is very close to either `zmin` or `zmax` (using `np.allclose`). The final result is returned as `result << cu.redshift` (a Quantity with redshift units).\n\n6. **Equivalency construction**: The function returns `u.Equivalency([(redshift, u.Mpc, z_to_distance, distance_to_z)], \"redshift_distance\", {\"cosmology\": cosmology, \"distance\": kind})`. The `Equivalency` class extends Python's `list` and stores `name` and `kwargs` as list attributes for metadata.\n\n7. **Integration in with_redshift**: The `with_redshift` function composes multiple equivalencies by calling `redshift_distance`, `redshift_hubble`, and `redshift_temperature`, extending a list with their results via the `extend()` method (since `Equivalency` is a list subclass).\n\n8. **Global registration**: The `dimensionless_redshift()` equivalency exists to make redshift<->dimensionless conversion available. The mechanism for global enablement would involve calling `_add_enabled_equivalencies(dimensionless_redshift())` at import time, though this registration step is not shown in the provided source files.",
    "rubric": [
      "Explains cosmology resolution: if None, gets default via default_cosmology.get(), then uses default_cosmology.set() as context manager (which calls validate to handle string-to-Cosmology lookup) and gets again inside that context",
      "Explains method lookup via getattr(cosmology, kind + '_distance') where kind is 'comoving', 'lookback', or 'luminosity'",
      "Explains forward conversion (z_to_distance) simply calls method(z) on the cosmology instance",
      "Explains reverse conversion (distance_to_z) calls z_at_value(method, d << u.Mpc, **atzkw) where << creates a Quantity with Mpc units",
      "Explains z_at_value uses scipy.optimize.minimize_scalar with default method 'Brent' and minimizes abs(func(z) - fval)",
      "Mentions default parameters: zmin=1e-8, zmax=1000, ztol=1e-8, maxfun=500",
      "Explains out-of-bounds penalty: returns 1e300 * (1 + offset) for z values outside [zmin, zmax]",
      "Explains CosmologyError is raised if result is very close to zmin or zmax (checked via np.allclose)",
      "Explains the result is returned as result << cu.redshift (a Quantity with redshift units)",
      "Explains the returned Equivalency tuple structure: [(redshift, u.Mpc, z_to_distance, distance_to_z)] with name and kwargs metadata"
    ],
    "key_files": [
      "astropy/cosmology/_src/units_equivalencies.py",
      "astropy/cosmology/_src/funcs/optimize.py",
      "astropy/cosmology/_src/units.py",
      "astropy/units/equivalencies.py",
      "astropy/cosmology/_src/default.py"
    ],
    "source_doc": "[docstring: astropy/cosmology/_src/units_equivalencies.py] astropy.cosmology._src.units_equivalencies.redshift_distance\nConvert quantities between redshift and distance.\n\n    Care should be taken to not misinterpret a relativistic, gravitational, etc\n    redshift as a cosmological one.\n\n    Parameters\n    ----------\n    cosmology : `~astropy.cosmology.Cosmology`, str, or None, optional\n        A cosmology realization or built-in cosmology's name (e.g. 'Planck18').\n        If None, will use the default cosmology\n        (controlled by |default_cosmology|).\n    kind : {'comoving', 'lookback', 'luminosity'}, optional\n        The distance type for the Equivalency.\n        Note this does NOT include the angular diameter distance as this\n        distance measure is not monotonic.\n    **atzkw\n        keyword arguments for :func:`~astropy.cosmology.z_at_value`, which is used to\n        convert distance to redshift.\n\n    Returns\n    -------\n    `~astropy.units.equivalencies.Equivalency`\n        Equivalency between redshift and temperature.\n\n    Raises\n    ------\n    `~astropy.cosmology.CosmologyError`\n        If the distance corresponds to a redshift that is larger than ``zmax``.\n    Exception\n        See :func:`~astropy.cosmology.z_at_value` for possible exceptions, e.g. if the\n        distance maps to a redshift that is larger than ``zmax``, the maximum redshift.\n\n    Examples\n    --------\n    >>> import astropy.units as u\n    >>> import astropy.cosmology.units as cu\n    >>> from astropy.cosmology import WMAP9\n\n    >>> z = 1100 * cu.redshift\n    >>> d = z.to(u.Mpc, cu.redshift_distance(WMAP9, kind=\"comoving\"))\n    >>> d  # doctest: +FLOAT_CMP\n    <Quantity 14004.03157418 Mpc>\n\n    The reverse operation is also possible, though not always as simple. To convert a\n    very large distance to a redshift it might be necessary to specify a large enough\n    ``zmax`` value. See :func:`~astropy.cosmology.z_at_value` for details.\n\n    >>> d.to(cu.redshift, cu.redshift_distance(WMAP9, kind=\"comoving\", zmax=1200))  # doctest: +FLOAT_CMP\n    <Quantity 1100.000 redshift>",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states the function returns 'Equivalency between redshift and temperature' which appears to be a typo in the docs (should be 'distance'), and the answer correctly describes it as an equivalency between redshift and distance.",
      "The answer contains many implementation details (z_at_value internals, scipy.optimize usage, penalty values, default parameters like zmin=1e-8, zmax=1000, ztol=1e-8, maxfun=500, Brent method, etc.) that cannot be verified from the provided documentation alone - these are plausible but unverifiable extrapolations about code internals.",
      "The answer mentions 'dimensionless_redshift()' being automatically added via '_add_enabled_equivalencies' - this is not mentioned in the documentation and cannot be verified.",
      "The answer describes the Equivalency constructor call with specific kwargs like 'redshift_distance' name and {'cosmology': cosmology, 'distance': kind} - these are implementation details not in the docs.",
      "The answer's claim about the `<<` operator 'without conversion, just attaching the unit' is a simplification - `<<` actually does perform conversion if needed, but for raw numeric values it effectively attaches units."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor rewrites needed for rationale about unit attachment semantics and global registration details not present in shown files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_42",
    "repo": "astropy",
    "question": "Why does the ECSV writer override `update_table_data` and wrap it in a serialize context manager?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `Ecsv.update_table_data` method converts mixin columns into plain `Column` or `MaskedColumn` objects before writing. Here's how it works:\n\n**Implementation**: The Ecsv class is the only ASCII format class that overrides `update_table_data` to call `serialize.represent_mixins_as_columns(table)`. The base class `update_table_data` simply returns the table unchanged (as implied by the override pattern). The ECSV override wraps the call to `serialize.represent_mixins_as_columns(table)` inside a `serialize_context_as(\"ecsv\")` context manager, which temporarily sets `BaseColumnInfo._serialize_context = \"ecsv\"` so that downstream serialization code can access the current serialization context.\n\n**Where it's called**: Within the `BaseReader.write()` method, `update_table_data` is called during the write process. The `_check_multidim_table` function (visible in `core.py`) returns early without checking if `max_ndim is None`, and the `Ecsv` class sets `max_ndim = None`, allowing multi-dimensional columns that mixin decomposition may produce.\n\n**What `represent_mixins_as_columns` does**: It iterates over all table columns and calls `_represent_mixin_as_column` for each. Mixin columns are decomposed into one or more plain columns. The function also adds `__serialized_columns__` to table.meta with reconstruction metadata. If no mixin columns exist, the original table is returned unmodified.\n\n**Symmetry with reading**: On the read side, `EcsvOutputter.__call__` calls `serialize._construct_mixins_from_columns(out)` which uses the `__serialized_columns__` metadata to reconstruct the original mixin columns from the serialized plain columns. The `_construct_mixin_from_obj_attrs_and_info` function checks a whitelist of allowed classes (`__construct_mixin_classes` tuple with entries like `astropy.time.core.Time`, `astropy.coordinates.sky_coordinate.SkyCoord`, etc.) before importing and instantiating classes.\n\n**The context manager mechanism**: `serialize_context_as` (defined in `astropy/utils/data_info.py`) is a `@contextmanager` that saves/restores `BaseColumnInfo._serialize_context`. This class variable is defined on `BaseColumnInfo` and can be accessed by mixin columns' serialization methods to determine the current output format context.",
    "rubric": [
      "Explains that update_table_data calls serialize.represent_mixins_as_columns to convert mixin columns into plain Column/MaskedColumn objects for writing",
      "Mentions that the serialize_context_as('ecsv') context manager sets BaseColumnInfo._serialize_context so downstream serialization code knows the output format",
      "Notes that the base class update_table_data simply returns the table unchanged, making ECSV the only ASCII format that overrides it",
      "Explains that represent_mixins_as_columns decomposes mixin columns and adds __serialized_columns__ metadata for reconstruction on read",
      "Mentions the read-side symmetry: EcsvOutputter uses _construct_mixins_from_columns to reconstruct mixins from the serialized plain columns",
      "Notes that ECSV sets max_ndim = None to allow multi-dimensional columns that mixin decomposition may produce"
    ],
    "key_files": [
      "astropy/io/ascii/ecsv.py",
      "astropy/io/ascii/core.py",
      "astropy/table/serialize.py",
      "astropy/utils/data_info.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/ecsv.py] astropy.io.ascii.ecsv.Ecsv\nastropy.io.ascii.ecsv.Ecsv:\n  ECSV (Enhanced Character Separated Values) format table.\n\n    Th ECSV format allows for specification of key table and column meta-data, in\n    particular the data type and unit.\n\n    See: https://github.com/astropy/astropy-APEs/blob/main/APE6.rst\n\n    Examples\n    --------\n    >>> from astropy.table import Table\n    >>> ecsv_content = '''# %ECSV 0.9\n    ... # ---\n    ... # datatype:\n    ... # - {name: a, unit: m / s, datatype: int64, format: '%03d'}\n    ... # - {name: b, unit: km, datatype: int64, description: This is column b}\n    ... a b\n    ... 001 2\n    ... 004 3\n    ... '''\n\n    >>> Table.read(ecsv_content, format='ascii.ecsv')\n    <Table length=2>\n      a     b\n    m / s   km\n    int64 int64\n    ----- -----\n      001     2\n      004     3\n\nastropy.io.ascii.ecsv.Ecsv.update_table_data:\n  Update table columns in place if mixin columns are present.\n\n        This is a hook to allow updating the table columns after name\n        filtering but before setting up to write the data.  This is currently\n        only used by ECSV and is otherwise just a pass-through.\n\n        Parameters\n        ----------\n        table : `astropy.table.Table`\n            Input table for writing\n\n        Returns\n        -------\n        table : `astropy.table.Table`\n            Output table for writing",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many specific implementation details (line numbers like 'core.py line 1522', 'line 530', 'line 1575'; specific method names like `_apply_include_exclude_names`, `_check_multidim_table`; the `serialize_context_as` context manager; the `__construct_mixin_classes` tuple with '~40 entries'; `astropy/utils/data_info.py line 54`; `max_ndim = None` vs `max_ndim = 1`) are not present in the provided documentation and cannot be verified from it \u2014 they may be plausible extrapolations from actual source code but are not supported by the given doc chunk.",
      "The core claims about `update_table_data` being a hook for mixin columns, being used only by ECSV, and being a pass-through otherwise are well-supported by the documentation.",
      "The documentation says the method takes and returns a `astropy.table.Table`, which is consistent with the answer's description."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code structure, with some design rationale and line number references needing removal or adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_43",
    "repo": "astropy",
    "question": "Where does the IPAC format writer implement its write logic, and how does data flow from the table through header and data output?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `Ipac` format writer's `write()` method is located in `astropy/io/ascii/ipac.py`. It overrides the parent `BaseReader.write()` method from `astropy/io/ascii/core.py` with a completely custom implementation. The data flow during writing is:\n\n1. `Ipac.write()` is called with a Table object\n2. It appends `(core.masked, 'null')` as a default fill value\n3. It calls `core._apply_include_exclude_names()` for column filtering\n4. Comments are written using `textwrap.wrap()` with width=80, prefix `\"\\\\ \"`\n5. Keywords are formatted using `f\"\\\\{keyword.strip()}={val!r}\"` (note: uses Python `repr()` for value formatting)\n6. `self.data._set_fill_values()` is called (from `core.BaseData`)\n7. Column widths are computed as `max(col.width, col.headwidth)` where headwidth comes from `header.str_vals()` and col.width from data values\n8. `self.header.write(lines, widths)` writes 4 header rows via `IpacHeaderSplitter.join()`\n9. `self.data.write(lines, widths, data_str_vals)` writes data rows via `IpacDataSplitter.join()`\n\nThe `IpacHeaderSplitter.join()` always uses pipe (`|`) bookends unconditionally. The `IpacDataSplitter` (inheriting `FixedWidthSplitter.join()`) uses `delimiter=' '`, `bookend=True`, `delimiter_pad=''` \u2014 resulting in data lines that start with a space, separate columns with spaces, and end with a space.\n\nThe `IpacHeader.get_col_type()` method uses `startswith` matching against an ordered `col_type_list` tuple, meaning column types can be abbreviated (e.g., 'i' matches 'integer', 'd' matches 'double' before 'date'). The `IpacHeader.process_lines()` identifies header lines as those that both start and end with the pipe delimiter.\n\nThe class hierarchy is: `Ipac` \u2192 `basic.Basic` \u2192 `core.BaseReader` (via `MetaBaseReader` metaclass which auto-registers the format as 'ipac' in `FORMAT_CLASSES`).",
    "rubric": [
      "Identifies that Ipac.write() is in astropy/io/ascii/ipac.py and overrides the parent BaseReader.write() with a custom implementation",
      "Mentions that fill values are appended (core.masked, 'null') as a default before writing",
      "Explains that core._apply_include_exclude_names() is called for column filtering",
      "Describes comments being wrapped with textwrap.wrap() at width 80 with prefix '\\ '",
      "Describes keywords formatted using f-string with repr() for values",
      "Notes that self.data._set_fill_values() is called before computing widths",
      "Explains column widths are computed as max(col.width, col.headwidth) from header str_vals and data str_vals",
      "Explains that self.header.write(lines, widths) writes header rows via IpacHeaderSplitter.join() which always uses pipe bookends",
      "Explains that self.data.write(lines, widths, data_str_vals) writes data rows via IpacDataSplitter (inheriting FixedWidthSplitter.join()) with delimiter=' ', bookend=True, delimiter_pad=''",
      "Mentions the class hierarchy: Ipac \u2192 basic.Basic \u2192 core.BaseReader with MetaBaseReader metaclass registering the format"
    ],
    "key_files": [
      "astropy/io/ascii/ipac.py",
      "astropy/io/ascii/core.py",
      "astropy/io/ascii/fixedwidth.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/ipac.py] astropy.io.ascii.ipac.Ipac\nastropy.io.ascii.ipac.Ipac:\n  IPAC format table.\n\n    See: https://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html\n\n    Example::\n\n      \\\\name=value\n      \\\\ Comment\n      |  column1 |   column2 | column3 | column4  |    column5    |\n      |  double  |   double  |   int   |   double |     char      |\n      |  unit    |   unit    |   unit  |    unit  |     unit      |\n      |  null    |   null    |   null  |    null  |     null      |\n       2.0978     29.09056    73765     2.06000    B8IVpMnHg\n\n    Or::\n\n      |-----ra---|----dec---|---sao---|------v---|----sptype--------|\n        2.09708   29.09056     73765   2.06000    B8IVpMnHg\n\n    The comments and keywords defined in the header are available via the output\n    table ``meta`` attribute::\n\n      >>> import os\n      >>> from astropy.io import ascii\n      >>> filename = os.path.join(ascii.__path__[0], 'tests/data/ipac.dat')\n      >>> data = ascii.read(filename)\n      >>> print(data.meta['comments'])\n      ['This is an example of a valid comment']\n      >>> for name, keyword in data.meta['keywords'].items():\n      ...     print(name, keyword['value'])\n      ...\n      intval 1\n      floatval 2300.0\n      date Wed Sp 20 09:48:36 1995\n      key_continue IPAC keywords can continue across lines\n\n    Note that there are different conventions for characters occurring below the\n    position of the ``|`` symbol in IPAC tables. By default, any character\n    below a ``|`` will be ignored (since this is the current standard),\n    but if you need to read files that assume characters below the ``|``\n    symbols belong to the column before or after the ``|``, you can specify\n    ``definition='left'`` or ``definition='right'`` respectively when reading\n    the table (the default is ``definition='ignore'``). The following examples\n    demonstrate the different conventions:\n\n    * ``definition='ignore'``::\n\n        |   ra  |  dec  |\n        | float | float |\n          1.2345  6.7890\n\n    * ``definition='left'``::\n\n        |   ra  |  dec  |\n        | float | float |\n           1.2345  6.7890\n\n    * ``definition='right'``::\n\n        |   ra  |  dec  |\n        | float | float |\n        1.2345  6.7890\n\n    IPAC tables can specify a null value in the header that is shown in place\n    of missing or bad data. On writing, this value defaults to ``null``.\n    To specify a different null value, use the ``fill_values`` option to\n    replace masked values with a string or number of your choice as\n    described in :ref:`astropy:io_ascii_write_parameters`::\n\n        >>> from astropy.io.ascii import masked\n        >>> fill = [(masked, 'N/A', 'ra'), (masked, -999, 'sptype')]\n        >>> ascii.write(data, format='ipac', fill_values=fill)\n        \\ This is an example of a valid comment\n        ...\n        |          ra|         dec|      sai|          v2|            sptype|\n        |      double|      double|     long|      double|              char|\n        |        unit|        unit|     unit|        unit|              ergs|\n        |         N/A|        null|     null|        null|              -999|\n                  N/A     29.09056      null         2.06               -999\n         2345678901.0 3456789012.0 456789012 4567890123.0 567890123456789012\n\n    When writing a table with a column of integers, the data type is output\n    as ``int`` when the column ``dtype.itemsize`` is less than or equal to 2;\n    otherwise the data type is ``long``. For a column of floating-point values,\n    the data type is ``float`` when ``dtype.itemsize`` is less than or equal\n    to 4; otherwise the data type is ``double``.\n\n    Parameters\n    ----------\n    definition : str, optional\n        Specify the convention for characters in the data table that occur\n        directly below the pipe (``|``) symbol in the header column definition:\n\n          * 'ignore' - Any character beneath a pipe symbol is ignored (default)\n          * 'right' - Character is associated with the column to the right\n          * 'left' - Character is associated with the column to the left\n\n    DBMS : bool, optional\n        If true, this verifies that written tables adhere (semantically)\n        to the `IPAC/DBMS\n        <https://irsa.ipac.caltech.edu/applications/DDGEN/Doc/DBMSrestriction.html>`_\n        definition of IPAC tables. If 'False' it only checks for the (less strict)\n        `IPAC <https://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html>`_\n        definition.\n\nastropy.io.ascii.ipac.Ipac.write:\n  Write ``table`` as list of strings.\n\n        Parameters\n        ----------\n        table : `~astropy.table.Table`\n            Input table data\n\n        Returns\n        -------\n        lines : list\n            List of strings corresponding to ASCII table",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, internal method calls, class hierarchies) that cannot be verified from the provided documentation alone. These appear to be plausible code-level details but are not directly supported by the documentation.",
      "The claim about keyword formatting using `f\"\\\\{keyword.strip()}={val!r}\"` (Python repr() for value formatting) cannot be verified from the documentation. The documentation shows keyword output like `\\name=value` but doesn't specify the exact formatting mechanism.",
      "The claim about `IpacDataSplitter` resulting in 'data lines that start with a space' is consistent with the documentation examples showing data lines starting with spaces.",
      "The claim about `col_type_list` ordering where 'd' matches 'double' before 'date' cannot be verified from the documentation, though the documentation does mention keyword 'date' as a keyword value, not a column type.",
      "The claim about default fill value being `(core.masked, 'null')` is consistent with the documentation stating 'On writing, this value defaults to null'.",
      "The claim about integer column dtype output rules is consistent with the documentation's description of int vs long and float vs double based on itemsize."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only specific line number references needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_44",
    "repo": "astropy",
    "question": "How does the Latex reader/writer class handle reading and writing LaTeX tables, including line detection, splitting, and the shared configuration between header and data?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `Latex` class in `astropy/io/ascii/latex.py` implements LaTeX table reading and writing through a multi-component architecture. Here's how the algorithm works:\n\n**Writing Algorithm:**\n1. `Latex.write()` overrides `BaseReader.write()` by first setting `self.header.start_line = None` and `self.data.start_line = None` (both `LatexData.start_line` and `LatexHeader.start_line` are methods, and setting them to `None` avoids issues when the parent class processes them).\n2. It then delegates to `core.BaseReader.write()`.\n3. `LatexHeader.write()` constructs the table preamble: it auto-generates column alignment as `'c' * len(self.cols)` if not specified, outputs `\\begin{tabletype}[tablealign]`, then preamble, caption (wrapped in `\\caption{}`), `\\begin{tabular}{col_align}`, header_start lines, column names joined with `&`, an optional units row, and header_end lines.\n4. Units are obtained via `_get_units()` which calls `unit.to_string(format=\"latex_inline\")` on each column's unit. If this raises `AttributeError`, it falls back to using the unit value directly. The `latexdict['units']` dictionary then updates (overrides) auto-detected units.\n5. `LatexData.write()` appends data_start lines, then calls `core.BaseData.write()`. The `LatexSplitter.join()` method uses ` & ` as delimiter and appends ` \\\\` to each line. Finally, data_end, `\\end{tabular}`, tablefoot, and `\\end{tabletype}` are appended.\n\n**Reading Algorithm:**\n1. The `LatexInputter.process_lines()` strips whitespace from each line.\n2. The comment regex is constructed as `\"%|\\\\hline|\\\\vspace|\\\\tableline|\\\\toprule|\\\\midrule|\\\\bottomrule\"` (built from `ignore_latex_commands`), which filters out lines starting with `%` or any of those LaTeX commands.\n3. `LatexHeader.start_line()` uses `find_latex_line()` to locate `\\begin{tabular}` and returns that index + 1.\n4. `LatexData.start_line()` returns `header.start_line() + 1` (one line after column names) if no explicit `data_start` is set.\n5. `LatexData.end_line()` finds `\\end{tabular}` using `find_latex_line()`.\n6. `LatexSplitter.__call__()` first checks if the last data line ends with `\\\\` (after stripping LaTeX comments using `RE_COMMENT` which matches `%` not preceded by `\\`). If not, it appends `\\\\` to allow proper splitting.\n7. `LatexSplitter.process_line()` strips LaTeX comments (via `RE_COMMENT.split(line)[0]`), raises `InconsistentTableError` if the line doesn't end with `\\\\`, then removes the trailing `\\\\` using `str.removesuffix()`.\n8. `LatexSplitter.process_val()` strips whitespace and removes enclosing `{}` braces from individual cell values.\n\n**Key Design Details:**\n- The `self.latex` dictionary is a single shared object between the reader, header, and data instances (set via `self.header.latex = self.latex` and `self.data.latex = self.latex`).\n- `max_ndim = None` is set explicitly on the `Latex` class.\n- The `AASTex` subclass defaults `tabletype` to `'deluxetable'` and uses `\\tablehead{\\colhead{col1} & ...}` for headers and `\\startdata`/`\\enddata` for data boundaries.\n- The `LatexInline` unit formatter (in `astropy/units/format/latex.py`) defaults `fraction=False` and wraps the output in `$\\mathrm{...}$`.",
    "rubric": [
      "Explains that the latex dictionary is shared between the reader, header, and data instances via assignment (self.header.latex = self.latex and self.data.latex = self.latex)",
      "Describes writing flow: Latex.write() sets header.start_line and data.start_line to None, then delegates to core.BaseReader.write()",
      "Explains LatexHeader.write() generates column alignment as 'c' * len(cols) if not specified, outputs \\begin{tabletype}, preamble, caption, \\begin{tabular}{col_align}, header_start, column names joined with &, optional units row, and header_end",
      "Describes how _get_units() tries unit.to_string(format='latex_inline') and falls back to using the unit value directly on AttributeError, then latexdict['units'] updates/overrides those",
      "Explains LatexData.write() adds data_start lines, calls core.BaseData.write(), then appends data_end, \\end{tabular}, tablefoot, and \\end{tabletype}",
      "Describes reading: LatexHeader.start_line() uses find_latex_line() to locate \\begin{tabular} and returns index + 1",
      "Explains LatexData.start_line() returns header.start_line() + 1 if no explicit data_start is set",
      "Describes LatexSplitter: process_line() strips LaTeX comments via RE_COMMENT, checks for trailing \\\\, removes it with removesuffix; join() uses ' & ' delimiter and appends ' \\\\'",
      "Explains the comment regex is built from ignore_latex_commands as '%|\\hline|\\vspace|...' to filter out those LaTeX commands during reading",
      "Describes LatexSplitter.process_val() strips whitespace and removes enclosing {} braces from cell values"
    ],
    "key_files": [
      "astropy/io/ascii/latex.py",
      "astropy/io/ascii/core.py",
      "astropy/units/format/latex.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/latex.py] astropy.io.ascii.latex.Latex\nastropy.io.ascii.latex.Latex:\n  LaTeX format table.\n\n    This class implements some LaTeX specific commands.  Its main\n    purpose is to write out a table in a form that LaTeX can compile. It\n    is beyond the scope of this class to implement every possible LaTeX\n    command, instead the focus is to generate a syntactically valid\n    LaTeX tables.\n\n    This class can also read simple LaTeX tables (one line per table\n    row, no ``\\multicolumn`` or similar constructs), specifically, it\n    can read the tables that it writes.\n    When reading, it will look for the Latex commands to start and end tabular\n    data (``\\begin{tabular}`` and ``\\end{tabular}``). That means that\n    those lines have to be present in the input file; the benefit is that this\n    reader can be used on a LaTeX file with text, tables, and figures and it\n    will read the first valid table.\n\n    .. note:: **Units in LaTeX tables**\n\n        The LaTeX writer will output units in the table if they are present in the\n        column info::\n\n            >>> import io\n            >>> out = io.StringIO()\n            >>> import sys\n            >>> import astropy.units as u\n            >>> from astropy.table import Table\n            >>> t = Table({'v': [1, 2] * u.km/u.s, 'class': ['star', 'jet']})\n            >>> t.write(out, format='ascii.latex')\n            >>> print(out.getvalue())\n            \\begin{table}\n            \\begin{tabular}{cc}\n            v & class \\\\\n            $\\mathrm{km\\,s^{-1}}$ &  \\\\\n            1.0 & star \\\\\n            2.0 & jet \\\\\n            \\end{tabular}\n            \\end{table}\n\n        However, it will fail to read a table with units. There are so\n        many ways to write units in LaTeX (enclosed in parenthesis or square brackets,\n        as a separate row are as part of the column headers, using plain text, LaTeX\n        symbols etc. ) that it is not feasible to implement a\n        general reader for this. If you need to read a table with units, you can\n        skip reading the lines with units to just read the numerical values using the\n        ``data_start`` parameter to set the first line where numerical data values appear::\n\n            >>> Table.read(out.getvalue(), format='ascii.latex', data_start=4)\n            <Table length=2>\n               v    class\n            float64  str4\n            ------- -----\n                1.0  star\n                2.0   jet\n\n        Alternatively, you can write a custom reader using your knowledge of the exact\n        format of the units in that case, by extending this class.\n\n    Reading a LaTeX table, the following keywords are accepted:\n\n    **ignore_latex_commands** :\n        Lines starting with these LaTeX commands will be treated as comments (i.e. ignored).\n\n    When writing a LaTeX table, the some keywords can customize the\n    format.  Care has to be taken here, because python interprets ``\\\\``\n    in a string as an escape character.  In order to pass this to the\n    output either format your strings as raw strings with the ``r``\n    specifier or use a double ``\\\\\\\\``.\n\n    Examples::\n\n        caption = r'My table \\label{mytable}'\n        caption = 'My table \\\\\\\\label{mytable}'\n\n    **latexdict** : Dictionary of extra parameters for the LaTeX output\n\n        * tabletype : used for first and last line of table.\n            The default is ``\\\\begin{table}``.  The following would generate a table,\n            which spans the whole page in a two-column document::\n\n                ascii.write(data, sys.stdout, format=\"latex\",\n                            latexdict={'tabletype': 'table*'})\n\n            If ``None``, the table environment will be dropped, keeping only\n            the ``tabular`` environment.\n\n        * tablealign : positioning of table in text.\n            The default is not to specify a position preference in the text.\n            If, e.g. the alignment is ``ht``, then the LaTeX will be ``\\\\begin{table}[ht]``.\n\n        * col_align : Alignment of columns\n            If not present all columns will be centered.\n\n        * caption : Table caption (string or list of strings)\n            This will appear above the table as it is the standard in\n            many scientific publications.  If you prefer a caption below\n            the table, just write the full LaTeX command as\n            ``latexdict['tablefoot'] = r'\\caption{My table}'``\n\n        * preamble, header_start, header_end, data_start, data_end, tablefoot: Pure LaTeX\n            Each one can be a string or a list of strings. These strings\n            will be inserted into the table without any further\n            processing. See the examples below.\n\n        * units : dictionary of strings\n            Keys in this dictionary should be names of columns. If\n            present, a line in the LaTeX table directly below the column\n            names is added, which contains the values of the\n            dictionary. Example::\n\n              from astropy.io import ascii\n              data = {'name': ['bike', 'car'], 'mass': [75,1200], 'speed': [10, 130]}\n              ascii.write(data, format=\"latex\",\n                          latexdict={'units': {'mass': 'kg', 'speed': 'km/h'}})\n\n            If the column has no entry in the ``units`` dictionary, it defaults\n            to the **unit** attribute of the column. If this attribute is not\n            specified (i.e. it is None), the unit will be written as ``' '``.\n\n        Run the following code to see where each element of the\n        dictionary is inserted in the LaTeX table::\n\n            from astropy.io import ascii\n            data = {'cola': [1,2], 'colb': [3,4]}\n            ascii.write(data, format=\"latex\", latexdict=ascii.latex.latexdicts['template'])\n\n        Some table styles are predefined in the dictionary\n        ``ascii.latex.latexdicts``. The following generates in table in\n        style preferred by A&A and some other journals::\n\n            ascii.write(data, format=\"latex\", latexdict=ascii.latex.latexdicts['AA'])\n\n        As an example, this generates a table, which spans all columns\n        and is centered on the page::\n\n            ascii.write(data, format=\"latex\", col_align='|lr|',\n                        latexdict={'preamble': r'\\begin{center}',\n                                   'tablefoot': r'\\end{center}',\n                                   'tabletype': 'table*'})\n\n    **caption** : Set table caption\n        Shorthand for::\n\n            latexdict['caption'] = caption\n\n    **col_align** : Set the column alignment.\n        If not present this will be auto-generated for centered\n        columns. Shorthand for::\n\n            latexdict['col_align'] = col_align",
    "verification_verdict": "pass",
    "verification_issues": [
      "Many implementation details (line numbers, specific method internals, regex patterns, etc.) cannot be verified from the documentation alone, but they are plausible code-level details that don't contradict the documentation.",
      "The claim about units using `unit.to_string(format=\"latex_inline\")` producing `$\\mathrm{km\\,s^{-1}}$` is consistent with the documentation's example output.",
      "The description of `latexdict` keys (tabletype, tablealign, col_align, caption, preamble, header_start, header_end, data_start, data_end, tablefoot, units) matches the documentation.",
      "The claim about `AASTex` subclass using `deluxetable` and `\\tablehead{\\colhead{}}` is not mentioned in the documentation but is a plausible code-level detail.",
      "The claim about `max_ndim = None` and Jupyter notebook issue #11695 is not in the documentation but is a plausible implementation detail."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor doc leakage in rationale for max_ndim=None (Jupyter issue #11695) and some interpretive explanations.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_45",
    "repo": "astropy",
    "question": "What is the architecture of the Mesa reader class, including its components, metadata handling, and restart removal algorithm?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `Mesa` class in `astropy/io/ascii/mesa.py` is a reader that inherits from `core.BaseReader`. Here are the key architectural and implementation details:\n\n**Class hierarchy and registration:**\n- `Mesa` inherits from `core.BaseReader`. It has `_format_name = \"mesa\"` and `_description = \"MESA stellar evolution code output\"`.\n- `_io_registry_can_write = False` is set on the class.\n- The class does NOT define `_io_registry_suffix` or `_io_registry_format_aliases`.\n\n**Component classes:**\n- `MesaHeader` extends `core.BaseHeader` and completely overrides `get_cols()` \u2014 instead of using the parent's `process_lines()` + splitter mechanism, it directly indexes into the original lines list at position 5 (0-indexed) and calls `.split()` on that line. It raises `core.InconsistentTableError` if fewer than 6 lines exist.\n- `MesaData` extends `core.BaseData` with `start_line = 5`, `comment = None`, and `delimiter = None`.\n\n**Metadata handling:**\n- `MesaHeader.update_meta()` stores metadata in `meta.setdefault(\"table\", {})[\"header\"]` (key is `\"header\"`), which becomes accessible as `table.meta['header']` in the final Table.\n- Metadata values are type-converted: quoted strings have quotes stripped, otherwise integers are tried first via `int()`, then `float()`, with fallback to string.\n\n**Restart removal algorithm:**\n- `_remove_restart_rows` uses `np.minimum.accumulate(model_number[::-1])` to compute the running minimum going backwards through the model numbers.\n- Then `np.unique(smallest_seen[:], return_index=True)` is used to find the first index of each unique minimum value in the reversed array.\n- The kept indices are computed as `len(table) - reverse_index - 1` to transform back to original array indices.\n- This is only triggered when both `self.remove_restart_rows` is True AND `\"model_number\"` is in the table's column names.\n\n**Data flow:**\n- `Mesa.read()` calls `super().read(table)` which invokes `BaseReader.read()`. The result is then optionally filtered by `_remove_restart_rows`.\n- Integer columns like `model_number` get integer dtype (kind 'i'), while floating-point columns get float dtype (kind 'f').\n\n**Test data files:**\n- `history_mesa.data`: 25 models, 3 data columns (`model_number`, `num_zones`, `star_age`), with metadata including `version_number` and `compiler`.\n- `history_mesa_with_restarts.data`: 31 total rows that clean to 20 rows after restart removal (cleaned model numbers are `list(range(1, 11)) + list(range(12, 31, 2))`).\n- `profile_mesa.data`: Referenced in tests as a profile file with column name `zone` instead of `model_number`.",
    "rubric": [
      "Identifies that Mesa inherits from core.BaseReader with _format_name='mesa' and _io_registry_can_write=False",
      "Explains that MesaHeader extends core.BaseHeader and overrides get_cols() to directly index line 5 (0-indexed) and call .split(), raising InconsistentTableError if fewer than 6 lines exist",
      "Explains that MesaData extends core.BaseData with start_line=5, comment=None, and delimiter=None",
      "Describes metadata handling: update_meta stores metadata in meta['table']['header'] with type conversion (strip quotes for strings, try int then float with string fallback)",
      "Explains the restart removal algorithm: uses np.minimum.accumulate on reversed model_number, then np.unique with return_index=True, and transforms indices back with len(table) - reverse_index - 1",
      "Notes that restart removal is only triggered when remove_restart_rows is True AND 'model_number' is in column names",
      "Describes the read() flow: calls super().read(table) then optionally filters with _remove_restart_rows",
      "Mentions test data: history_mesa.data has 25 models/3 columns, history_mesa_with_restarts.data has 31 rows cleaned to 20"
    ],
    "key_files": [
      "astropy/io/ascii/mesa.py",
      "astropy/io/ascii/core.py",
      "astropy/io/ascii/tests/test_mesa.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/mesa.py] astropy.io.ascii.mesa.Mesa\nastropy.io.ascii.mesa.Mesa:\n  MESA stellar evolution code output format.\n\n    MESA (Modules for Experiments in Stellar Astrophysics) is a widely-used\n    one-dimensional stellar evolution code. This reader handles both history\n    files (evolution over time) and profile files (spatial profiles at a\n    single timestep). It does NOT handle model (.mod) files or output from\n    associated packages like GYRE, adipls, or Stella.\n\n    The MESA format consists of:\n\n    - Lines 0-2: Metadata section with column indices, names, and values\n    - Line 3: Blank line\n    - Line 4: Column indices for main data table\n    - Line 5: Column names for main data table\n    - Line 6+: Data rows (whitespace-delimited)\n\n    The metadata from lines 1-2 is stored in the table's ``meta['header']``\n    dictionary.\n\n    For history files, this reader automatically detects and removes restart\n    artifacts. When a MESA run is restarted from an earlier model, the history\n    file will contain duplicate model numbers. This reader removes the earlier\n    instances so that the resulting table has monotonically increasing model\n    numbers suitable for analysis and plotting.\n\n    Example::\n\n                                           1                    2\n                              version_number             compiler\n                                  \"r24.03.1\"           \"gfortran\"\n\n                                           1                    2                    3\n                                model_number            num_zones             star_age\n                                           1                 1004   1.00000000000E-05\n                                           2                 1025   2.20000000000E-05\n\n    Example usage::\n\n        >>> from astropy.table import Table\n        >>> from astropy.utils.data import get_pkg_data_filename\n        >>> # Find the location of an example file included in astropy\n        >>> mesa_file = get_pkg_data_filename('data/history_mesa.data',\n        ...                                      package='astropy.io.ascii.tests')\n        >>> t = Table.read(mesa_file, format='ascii.mesa')\n        >>> print(t.meta['header']['version_number'])\n        r24.03.1\n        >>> t['model_number', 'star_age'][:3]\n        <Table length=3>\n        model_number        star_age\n        int64            float64\n        ------------ ----------------------\n                1                  1e-05\n                2                2.2e-05\n                3 3.6400000000000004e-05\n\nastropy.io.ascii.mesa.Mesa.__init__:\n  Initialize MESA reader.\n\n        Parameters\n        ----------\n        remove_restart_rows : bool, optional\n            If True (default), automatically remove restart artifacts from history\n            files by detecting non-monotonic model numbers. Set to False to keep\n            all rows including restart artifacts.\n\nastropy.io.ascii.mesa.Mesa.read:\n  Read input data into a Table and return the result.\n\n        Parameters\n        ----------\n        table : str, file-like, list\n            Input table data\n\n        Returns\n        -------\n        out : `~astropy.table.Table`\n            Output table with restart artifacts removed (if applicable)\n\nastropy.io.ascii.mesa.Mesa._remove_restart_rows:\n  Remove rows from history files that correspond to restarts.\n\n        When MESA restarts from an earlier model, the history file contains\n        duplicate model numbers. We iterate backwards through the model_number\n        column and remove rows where the model number is >= the minimum seen\n        so far (going backwards). This identifies the \"old\" evolution that was\n        restarted.\n\n        Example:\n            Model numbers: [1, 2, 3, 4, 5, 3, 4, 5, 6, 7]\n            Going backwards:\n            - i=9: model=7, min_seen=7, keep\n            - i=8: model=6, min_seen=6, keep\n            - i=7: model=5, min_seen=5, keep\n            - i=6: model=4, min_seen=4, keep\n            - i=5: model=3, min_seen=3, keep\n            - i=4: model=5, 5 >= 3, REMOVE\n            - i=3: model=4, 4 >= 3, REMOVE\n            - i=2: model=3, 3 >= 3, REMOVE\n            - i=1: model=2, 2 < 3, min_seen=2, keep\n            - i=0: model=1, 1 < 2, min_seen=1, keep\n            Result: [1, 2, 3, 4, 5, 6, 7]\n\n        Parameters\n        ----------\n        table : `~astropy.table.Table`\n            Input table with potential restart artifacts\n\n        Returns\n        -------\n        table : `~astropy.table.Table`\n            Table with restart artifacts removed",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states metadata is stored in `meta['header']` dictionary, while the answer describes a more complex path through `meta.setdefault('table', {})['header']` which eventually becomes `table.meta['header']`. This is a plausible implementation detail but not directly verifiable from the doc.",
      "The answer claims `MesaHeader` indexes into position 5 (0-indexed) for column names - the documentation says Line 5 is 'Column names for main data table' which is consistent (0-indexed).",
      "The restart removal algorithm description in the answer uses `np.minimum.accumulate` and `np.unique` - the documentation describes the algorithm conceptually (iterating backwards) but doesn't specify the numpy implementation. This is plausible but unverifiable from the doc alone.",
      "Test data file details (25 models, 31 rows cleaning to 20, specific column names like 'logT', 'logRho', 11 metadata fields, 5 metadata fields) are not mentioned in the documentation and cannot be verified - these are plausible but potentially hallucinated specifics.",
      "The claim about `_io_registry_can_write = False` and `MetaBaseReader` metaclass are implementation details not in the documentation - plausible but unverifiable.",
      "The answer states `MesaData` has `start_line = 5` but the documentation says Line 6+ are data rows (0-indexed line 6), suggesting `start_line` should be 6. However, if `start_line` is 0-indexed and refers to the data section starting point after headers are processed, this could be confusing but is a minor discrepancy."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor leakage around metaclass registration behavior, BaseReader.read() internals (truncated), and data file specifics not shown in code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_46",
    "repo": "astropy",
    "question": "Why does the QDP line-type classification recompile its regex on every call instead of caching it, and how does this affect performance?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `_get_type_from_list_of_lines` function performs a two-pass approach: first classifying ALL lines and validating column consistency (the pre-processing pass), then the results are consumed by `_get_tables_from_qdp_file` in a second pass. This design choice has a notable performance implication:\n\n1. **Regex recompilation on every line**: The `_line_type` function (called once per line via a list comprehension) recompiles a complex regex pattern (`re.compile(_type_re, re.IGNORECASE)`) inside the function body on every single invocation. The regex is NOT cached or pre-compiled at module level, and no `@functools.lru_cache` or similar caching mechanism is used. This means for a file with N lines, the regex is compiled N times.\n\n2. **Column validation before data parsing**: The function validates that all data lines have the same number of columns BEFORE any actual data parsing happens. `_get_tables_from_qdp_file` (the only caller) uses the returned `ncol` value to determine column names via `_interpret_err_lines(err_specs, ncol, names=input_colnames)` before processing any data rows.\n\n3. **The ncol extraction uses string slicing (`int(type_[5:])`)**: Rather than re-parsing lines or re-splitting them, the function encodes the column count in the type string itself (e.g., `'data,3'`) and extracts it with `int(type_[5:])`.\n\n4. **\"new\" lines (table separators like \"NO NO NO NO NO\") are excluded from column count validation**: The column count check only examines types starting with `\"data,\"`, not `\"new\"` type lines. In the code, `_line_type` classifies lines matching the `NO NO...` pattern as `\"new\"` rather than as data, so they are skipped in the consistency check loop that only tests `type_.startswith(\"data,\")`.",
    "rubric": [
      "Explains that _line_type recompiles the regex (re.compile(_type_re, re.IGNORECASE)) inside the function body on every invocation, meaning it's compiled N times for N lines",
      "Notes that no caching mechanism (module-level pre-compilation, lru_cache, etc.) is used to avoid repeated compilation",
      "Mentions that _get_type_from_list_of_lines calls _line_type via list comprehension for every line in the file",
      "Explains that the column count is encoded in the type string (e.g., 'data,3') and extracted with int(type_[5:]) to avoid re-parsing lines",
      "Notes that 'new' type lines (NO NO NO patterns) are excluded from the column count consistency validation"
    ],
    "key_files": [
      "astropy/io/ascii/qdp.py",
      "astropy/io/ascii/tests/test_qdp.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/qdp.py] astropy.io.ascii.qdp._get_type_from_list_of_lines\nRead through the list of QDP file lines and label each line by type.\n\n    Parameters\n    ----------\n    lines : list\n        List containing one file line in each entry\n\n    Returns\n    -------\n    contents : list\n        List containing the type for each line (see `line_type_and_data`)\n    ncol : int\n        The number of columns in the data lines. Must be the same throughout\n        the file\n\n    Examples\n    --------\n    >>> line0 = \"! A comment\"\n    >>> line1 = \"543 12 456.0\"\n    >>> lines = [line0, line1]\n    >>> types, ncol = _get_type_from_list_of_lines(lines)\n    >>> types[0]\n    'comment'\n    >>> types[1]\n    'data,3'\n    >>> ncol\n    3\n    >>> lines.append(\"23\")\n    >>> _get_type_from_list_of_lines(lines)\n    Traceback (most recent call last):\n        ...\n    ValueError: Inconsistent number of columns",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the function's interface (parameters, returns, examples). The generated answer makes extensive claims about internal implementation details (regex recompilation, string slicing with int(type_[5:]), two-pass approach, line numbers, _line_type function, etc.) that cannot be verified from the provided documentation alone.",
      "The claim about 'new' lines and 'NO NO NO NO NO' table separators is not mentioned anywhere in the provided documentation chunk.",
      "The claim about _get_tables_from_qdp_file being the 'only caller' at 'line 262' cannot be verified from the documentation.",
      "The claim about regex recompilation at 'line 74' and list comprehension at 'line 125' cannot be verified from the documentation.",
      "However, the basic behavior described (classifying lines, returning types and ncol, raising ValueError for inconsistent columns) is consistent with the documentation's examples."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Some design rationale claims ('deliberate', 'intentional', 'prioritizes correctness over performance', 'acceptable for small files') are not verifiable from stripped code, but the core structural and behavioral claims are all verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_47",
    "repo": "astropy",
    "question": "Where is `_interpret_err_lines` defined and called from, and how does the data flow around it in the QDP reader?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `_interpret_err_lines` function is defined at line 152 in `astropy/io/ascii/qdp.py`. It is called from exactly one location: line 305 within `_get_tables_from_qdp_file()` in the same file. The call passes the `err_specs` dictionary (built from parsing `READ SERR` and `READ TERR` command lines), the `ncol` value (determined by `_get_type_from_list_of_lines()`), and the optional `input_colnames` parameter.\n\nThe function's only caller (`_get_tables_from_qdp_file`) constructs the `err_specs` dict by parsing command lines: `err_specs[command[1].lower()] = [int(c) for c in command[2:]]` (line 303). This means the dict keys are always lowercase ('serr' or 'terr').\n\nThe inverse function `_understand_err_col` (line 348) converts column names back to serr/terr specifications, used in `_write_table_qdp` (line 464) when no explicit `err_specs` is provided.\n\nThe `QDP` reader class (line 532) that ultimately invokes this flow has `_format_name = 'qdp'`, `_io_registry_suffix = '.qdp'`, and `_io_registry_can_write = True`. It is registered into the `FORMAT_CLASSES` dict and io handlers with the format name `'ascii.qdp'` (as seen in test usage).\n\nThe function uses `copy.deepcopy(err_specs)` (line 202) and then extracts keys via `.pop()`, avoiding mutation of the original dictionary. The function ends with an assertion `assert not np.any([c == '' for c in colnames])` (line 235) to verify all column slots were filled.",
    "rubric": [
      "_interpret_err_lines is defined in astropy/io/ascii/qdp.py (around line 152)",
      "It is called from _get_tables_from_qdp_file in the same file (around line 305)",
      "The err_specs dict passed to it is built by parsing READ SERR/TERR command lines with keys lowercased ('serr' or 'terr')",
      "The function uses copy.deepcopy(err_specs) to avoid mutating the original dictionary",
      "It ends with an assertion that all column name slots are filled (no empty strings)",
      "The inverse function _understand_err_col converts column names back to serr/terr specifications for writing",
      "The QDP class has _format_name = 'qdp' and is registered as 'ascii.qdp'"
    ],
    "key_files": [
      "astropy/io/ascii/qdp.py",
      "astropy/io/ascii/core.py",
      "astropy/io/ascii/tests/test_qdp.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/qdp.py] astropy.io.ascii.qdp._interpret_err_lines\nGive list of column names from the READ SERR and TERR commands.\n\n    Parameters\n    ----------\n    err_specs : dict\n        ``{'serr': [n0, n1, ...], 'terr': [n2, n3, ...]}``\n        Error specifications for symmetric and two-sided errors\n    ncols : int\n        Number of data columns\n\n    Other Parameters\n    ----------------\n    names : list of str\n        Name of data columns (defaults to ['col1', 'col2', ...]), _not_\n        including error columns.\n\n    Returns\n    -------\n    colnames : list\n        List containing the column names. Error columns will have the name\n        of the main column plus ``_err`` for symmetric errors, and ``_perr``\n        and ``_nerr`` for positive and negative errors respectively\n\n    Examples\n    --------\n    >>> col_in = ['MJD', 'Rate']\n    >>> cols = _interpret_err_lines(None, 2, names=col_in)\n    >>> cols[0]\n    'MJD'\n    >>> err_specs = {'terr': [1], 'serr': [2]}\n    >>> ncols = 5\n    >>> cols = _interpret_err_lines(err_specs, ncols, names=col_in)\n    >>> cols[0]\n    'MJD'\n    >>> cols[2]\n    'MJD_nerr'\n    >>> cols[4]\n    'Rate_err'\n    >>> _interpret_err_lines(err_specs, 6, names=col_in)\n    Traceback (most recent call last):\n        ...\n    ValueError: Inconsistent number of input colnames",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific claims about line numbers, internal implementation details, and other functions that cannot be verified from the provided documentation alone. These are plausible but unverifiable from the given doc chunk.",
      "The claim about `copy.deepcopy(err_specs)` at line 202 and `.pop()` usage is unverifiable from the documentation but plausible.",
      "The claim about the assertion at line 235 is unverifiable from the documentation but plausible.",
      "The answer correctly reflects the documented behavior of the function (column naming with _err, _perr, _nerr suffixes) and the examples shown in the docstring.",
      "The answer's description of err_specs dict structure is consistent with the documentation's parameter description."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for metaclass registration details and deepcopy rationale phrasing.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "astropy_gen_48",
    "repo": "astropy",
    "question": "How does the RST reader/writer in astropy.io.ascii produce and parse the triple-separator-line table format?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The RST format in astropy.io.ascii is implemented through a class hierarchy: `RST` extends `FixedWidth` (which extends `Basic` \u2192 `BaseReader`), with custom header (`SimpleRSTHeader`) and data (`SimpleRSTData`) classes.\n\n**How RST writing works:**\n\n1. `RST.__init__` calls `FixedWidth.__init__(delimiter_pad=None, bookend=False, header_rows=header_rows)`, which configures the data splitter with no padding and no bookend characters. If `header_rows` is None, it defaults to `['name']`.\n\n2. `RST.write(lines)` calls `super().write(lines)` which goes to `BaseReader.write()`. This calls `self.write_header(lines, table.meta)` (which is a no-op for `FixedWidthHeader.write()` since it just does `pass`) and then `self.data.write(lines)` (i.e., `FixedWidthData.write`).\n\n3. `FixedWidthData.write()` computes column widths from both header and data values, right-justifies all values using `FixedWidthSplitter.join()`, and appends: (a) header row(s) like column names, (b) a separator line of `=` characters (because `SimpleRSTHeader.position_line = 0` is not None and `position_char = '='`), then (c) data lines.\n\n4. Back in `RST.write()`, `idx = len(self.header.header_rows)` points to the separator line index. It returns `[lines[idx]] + lines + [lines[idx]]` \u2014 prepending and appending the `===` separator line to create the full 3-separator RST table.\n\n**How RST reading works:**\n\n1. `RST.read(table)` sets `self.data.start_line = 2 + len(self.header.header_rows)` to skip past the opening separator, header row(s), and the second separator.\n\n2. `SimpleRSTHeader` uses `position_line = 0` (first line of `=` chars) to determine column positions, `start_line = 1` to find column names, and `splitter_class = DefaultSplitter` (which splits on whitespace using Python's csv module).\n\n3. `SimpleRSTHeader.get_fixedwidth_params()` overrides the parent to set `ends[-1] = None`, making the rightmost column unbounded.\n\n4. `SimpleRSTData` sets `end_line = -1` and uses `FixedWidthTwoLineDataSplitter` (delimiter = `\" \"`) for data splitting.\n\n**Key implementation details:**\n- The `FixedWidthSplitter.join()` method right-justifies values: `\" \" * (width - len(val)) + val`\n- With `delimiter_pad=None` (falsy) and `bookend=False`, columns are separated by a single space (from `FixedWidthTwoLineDataSplitter.delimiter = \" \"`)\n- The RST format has `_format_name = \"rst\"` and is registered via `MetaBaseReader` metaclass into the `FORMAT_CLASSES` dict\n- `FixedWidthHeader.write()` is a no-op (`pass`)",
    "rubric": [
      "Explains that RST extends FixedWidth, which is initialized with delimiter_pad=None, bookend=False, and default header_rows=['name']",
      "Describes that RST.write() calls super().write() (BaseReader.write), which calls FixedWidthHeader.write() (a no-op/pass) then FixedWidthData.write()",
      "Explains that FixedWidthData.write() computes column widths, right-justifies values via FixedWidthSplitter.join(), and appends header rows, a separator line of '=' characters (because position_line=0 and position_char='='), then data lines",
      "Describes how RST.write() then wraps the output by prepending and appending the separator line: [lines[idx]] + lines + [lines[idx]], where idx = len(header_rows)",
      "Explains that RST.read() sets data.start_line = 2 + len(header_rows) to skip past the opening separator, header row(s), and second separator",
      "Mentions that SimpleRSTHeader uses position_line=0 to determine column positions from the first '=' line and start_line=1 for column names",
      "Explains that SimpleRSTHeader.get_fixedwidth_params() sets ends[-1] = None to make the rightmost column unbounded",
      "Mentions that SimpleRSTData sets end_line=-1 to exclude the trailing separator line and uses FixedWidthTwoLineDataSplitter"
    ],
    "key_files": [
      "astropy/io/ascii/rst.py",
      "astropy/io/ascii/fixedwidth.py",
      "astropy/io/ascii/core.py"
    ],
    "source_doc": "[docstring: astropy/io/ascii/rst.py] astropy.io.ascii.rst.RST\nastropy.io.ascii.rst.RST:\n  reStructuredText simple format table.\n\n    See: https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#simple-tables\n\n    Example::\n\n      >>> from astropy.table import QTable\n      >>> import astropy.units as u\n      >>> import sys\n      >>> tbl = QTable({\"wave\": [350, 950] * u.nm, \"response\": [0.7, 1.2] * u.count})\n      >>> tbl.write(sys.stdout,  format=\"ascii.rst\")\n      ===== ========\n       wave response\n      ===== ========\n      350.0      0.7\n      950.0      1.2\n      ===== ========\n\n    Like other fixed-width formats, when writing a table you can provide ``header_rows``\n    to specify a list of table rows to output as the header.  For example::\n\n      >>> tbl.write(sys.stdout,  format=\"ascii.rst\", header_rows=['name', 'unit'])\n      ===== ========\n       wave response\n         nm       ct\n      ===== ========\n      350.0      0.7\n      950.0      1.2\n      ===== ========\n\n    Currently there is no support for reading tables which utilize continuation lines,\n    or for ones which define column spans through the use of an additional\n    line of dashes in the header.\n\nastropy.io.ascii.rst.RST.write:\n  Write the table data in RST format including header and footer separator lines.\n\n        Parameters\n        ----------\n        lines : list of str\n            The formatted table lines to be written.\n\n        Returns\n        -------\n        list[str]\n            The complete RST table with header separator, table content, and\n            footer separator line.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rationale/purpose statements were trimmed from a few claims.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_02",
    "repo": "sphinx",
    "question": "Why does NumpyDocstring inherit from GoogleDocstring and what design patterns allow the subclass to customize parsing behavior?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `NumpyDocstring` class inherits from `GoogleDocstring` and overrides a targeted set of methods to handle the structural differences between NumPy-style and Google-style docstrings. Both styles share the same fundamental parsing algorithm \u2014 a `_parse()` loop that detects section headers, dispatches to per-section parsers, and accumulates reStructuredText output \u2014 with only the syntactic conventions differing (underline-based vs. colon-based headers, colon-separated vs. parenthesized types).\n\nKey design decisions revealed by reading the code:\n\n1. **`hasattr` guard pattern for subclass customization**: In `GoogleDocstring.__init__`, there are `if not hasattr(self, '_directive_sections')` and `if not hasattr(self, '_sections')` guards. This is because `NumpyDocstring.__init__` sets `self._directive_sections = ['.. index::']` *before* calling `super().__init__()`. The `hasattr` check prevents the parent from overwriting the child's value with an empty list. This allows subclasses to pre-configure attributes before the parent's initialization logic runs.\n\n2. **Numpy sections use underline-based detection (`_numpy_section_regex`) while Google sections use colon-based detection (`_google_section_regex`)**. The `_is_section_header()` override in `NumpyDocstring` checks that the next line after the section name matches the regex `r'^[=\\-`:'\\\"~^_*+#<>]{2,}\\s*$'` (a line of at least 2 punctuation characters). Google's version checks that the section name ends with a colon and that subsequent content is more indented.\n\n3. **`_is_section_break()` handles double blank lines differently**: `NumpyDocstring._is_section_break()` adds an extra termination condition: `(not line1 and not line2)` \u2014 two consecutive blank/empty lines end a section, unlike Google-style where only indentation changes matter.\n\n4. **`_consume_field` splits type differently**: Google's `_consume_field` uses `_google_typed_arg_regex` (pattern `(.+?)\\(\\s*(.*[^\\s]+)\\s*\\)`) to extract types from parenthesized notation like `arg(int)`, while NumPy's override uses `_partition_field_on_colon` to split `name : type` on the colon. Additionally, NumpyDocstring's `_consume_field` calls `self._lookup_annotation(_name)` when no type is found in the docstring, while GoogleDocstring's version does not have this automatic annotation lookup in `_consume_field`.\n\n5. **`_consume_returns_section` is fundamentally different**: Google's version reads all content to the next section and parses a single `type: description` pair. NumPy's version simply delegates to `_consume_fields(prefer_type=True)`, which allows multiple individually typed return values.\n\n6. **`_parse_see_also_section` has a fallback**: NumpyDocstring's `_parse_see_also_section` first tries the specialized `_parse_numpydoc_see_also_section` parser, and if it raises `ValueError`, falls back to a generic `_format_admonition('seealso', lines)`. GoogleDocstring simply delegates to `_parse_admonition('seealso', section)` without any special parsing. The specialized parser generates `:py:obj:` cross-references by default but uses the specified role if one is provided.\n\n7. **`_escape_args_and_kwargs` handles comma-separated names**: NumpyDocstring overrides this method to split on `', '` and apply the parent's escaping to each name individually.\n\n8. **Processing order in `_process_docstring`**: NumPy docstring processing runs *before* Google processing, so NumPy-style sections are converted to reST first, and then the result is passed through Google parsing. Both are enabled by default via `napoleon_numpy_docstring=True` and `napoleon_google_docstring=True`.\n\n9. **`_consume_section_header` consumes the underline**: NumpyDocstring's override consumes two lines (the section title and its underline), while GoogleDocstring's version only consumes the colon-terminated header line and strips the trailing colon.",
    "rubric": [
      "Explains that NumpyDocstring inherits from GoogleDocstring because both share the same fundamental parsing algorithm (_parse() loop) with only syntactic conventions differing",
      "Describes the hasattr guard pattern in GoogleDocstring.__init__ that prevents overwriting _directive_sections and _sections pre-configured by NumpyDocstring before calling super().__init__()",
      "Explains that NumpyDocstring uses underline-based section detection (_numpy_section_regex matching punctuation lines) while GoogleDocstring uses colon-based detection",
      "Notes that NumpyDocstring's _is_section_break adds a double blank line termination condition (not line1 and not line2)",
      "Explains that _consume_field differs: Google uses parenthesized type parsing while NumPy uses colon-based partitioning and includes automatic annotation lookup",
      "Notes that NumpyDocstring's _consume_returns_section delegates to _consume_fields(prefer_type=True) allowing multiple return values, unlike Google's single-pair approach",
      "Mentions that _parse_see_also_section in NumpyDocstring uses a try/except fallback pattern with _parse_numpydoc_see_also_section",
      "Notes that _consume_section_header in NumpyDocstring consumes both the title and underline (two lines) vs Google consuming only the colon-terminated header"
    ],
    "key_files": [
      "sphinx/ext/napoleon/docstring.py",
      "sphinx/ext/napoleon/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/napoleon/docstring.py] sphinx.ext.napoleon.docstring.NumpyDocstring\nsphinx.ext.napoleon.docstring.NumpyDocstring:\n  Convert NumPy style docstrings to reStructuredText.\n\n    Parameters\n    ----------\n    docstring : :obj:`str` or :obj:`list` of :obj:`str`\n        The docstring to parse, given either as a string or split into\n        individual lines.\n    config: :obj:`sphinx.ext.napoleon.Config` or :obj:`sphinx.config.Config`\n        The configuration settings to use. If not given, defaults to the\n        config object on `app`; or if `app` is not given defaults to the\n        a new :class:`sphinx.ext.napoleon.Config` object.\n\n\n    Other Parameters\n    ----------------\n    app : :class:`sphinx.application.Sphinx`, optional\n        Application object representing the Sphinx process.\n    what : :obj:`str`, optional\n        A string specifying the type of the object to which the docstring\n        belongs. Valid values: \"module\", \"class\", \"exception\", \"function\",\n        \"method\", \"attribute\".\n    name : :obj:`str`, optional\n        The fully qualified name of the object.\n    obj : module, class, exception, function, method, or attribute\n        The object to which the docstring belongs.\n    options : :class:`sphinx.ext.autodoc.Options`, optional\n        The options given to the directive: an object with attributes\n        inherited_members, undoc_members, show_inheritance and no_index that\n        are True if the flag option of same name was given to the auto\n        directive.\n\n\n    Example\n    -------\n    >>> from sphinx.ext.napoleon import Config\n    >>> config = Config(napoleon_use_param=True, napoleon_use_rtype=True)\n    >>> docstring = '''One line summary.\n    ...\n    ... Extended description.\n    ...\n    ... Parameters\n    ... ----------\n    ... arg1 : int\n    ...     Description of `arg1`\n    ... arg2 : str\n    ...     Description of `arg2`\n    ... Returns\n    ... -------\n    ... str\n    ...     Description of return value.\n    ... '''\n    >>> print(NumpyDocstring(docstring, config))\n    One line summary.\n    <BLANKLINE>\n    Extended description.\n    <BLANKLINE>\n    :param arg1: Description of `arg1`\n    :type arg1: int\n    :param arg2: Description of `arg2`\n    :type arg2: str\n    <BLANKLINE>\n    :returns: Description of return value.\n    :rtype: str\n    <BLANKLINE>\n\n    Methods\n    -------\n    __str__()\n        Return the parsed docstring in reStructuredText format.\n\n        Returns\n        -------\n        str\n            UTF-8 encoded version of the docstring.\n\n    __unicode__()\n        Return the parsed docstring in reStructuredText format.\n\n        Returns\n        -------\n        unicode\n            Unicode version of the docstring.\n\n    lines()\n        Return the parsed lines of the docstring in reStructuredText format.\n\n        Returns\n        -------\n        list(str)\n            The lines of the docstring in a list.\n\nsphinx.ext.napoleon.docstring.NumpyDocstring._parse_numpydoc_see_also_section:\n  See Also\n        --------\n        func_name : Descriptive text\n            continued text\n        another_func_name : Descriptive text\n        func_name1, func_name2, :meth:`func_name`, func_name3\n\n        Licence\n        -------\n\n        Derived from the NumpyDoc implementation of ``_parse_see_also``,\n        which was under the following licence:\n\n            Copyright (C) 2008 Stefan van der Walt <stefan@mentat.za.net>,\n                               Pauli Virtanen <pav@iki.fi>\n\n            Redistribution and use in source and binary forms, with or without\n            modification, are permitted provided that the following conditions are\n            met:\n\n             1. Redistributions of source code must retain the above copyright\n                notice, this list of conditions and the following disclaimer.\n             2. Redistributions in binary form must reproduce the above copyright\n                notice, this list of conditions and the following disclaimer in\n                the documentation and/or other materials provided with the\n                distribution.\n\n            THIS SOFTWARE IS PROVIDED BY THE AUTHOR \"AS IS\" AND ANY EXPRESS OR\n            IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\n            WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\n            DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,\n            INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n            (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n            SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n            HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\n            STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING\n            IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\n            POSSIBILITY OF SUCH DAMAGE.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific regex patterns, method behaviors) that go far beyond what the provided documentation describes. These claims cannot be verified from the given documentation chunk alone.",
      "The documentation only describes the public interface (parameters, methods like __str__, __unicode__, lines()), the example usage, and the _parse_numpydoc_see_also_section license/format. Most of the answer's claims about internal implementation are extrapolations that cannot be confirmed or denied from this documentation.",
      "The claim about NumpyDocstring inheriting from GoogleDocstring is not stated in the provided documentation, though it is plausible given the code structure.",
      "The claim about processing order (NumPy before Google) and both being enabled by default cannot be verified from this documentation chunk.",
      "The answer's claim about _parse_numpydoc_see_also_section generating ':py:obj:' cross-references is not mentioned in the documentation; only the format 'func_name : Descriptive text' and the BSD license are shown.",
      "The claim about ValueError fallback in _parse_see_also_section is not supported by the provided documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from code structure; minor doc-leakage in rationale descriptions and provenance attributions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_01",
    "repo": "sphinx",
    "question": "What is the architecture of the GoogleDocstring parser class, including its class hierarchy, data structures, section registry, and how it integrates with the Sphinx autodoc pipeline?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `GoogleDocstring` class in `sphinx/ext/napoleon/docstring.py` is the core parser class for processing docstrings. Here is its architecture and key implementation details:\n\n**Class Hierarchy:**\n- `GoogleDocstring` is the base class\n- `NumpyDocstring` extends `GoogleDocstring`, as evidenced by imports in `__init__.py`\n\n**Internal Data Structure:**\n- Input lines are stored in a `Deque` instance, which is a custom subclass of `collections.deque` that adds a `get(n)` method returning a class-level `sentinel` object when index exceeds size, and a `next()` method that calls `popleft()` or raises `StopIteration`\n- Parsed output is accumulated in `self._parsed_lines: list[str]`\n\n**Section Registry:**\n- The `__init__` method builds a `_sections` dictionary mapping 34 section name variants (lowercased) to parsing method callables. For example, both `'args'` and `'arguments'` and `'parameters'` all map to `self._parse_parameters_section`. Admonition sections like 'attention', 'caution', 'danger', 'error', 'hint', 'important', 'note', 'tip', 'todo', 'warning', and 'warnings' use `functools.partial(self._parse_admonition, admonition_name)`. Notably, `'warnings'` maps to the admonition `'warning'` (not `'warnings'`).\n\n**Auto-detection of `what` parameter:**\n- When `what='object'` (the default), the constructor inspects the `obj` using `inspect.isclass()`, `inspect.ismodule()`, and `callable()` to auto-determine the object type.\n\n**Recursive Parsing:**\n- Both `_consume_field` and `_consume_inline_attribute` recursively create new instances of `self.__class__` (using `self.__class__(_descs, self._config).lines()`) to parse field descriptions. This means NumpyDocstring fields are parsed through NumpyDocstring instances recursively, preserving the correct subclass behavior.\n\n**Integration with Sphinx:**\n- The napoleon extension registers via `setup()` in `__init__.py`, connecting `_process_docstring` to the `'autodoc-process-docstring'` event and `_skip_member` to `'autodoc-skip-member'`\n- In `_process_docstring`, NumpyDocstring is processed **first**, then GoogleDocstring is processed on the result, meaning both styles can coexist\n- The extension also patches the Python domain via `_patch_python_domain()` to add 'keyword' as a recognized field type with `PyTypedField`\n\n**`_AutodocObjType`:**\n- Defined in `sphinx/ext/autodoc/_property_types.py` as a `Literal` type alias with values: `'module'`, `'class'`, `'exception'`, `'function'`, `'decorator'`, `'method'`, `'property'`, `'attribute'`, `'data'`, `'type'`. The docstring `what` parameter additionally allows `'object'` via union type.\n\n**Annotation Lookup:**\n- `sphinx.util.typing.get_type_hints()` wraps Python's `typing.get_type_hints()` with fallback to `__annotations__` on NameError, AttributeError, TypeError, or KeyError.\n\n**Type Preprocessing:**\n- When `napoleon_preprocess_types=True`, types are tokenized via `_tokenize_type_spec` using `_token_regex` (which splits on `, or `, ` or `, ` of `, `: `, ` to `, `, and `, ` and `, `, `, `{`, `}`, and quoted strings). Tokens classified as 'obj' type are wrapped in `:py:class:` roles via `_convert_type_spec_obj`, while singletons from the frozenset `{'None', 'True', 'False', 'Ellipsis', '...'}` get `:py:obj:` roles. The special case `'...'` is converted to `:py:obj:`... <Ellipsis>``.\n\n**`_no_index` handling:**\n- The constructor checks for both `options.no_index` and `options.noindex`, catching both `AttributeError` and `TypeError`.\n\n**Config class:**\n- The `Config` class in `__init__.py` defines 16 configuration values in `_config_values` (a sequence of tuples with name, default, rebuild strategy 'env', and allowed types). The constructor iterates these to set defaults, then applies any `**settings` overrides.",
    "rubric": [
      "Identifies that NumpyDocstring extends GoogleDocstring as a subclass",
      "Describes the Deque class as a custom subclass of collections.deque with get() (returns sentinel when out of bounds) and next() (calls popleft or raises StopIteration) methods",
      "Explains the _sections dictionary mapping ~34 lowercased section name variants to parsing method callables (e.g., 'args', 'arguments', 'parameters' all map to _parse_parameters_section)",
      "Mentions that admonition sections (attention, caution, danger, error, hint, important, note, tip, todo, warning, warnings) use functools.partial with _parse_admonition, and that 'warnings' maps to the admonition 'warning'",
      "Explains auto-detection of the what parameter: when what='object', the constructor uses inspect.isclass, inspect.ismodule, and callable to determine the object type",
      "Describes recursive parsing: _consume_field and _consume_inline_attribute create new instances using self.__class__(...) to parse field descriptions, preserving subclass behavior",
      "Explains Sphinx integration: setup() connects _process_docstring to 'autodoc-process-docstring' and _skip_member to 'autodoc-skip-member'",
      "Notes that in _process_docstring, NumpyDocstring is processed first, then GoogleDocstring is applied to the result",
      "Mentions that _patch_python_domain adds 'keyword' as a PyTypedField to the Python domain",
      "Describes type preprocessing: when napoleon_preprocess_types=True, types are tokenized via _tokenize_type_spec, and 'obj' tokens get :py:class: roles while SINGLETONS (None, True, False, Ellipsis, ...) get :py:obj: roles"
    ],
    "key_files": [
      "sphinx/ext/napoleon/docstring.py",
      "sphinx/ext/napoleon/__init__.py",
      "sphinx/ext/autodoc/_property_types.py",
      "sphinx/util/typing.py"
    ],
    "source_doc": "[docstring: sphinx/ext/napoleon/docstring.py] sphinx.ext.napoleon.docstring.GoogleDocstring\nsphinx.ext.napoleon.docstring.GoogleDocstring:\n  Convert Google style docstrings to reStructuredText.\n\n    Parameters\n    ----------\n    docstring : :obj:`str` or :obj:`list` of :obj:`str`\n        The docstring to parse, given either as a string or split into\n        individual lines.\n    config: :obj:`sphinx.ext.napoleon.Config` or :obj:`sphinx.config.Config`\n        The configuration settings to use. If not given, defaults to the\n        config object on `app`; or if `app` is not given defaults to the\n        a new :class:`sphinx.ext.napoleon.Config` object.\n\n\n    Other Parameters\n    ----------------\n    app : :class:`sphinx.application.Sphinx`, optional\n        Application object representing the Sphinx process.\n    what : :obj:`str`, optional\n        A string specifying the type of the object to which the docstring\n        belongs. Valid values: \"module\", \"class\", \"exception\", \"function\",\n        \"method\", \"attribute\".\n    name : :obj:`str`, optional\n        The fully qualified name of the object.\n    obj : module, class, exception, function, method, or attribute\n        The object to which the docstring belongs.\n    options : :class:`sphinx.ext.autodoc.Options`, optional\n        The options given to the directive: an object with attributes\n        inherited_members, undoc_members, show_inheritance and no_index that\n        are True if the flag option of same name was given to the auto\n        directive.\n\n\n    Example\n    -------\n    >>> from sphinx.ext.napoleon import Config\n    >>> config = Config(napoleon_use_param=True, napoleon_use_rtype=True)\n    >>> docstring = '''One line summary.\n    ...\n    ... Extended description.\n    ...\n    ... Args:\n    ...   arg1(int): Description of `arg1`\n    ...   arg2(str): Description of `arg2`\n    ... Returns:\n    ...   str: Description of return value.\n    ... '''\n    >>> print(GoogleDocstring(docstring, config))\n    One line summary.\n    <BLANKLINE>\n    Extended description.\n    <BLANKLINE>\n    :param arg1: Description of `arg1`\n    :type arg1: int\n    :param arg2: Description of `arg2`\n    :type arg2: str\n    <BLANKLINE>\n    :returns: Description of return value.\n    :rtype: str\n    <BLANKLINE>\n\nsphinx.ext.napoleon.docstring.GoogleDocstring.__str__:\n  Return the parsed docstring in reStructuredText format.\n\n        Returns\n        -------\n        unicode\n            Unicode version of the docstring.\n\nsphinx.ext.napoleon.docstring.GoogleDocstring.lines:\n  Return the parsed lines of the docstring in reStructuredText format.\n\n        Returns\n        -------\n        list(str)\n            The lines of the docstring in a list.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped code; only minor description of purpose and one claim about _lookup_annotation's namespace merging needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_04",
    "repo": "sphinx",
    "question": "How does Sphinx's search system build its index during compilation and then perform client-side full-text search at runtime?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The Sphinx search system operates on two distinct levels: (1) a Python-side `IndexBuilder` that builds the search index during documentation compilation, and (2) a JavaScript-side `Search` module (`searchtools.js`) that performs client-side full-text search at runtime.\n\n**How the search index is built (Python side):**\n\nThe `IndexBuilder` class in `sphinx/search/__init__.py` is instantiated by the HTML builder (`sphinx/builders/html/__init__.py`) during `prepare_writing()`. It accepts the build environment, language code, search options, and an optional custom scoring file. The constructor resolves the language by looking up a `languages` dict that maps language codes (17 supported languages including 'en', 'ja', 'zh', etc.) to `SearchLanguage` subclasses. If the language isn't found and contains an underscore (e.g., 'zh_CN'), it falls back to the base language code ('zh'). If still not found, it defaults to `SearchEnglish`.\n\nThe `feed()` method processes each document's doctree. It calls `self._word_collector(doctree)` to collect words into a `WordStore` dataclass with three fields: `words`, `titles`, and `title_words`. Words are then stemmed using a memoized (`functools.cache`) version of the language's `stem()` method (always lowercased). The `word_filter()` method filters out digits and stopwords. A key optimization: if a word already appears in the title mapping for a document, it is NOT added to the body mapping (avoiding duplicates).\n\nThe `WordCollector` class (a docutils `NodeVisitor`) skips nodes with 'no-search' in their classes.\n\n**How the index is serialized:**\n\nThe `freeze()` method creates the serialization dict with keys: `docnames`, `filenames`, `titles`, `terms`, `objects`, `objtypes`, `objnames`, `titleterms`, `envversion`, `alltitles`, and `indexentries`. In `get_terms()`, terms appearing in only one document are stored as a single integer (the doc index), while terms in multiple documents become a sorted list of integers.\n\nThe default serialization format for HTML builders is `_JavaScriptIndex`, which wraps the JSON data with `Search.setIndex(` prefix and `)` suffix, written to `searchindex.js`.\n\n**How client-side search works (JavaScript side):**\n\nThe `searchtools.js` `Search` module loads the index via `Search.setIndex()`. When a user submits a query (via the `q` GET parameter), `_parseQuery()` splits the query using `splitQuery()` (which splits on non-Unicode-letter/number/emoji characters), filters stopwords and digits, then stems each term using a JavaScript `Stemmer`. Terms prefixed with `-` are added to excluded terms.\n\n`_performSearch()` then performs four types of searches: (1) title substring matching in `alltitles`, (2) index directive entry matching in `indexentries`, (3) object name matching via `performObjectSearch()`, and (4) full-text term matching via `performTermsSearch()`. The scoring system assigns: title=15, partialTitle=7, term=5, partialTerm=2, objNameMatch=11, objPartialMatch=6 points. Partial matching (regex-based) is only done for words longer than 2 characters. Results are deduplicated and sorted by score then alphabetically.",
    "rubric": [
      "Explains that IndexBuilder (Python side) builds the search index during doc compilation, while searchtools.js (JavaScript side) performs client-side search at runtime",
      "Describes how IndexBuilder resolves the search language from a languages dict, with fallback to base language code (stripping after underscore) and ultimately defaulting to SearchEnglish",
      "Explains that the feed() method processes each document's doctree using a WordCollector (NodeVisitor) that collects words into categories (words, titles, title_words) and skips nodes with 'no-search' class",
      "Mentions that words are stemmed using a memoized/cached version of the language's stem() method and filtered to exclude digits and stopwords",
      "Notes the optimization that words already in the title mapping for a document are not added to the body mapping",
      "Describes the freeze()/get_terms() serialization: terms in one document stored as a single integer, terms in multiple documents as a sorted list of integers",
      "Explains the JavaScript index format: JSON wrapped with 'Search.setIndex(' prefix and ')' suffix, written to searchindex.js",
      "Describes client-side query parsing: splitting on non-letter/number characters, filtering stopwords/digits, stemming terms, and handling excluded terms (prefixed with '-')",
      "Lists the four search types performed client-side: title substring matching, index entry matching, object name matching, and full-text term matching",
      "Mentions the scoring system with approximate values (title=15, partialTitle=7, term=5, partialTerm=2, objNameMatch=11, objPartialMatch=6) and that results are deduplicated and sorted by score then alphabetically"
    ],
    "key_files": [
      "sphinx/search/__init__.py",
      "sphinx/builders/html/__init__.py",
      "sphinx/themes/basic/static/searchtools.js",
      "sphinx/search/en.py",
      "sphinx/themes/basic/searchbox.html"
    ],
    "source_doc": "[doc_file: doc/usage/advanced/websupport/quickstart.rst] Performing searches\nTo use the search form built-in to the Sphinx sidebar, create a function to\nhandle requests to the URL 'search' relative to the documentation root.  The\nuser's search query will be in the GET parameters, with the key ``q``.  Then use\nthe :meth:`~sphinxcontrib.websupport.WebSupport.get_search_results` method to\nretrieve search results. In `Flask <https://flask.palletsprojects.com/>`_ that\nwould be like this::\n\n   @app.route('/search')\n   def search():\n       q = request.args.get('q')\n       document = support.get_search_results(q)\n       return render_template('doc.html', document=document)\n\nNote that we used the same template to render our search results as we did to\nrender our documents.  That's because :meth:`~.WebSupport.get_search_results`\nreturns a context dict in the same format that :meth:`~.WebSupport.get_document`\ndoes.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer goes far beyond what the documentation covers, discussing internal implementation details (IndexBuilder, searchtools.js, scoring systems, etc.) that are not mentioned in the provided documentation chunk.",
      "The documentation only describes how to use get_search_results() in a Flask app with the websupport API, while the answer discusses extensive internals of Sphinx's search system.",
      "The answer's claim about get_search_results() being part of an external package 'not in the Sphinx core repo' is not stated in the documentation (the doc simply references sphinxcontrib.websupport.WebSupport.get_search_results).",
      "The documentation explicitly states that get_search_results() returns a context dict in the same format as get_document(), which the answer does not mention or address.",
      "The documentation states the search query is in GET parameters with key 'q', which the answer briefly mentions but buries among extensive unsupported implementation details."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only the websupport reference and .tmp file write strategy rely on documentation not present in the code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_03",
    "repo": "sphinx",
    "question": "Where does Napoleon's `_skip_member` function get invoked from, and how does its return value flow back to affect member inclusion?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `_skip_member` function in `sphinx/ext/napoleon/__init__.py` is registered as a handler for the `'autodoc-skip-member'` event via `app.connect('autodoc-skip-member', _skip_member)` within the `setup()` function. The data flows as follows:\n\n1. **Event emission source**: The `'autodoc-skip-member'` event is emitted using `events.emit_firstresult()` from within the member filtering logic in `sphinx/ext/autodoc/_dynamic/_member_finder.py`.\n\n2. **Event dispatching mechanism**: The `emit_firstresult` method in `sphinx/events.py` calls `self.emit()` which iterates over all connected listeners sorted by priority, and returns the first non-None result. If all handlers return None, it returns None.\n\n3. **Return value consumption**: When `_skip_member` returns `False` (don't skip), the member is included in documentation. When `_skip_member` returns `None`, the autodoc default decision is preserved.\n\n4. **Inside `_skip_member` control flow**:\n   - First checks if `name != '__weakref__'`, `obj` has a `__doc__` attribute, and `what` is in `{'class', 'exception', 'module'}` (the `is_member` check).\n   - For `'class'` and `'exception'` types, it determines class ownership by extracting `__qualname__`, partitioning on '.', and then looking up the owning class. For nested classes (with '.' in `cls_path`), it uses `functools.reduce(getattr, mod_path, mod)` after importing the module. For simple paths, it uses `inspect.unwrap(obj).__globals__[cls_path]`.\n   - For `'module'` type or when `cls_is_owner` is True, it classifies the member as `is_init` (name == `'__init__'`), `is_special` (dunder but not `__init__`), or `is_private` (starts with `_` but not special/init).\n   - Only returns `False` (include) if the corresponding config flag is True; otherwise returns `None` (defer to autodoc's default).\n\n5. **Default config values**: All three relevant config values default to `False` in the `Config._config_values` tuple: `napoleon_include_init_with_doc`, `napoleon_include_private_with_doc`, and `napoleon_include_special_with_doc`.\n\n6. **`inspect.unwrap`** calls through to `sphinx/util/inspect.py` which wraps Python's `inspect.unwrap()` with special handling for mock objects (checking `__sphinx_mock__` attribute) and catching `ValueError`.",
    "rubric": [
      "Identifies that _skip_member is connected to the 'autodoc-skip-member' event via app.connect in the setup() function",
      "Identifies that the event is emitted from sphinx/ext/autodoc's member filtering logic (e.g., _dynamic/_member_finder.py) using emit_firstresult",
      "Explains that emit_firstresult returns the first non-None result from connected handlers, and None means defer to autodoc's default",
      "Explains that _skip_member returns False to include a member or None to defer to default behavior",
      "Describes the is_member check: verifies name != '__weakref__', obj has __doc__, and what is in {'class', 'exception', 'module'}",
      "Explains how class ownership is determined using __qualname__ partitioning and getattr/globals lookup for class and exception types",
      "Describes classification into is_init, is_special, and is_private categories",
      "Notes that the relevant config values (napoleon_include_init_with_doc, napoleon_include_private_with_doc, napoleon_include_special_with_doc) default to False"
    ],
    "key_files": [
      "sphinx/ext/napoleon/__init__.py",
      "sphinx/ext/autodoc/_dynamic/_member_finder.py",
      "sphinx/events.py",
      "sphinx/util/inspect.py"
    ],
    "source_doc": "[docstring: sphinx/ext/napoleon/__init__.py] sphinx.ext.napoleon._skip_member\nDetermine if private and special class members are included in docs.\n\n    The following settings in conf.py determine if private and special class\n    members or init methods are included in the generated documentation:\n\n    * ``napoleon_include_init_with_doc`` --\n      include init methods if they have docstrings\n    * ``napoleon_include_private_with_doc`` --\n      include private members if they have docstrings\n    * ``napoleon_include_special_with_doc`` --\n      include special members if they have docstrings\n\n    Parameters\n    ----------\n    app : sphinx.application.Sphinx\n        Application object representing the Sphinx process\n    what : str\n        A string specifying the type of the object to which the member\n        belongs. Valid values: \"module\", \"class\", \"exception\", \"function\",\n        \"method\", \"attribute\".\n    name : str\n        The name of the member.\n    obj : module, class, exception, function, method, or attribute.\n        For example, if the member is the __init__ method of class A, then\n        `obj` will be `A.__init__`.\n    skip : bool\n        A boolean indicating if autodoc will skip this member if `_skip_member`\n        does not override the decision\n    options : sphinx.ext.autodoc.Options\n        The options given to the directive: an object with attributes\n        inherited_members, undoc_members, show_inheritance and no_index that\n        are True if the flag option of same name was given to the auto\n        directive.\n\n    Returns\n    -------\n    bool\n        True if the member should be skipped during creation of the docs,\n        False if it should be included in the docs.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, file paths, internal logic) that cannot be verified from the provided documentation alone. These are plausible extrapolations based on likely source code, but are not directly supported by the given doc chunk.",
      "The documentation states the function returns 'True if the member should be skipped during creation of the docs, False if it should be included in the docs.' The answer claims it returns False (include) or None (defer to autodoc's default). The documentation doesn't explicitly mention None as a return value, though this is a plausible implementation detail for Sphinx event handlers.",
      "The answer's description of the control flow and classification of members (is_init, is_special, is_private) is consistent with the documented config options but the specific implementation details cannot be verified from the doc alone."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments needed for truncated file references and line numbers.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_05",
    "repo": "sphinx",
    "question": "What is the architecture of Sphinx's C domain, including its directive class hierarchy, symbol storage, and ID generation scheme?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The C Domain in Sphinx is implemented primarily in the `sphinx/domains/c/` package, consisting of `__init__.py`, `_ast.py`, `_ids.py`, `_parser.py`, and `_symbol.py`. The domain is registered as `CDomain` with name 'c' and label 'C'.\n\n**Architecture:**\n- `CObject` is the base class for all C directive objects, inheriting from `ObjectDescription[ASTDeclaration]`. It provides common functionality for parsing signatures, managing symbols, and generating cross-reference targets.\n- Specific directive classes inherit from `CObject`: `CMemberObject` (used for both `c:member` and `c:var` directives), `CFunctionObject`, `CMacroObject`, `CStructObject`, `CUnionObject`, `CEnumObject`, `CEnumeratorObject`, and `CTypeObject`.\n- Both `CFunctionObject` and `CMacroObject` share the same `_function_doc_field_types` list, which defines the `param`/`parameter`/`arg`/`argument`, `retval`/`retvals`, `returns`/`return`, and `rtype` field types.\n\n**Key implementation details:**\n1. The `_max_id` value is 1 (in `_ids.py`), meaning only a single ID version is generated. The `_id_prefix` sequence is `('', 'c.', 'Cv2.')`, so the newest ID prefix is `'c.'`.\n2. The `CMemberObject` class uses `object_type = 'member'` but overrides `display_object_type` as a property that returns `self.objtype` (either 'member' or 'var').\n3. The `CNamespaceObject` recognizes 'NULL', '0', and 'nullptr' as values that reset the namespace to root scope.\n4. Function parameters are automatically added as child symbols of the function's symbol via `Symbol._add_function_params()`, creating `ASTDeclaration` objects with `objectType='functionParam'`. These are exposed as the `functionParam` ObjType in the domain.\n5. The `DefinitionParser` uses '.' (dot) as the separator for nested names in C (not '::' as in C++). A leading dot indicates a rooted name.\n6. The `c_extra_keywords` config defaults to `_macro_keywords` which includes: 'alignas', 'alignof', 'bool', 'complex', 'imaginary', 'noreturn', 'static_assert', 'thread_local'. These are treated as keywords during parsing and cannot be used as identifiers unless removed from the config.\n7. The `ASTIdentifier` class uses `sys.intern()` on the name string and identifies anonymous names by checking if the name starts with '@'.\n8. The `c:func` role is the only role in the C domain that uses `fix_parens=True`, which automatically adds '()' to the display title.\n9. The domain's `setup()` function returns `env_version: 3` and reports both `parallel_read_safe` and `parallel_write_safe` as True.\n10. The `TypedField` for parameters uses `typerolename='expr'`.\n11. The `retval` `GroupedField` uses `can_collapse=True`.\n12. The `Symbol` class stores children in three data structures: `_children_by_name` (dict), `_children_by_docname` (nested dict), and `_anon_children` (set) for different access patterns.",
    "rubric": [
      "Identifies that CDomain is the main domain class with name 'c' and label 'C', registered via setup()",
      "Explains that CObject is the base class inheriting from ObjectDescription[ASTDeclaration], with specific subclasses like CMemberObject, CFunctionObject, CMacroObject, CStructObject, CUnionObject, CEnumObject, CEnumeratorObject, and CTypeObject",
      "Notes that CMemberObject handles both 'member' and 'var' directives with a display_object_type property returning self.objtype",
      "Mentions that CFunctionObject and CMacroObject share the same _function_doc_field_types list containing param/parameter/arg/argument, retval/retvals, returns/return, and rtype fields",
      "Describes that _max_id is 1 and _id_prefix is ('', 'c.', 'Cv2.')",
      "Explains that the Symbol class stores children in _children_by_name (dict), _children_by_docname (nested dict), and _anon_children (set)",
      "Notes that function parameters are added as child symbols via _add_function_params() creating ASTDeclaration objects with objectType='functionParam'",
      "Mentions that CNamespaceObject recognizes 'NULL', '0', and 'nullptr' as reset-to-root values",
      "Identifies that c:func is the only role with fix_parens=True",
      "Notes that setup() returns env_version: 3 with parallel_read_safe and parallel_write_safe both True"
    ],
    "key_files": [
      "sphinx/domains/c/__init__.py",
      "sphinx/domains/c/_ast.py",
      "sphinx/domains/c/_ids.py",
      "sphinx/domains/c/_parser.py",
      "sphinx/domains/c/_symbol.py"
    ],
    "source_doc": "[doc_file: doc/usage/domains/c.rst] The C Domain\n.. versionadded:: 1.0\n\nThe C domain (name **c**) is suited for documentation of C API.\n\n.. rst:directive:: .. c:member:: declaration\n                   .. c:var:: declaration\n\n   Describes a C struct member or variable. Example signature::\n\n      .. c:member:: PyObject *PyTypeObject.tp_bases\n\n   The difference between the two directives is only cosmetic.\n\n.. rst:directive:: .. c:function:: function prototype\n\n   Describes a C function. The signature should be given as in C, e.g.::\n\n      .. c:function:: PyObject *PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)\n\n   Note that you don't have to backslash-escape asterisks in the signature, as\n   it is not parsed by the reStructuredText inliner.\n\n   In the description of a function you can use the following info fields\n   (see also :ref:`info-field-lists`).\n\n   * ``param``, ``parameter``, ``arg``, ``argument``,\n     Description of a parameter.\n   * ``type``: Type of a parameter,\n     written as if passed to the :rst:role:`c:expr` role.\n   * ``returns``, ``return``: Description of the return value.\n   * ``rtype``: Return type,\n     written as if passed to the :rst:role:`c:expr` role.\n   * ``retval``, ``retvals``: An alternative to ``returns`` for describing\n     the result of the function.\n\n   .. versionadded:: 4.3\n      The ``retval`` field type.\n\n   For example::\n\n      .. c:function:: PyObject *PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)\n\n         :param type: description of the first parameter.\n         :param nitems: description of the second parameter.\n         :returns: a result.\n         :retval NULL: under some conditions.\n         :retval NULL: under some other conditions as well.\n\n   which renders as\n\n   .. c:function:: PyObject *PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)\n\n      ..\n         ** for some editors (e.g., vim) to stop bold-highlighting the source\n\n      :no-contents-entry:\n      :no-index-entry:\n      :param type: description of the first parameter.\n      :param nitems: description of the second parameter.\n      :returns: a result.\n      :retval NULL: under some conditions.\n      :retval NULL: under some other conditions as well.\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the function's parameters will be emitted on a single logical\n      line, overriding :confval:`c_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n\n.. rst:directive:: .. c:macro:: name\n                   .. c:macro:: name(arg list)\n\n   Describes a C macro, i.e., a C-language ``#define``, without the replacement\n   text.\n\n   In the description of a macro you can use the same info fields as for the\n   :rst:dir:`c:function` directive.\n\n   .. versionadded:: 3.0\n      The function style variant.\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the macro's parameters will be emitted on a single logical\n      line, overriding :confval:`c_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n.. rst:directive:: .. c:struct:: name\n\n   Describes a C struct.\n\n   .. versionadded:: 3.0\n\n.. rst:directive:: .. c:union:: name\n\n   Describes a C union.\n\n   .. versionadded:: 3.0\n\n.. rst:directive:: .. c:enum:: name\n\n   Describes a C enum.\n\n   .. versionadded:: 3.0\n\n.. rst:directive:: .. c:enumerator:: name\n\n   Describes a C enumerator.\n\n   .. versionadded:: 3.0\n\n.. rst:directive:: .. c:type:: typedef-like declaration\n                   .. c:type:: name\n\n   Describes a C type, either as a typedef, or the alias for an unspecified\n   type.\n\n.. _c-xref-roles:",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor documentation-dependent explanations of behavior semantics were trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_07",
    "repo": "sphinx",
    "question": "Where is the JavaScript domain implemented, and how are its directives, roles, and cross-reference resolution organized?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The JavaScript domain implementation is located in `sphinx/domains/javascript.py`. The key class hierarchy is: `JSObject` (extends `ObjectDescription[tuple[str, str]]`) \u2192 `JSCallable` (adds `has_arguments = True` and doc field types) \u2192 `JSConstructor` (adds `allow_nesting = True` and displays 'class' prefix). The `JSModule` directive extends `SphinxDirective` directly (not `ObjectDescription`), and both the `js:function` and `js:method` directives map to the same class `JSCallable`.\n\nKey architectural details:\n\n1. The domain is registered via a `setup()` function that returns `env_version: 3` and declares both `parallel_read_safe` and `parallel_write_safe` as `True`.\n\n2. Cross-reference resolution uses `find_obj()` which builds a search list combining mod_name, prefix, and name. When `searchorder == 0`, the list is reversed so that less-qualified names are checked first (i.e., bare name matches before fully-qualified). When `searchorder == 1` (refspecific, triggered by a leading `.` in the role target), more-qualified names take priority.\n\n3. The `JSXRefRole.process_link()` method handles `~` prefix (to show only the last component after the final dot) and `.` prefix (to set `refspecific=True` on the reference node).\n\n4. The `js:func` and `js:meth` roles are instantiated with `fix_parens=True`, while `js:class`, `js:data`, `js:attr`, and `js:mod` roles are created without it.\n\n5. The `JSModule.run()` method creates entries in both the `modules` dict (via `note_module()`) and the `objects` dict (via `note_object()`).\n\n6. The `_pseudo_parse_arglist` function is imported from `sphinx.domains.python._annotations` and used in the JS domain for parsing argument lists.\n\n7. Two config values are registered: `javascript_maximum_signature_line_length` (default `None`, types `int | NoneType`) and `javascript_trailing_comma_in_multi_line_signatures` (default `True`, type `bool`).\n\n8. The domain is accessed via `self.env.domains.javascript_domain` (a typed attribute on `_DomainsContainer`).",
    "rubric": [
      "Identifies the class hierarchy: JSObject extends ObjectDescription, JSCallable extends JSObject with has_arguments=True, JSConstructor extends JSCallable with allow_nesting=True and a 'class' display prefix",
      "Notes that JSModule extends SphinxDirective directly (not ObjectDescription)",
      "States that both js:function and js:method directives map to the same JSCallable class",
      "Explains that setup() returns env_version 3 and declares both parallel_read_safe and parallel_write_safe as True",
      "Describes find_obj() search list construction (mod_name+prefix+name combinations) and that searchorder=0 reverses the list so bare names are checked first, while searchorder=1 (refspecific, triggered by leading dot) prioritizes more-qualified names",
      "Explains JSXRefRole.process_link() handling of ~ prefix (show only last component) and . prefix (sets refspecific=True)",
      "Notes that js:func and js:meth roles use fix_parens=True while js:class, js:data, js:attr, and js:mod do not",
      "Mentions that JSModule.run() creates entries in both the modules dict (via note_module) and the objects dict (via note_object)",
      "Notes that _pseudo_parse_arglist is imported from the Python domain annotations module and reused in the JS domain",
      "Identifies the two config values: javascript_maximum_signature_line_length (default None) and javascript_trailing_comma_in_multi_line_signatures (default True)"
    ],
    "key_files": [
      "sphinx/domains/javascript.py",
      "sphinx/domains/_domains_container.py",
      "sphinx/domains/python/_annotations.py"
    ],
    "source_doc": "[doc_file: doc/usage/domains/javascript.rst] The JavaScript Domain\n.. versionadded:: 1.0\n\nThe JavaScript domain (name **js**) provides the following directives:\n\n.. rst:directive:: .. js:module:: name\n\n   This directive sets the module name for object declarations that follow\n   after. The module name is used in the global module index and in cross\n   references. This directive does not create an object heading like\n   :rst:dir:`py:class` would, for example.\n\n   By default, this directive will create a linkable entity and will cause an\n   entry in the global module index, unless the ``no-index`` option is\n   specified.  If this option is specified, the directive will only update the\n   current module name.\n\n   .. versionadded:: 1.6\n   .. versionchanged:: 5.2\n\n      Module directives support body content.\n\n.. rst:directive:: .. js:function:: name(signature)\n\n   Describes a JavaScript function or method.  If you want to describe\n   arguments as optional use square brackets as :ref:`documented <signatures>`\n   for Python signatures.\n\n   You can use fields to give more details about arguments and their expected\n   types, errors which may be thrown by the function, and the value being\n   returned::\n\n      .. js:function:: $.getJSON(href, callback[, errback])\n\n         :param string href: An URI to the location of the resource.\n         :param callback: Gets called with the object.\n         :param errback:\n             Gets called in case the request fails. And a lot of other\n             text so we need multiple lines.\n         :throws SomeError: For whatever reason in that case.\n         :returns: Something.\n\n   This is rendered as:\n\n   .. js:function:: $.getJSON(href, callback[, errback])\n      :no-contents-entry:\n      :no-index-entry:\n\n      :param string href: An URI to the location of the resource.\n      :param callback: Gets called with the object.\n      :param errback:\n          Gets called in case the request fails. And a lot of other\n          text so we need multiple lines.\n      :throws SomeError: For whatever reason in that case.\n      :returns: Something.\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the function's parameters will be emitted on a single logical\n      line, overriding :confval:`javascript_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n.. rst:directive:: .. js:method:: name(signature)\n\n   This directive is an alias for :rst:dir:`js:function`, however it describes\n   a function that is implemented as a method on a class object.\n\n   .. versionadded:: 1.6\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the function's parameters will be emitted on a single logical\n      line, overriding :confval:`javascript_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n.. rst:directive:: .. js:class:: name\n\n   Describes a constructor that creates an object.  This is basically like a\n   function but will show up with a `class` prefix::\n\n      .. js:class:: MyAnimal(name[, age])\n\n         :param string name: The name of the animal\n         :param number age: an optional age for the animal\n\n   This is rendered as:\n\n   .. js:class:: MyAnimal(name[, age])\n      :no-contents-entry:\n      :no-index-entry:\n\n      :param string name: The name of the animal\n      :param number age: an optional age for the animal\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the function's parameters will be emitted on a single logical\n      line, overriding :confval:`javascript_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n.. rst:directive:: .. js:data:: name\n\n   Describes a global variable or constant.\n\n.. rst:directive:: .. js:attribute:: object.name\n\n   Describes the attribute *name* of *object*.\n\n.. _js-xref-roles:\n\nThese roles are provided to refer to the described objects:\n\n.. rst:role:: js:mod\n              js:func\n              js:meth\n              js:class\n              js:data\n              js:attr",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale phrases ('mirroring PyXRefRole', 'to facilitate searching', 'optional bracket notation'); core claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_08",
    "repo": "sphinx",
    "question": "How does Sphinx's doctest extension process directives and execute test groups from parsing through to running?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The doctest extension in Sphinx (`sphinx/ext/doctest.py`) implements its directive-based testing system through the following algorithm:\n\n## Directive Processing (TestDirective.run)\n\n1. **Code assembly**: Content lines are joined with `'\\n'.join(self.content)`.\n\n2. **Presentation vs. test separation**: For `doctest` directives, if the code contains `<BLANKLINE>` markers or doctest option comments (matching `r'[ \\t]*#\\s*doctest:.+$'`), the original code is stored in a `node['test']` attribute while a cleaned version (with markers/flags removed) is used for display. The `no-trim-doctest-flags` option prevents stripping of doctest option comments.\n\n3. **Node type selection**: `testsetup`/`testcleanup` directives and any directive with the `hide` option produce `nodes.comment` nodes, while others produce `nodes.literal_block` nodes.\n\n4. **Group parsing**: Arguments are split by comma and stripped. Empty arguments default to `['default']`.\n\n5. **pyversion handling**: When a `pyversion` option is specified on a `doctest` directive, it gets the current Python version as `'.'.join(map(str, sys.version_info[:3]))` and uses `packaging.specifiers.SpecifierSet` and `packaging.version.Version` to evaluate. If the version doesn't match, it sets the `SKIP` doctest option flag to `True` on the node.\n\n## Test Execution (DocTestBuilder.test_doc + test_group)\n\n1. **Node collection**: The builder traverses the doctree using `doctree.findall(condition)`. When `doctest_test_doctest_blocks` is truthy, it uses `_condition_with_doctest` which also matches `nodes.doctest_block` nodes; otherwise it uses `_condition_default`.\n\n2. **Group assignment with wildcard**: Nodes with `'*'` in their groups are collected in an `add_to_all_groups` list and added to ALL existing groups only AFTER all other nodes have been processed \u2014 meaning they can only be added to groups that were already created by non-wildcard nodes.\n\n3. **Global setup/cleanup injection**: `doctest_global_setup` code is prepended (using `add_code(code, prepend=True)`) to every group's setup list, while `doctest_global_cleanup` is appended to every group's cleanup list.\n\n4. **Compile monkey-patch**: The builder replaces `doctest.compile` with its own `compile` method and uses `self.type` (initially set to `'single'`) to control compilation mode.\n\n5. **Shared _fakeout**: All three runners (`setup_runner`, `test_runner`, `cleanup_runner`) share the same `_fakeout` StringIO object from `setup_runner`.\n\n6. **testcode/testoutput pairing**: In `TestGroup.add_code`, a `testoutput` node replaces the second element of the most recently added test tuple (which was initially `(testcode, None)`).\n\n7. **skipif evaluation**: The `skipped()` method executes `doctest_global_setup` in a fresh context dict, then `eval()`s the skipif condition in that context, then runs `doctest_global_cleanup` on the same context before returning the boolean result.\n\n8. **Fail-fast support**: If `doctest_fail_fast` is True and a test fails, processing stops between documents (visible in `write_documents` where `break` is called if not success and `doctest_fail_fast` is set).",
    "rubric": [
      "Explains that directive content is joined with newlines and that doctest directives separate presentation code (with BLANKLINE/option comments stripped) from test code stored in node['test']",
      "Describes node type selection: testsetup/testcleanup or 'hide' option produce nodes.comment, others produce nodes.literal_block",
      "Explains group parsing from arguments (comma-split, stripped, defaults to ['default']) and the wildcard '*' mechanism where wildcard nodes are added only to already-existing groups",
      "Describes the pyversion handling using packaging.specifiers.SpecifierSet and how non-matching versions set the SKIP flag",
      "Explains that doctest.compile is monkey-patched and self.type toggles between 'single' and 'exec' to control compilation mode",
      "Describes that all three runners (setup, test, cleanup) share the same _fakeout StringIO from setup_runner",
      "Explains testcode/testoutput pairing: testcode adds (code, None) tuple, testoutput replaces the None in the most recent test entry",
      "Describes the skipif/skipped mechanism: executes global_setup in fresh context, evals condition, then runs global_cleanup",
      "Mentions doctest_global_setup is prepended to every group's setup list and doctest_global_cleanup is appended to every group's cleanup list",
      "Explains fail-fast support: processing breaks between documents when doctest_fail_fast is set and a failure occurs"
    ],
    "key_files": [
      "sphinx/ext/doctest.py",
      "sphinx/util/docutils.py",
      "sphinx/builders/__init__.py"
    ],
    "source_doc": "[doc_file: doc/usage/extensions/doctest.rst] Directives\nThe *group* argument below is interpreted as follows: if it is empty, the block\nis assigned to the group named ``default``.  If it is ``*``, the block is\nassigned to all groups (including the ``default`` group).  Otherwise, it must be\na comma-separated list of group names.\n\n.. rst:directive:: .. testsetup:: [group]\n\n   A setup code block.  This code is not shown in the output for other builders,\n   but executed before the doctests of the group(s) it belongs to.\n\n   .. rubric:: Options\n\n   .. rst:directive:option:: skipif: condition\n      :type: text\n\n      Skip the directive if the python expression *condition* is True.\n      See :ref:`skipping tests conditionally <doctest-skipif>`.\n\n\n.. rst:directive:: .. testcleanup:: [group]\n\n   A cleanup code block.  This code is not shown in the output for other\n   builders, but executed after the doctests of the group(s) it belongs to.\n\n   .. versionadded:: 1.1\n\n   .. rubric:: Options\n\n   .. rst:directive:option:: skipif: condition\n      :type: text\n\n      Skip the directive if the python expression *condition* is True.\n      See :ref:`skipping tests conditionally <doctest-skipif>`.\n\n\n.. rst:directive:: .. doctest:: [group]\n\n   A doctest-style code block.  You can use standard :mod:`doctest` flags for\n   controlling how actual output is compared with what you give as output.  The\n   default set of flags is specified by the :confval:`doctest_default_flags`\n   configuration variable.\n\n   .. rubric:: Options\n\n   .. rst:directive:option:: hide\n\n     Hide the doctest block in other builders.\n     By default it is shown as a highlighted doctest block.\n\n   .. rst:directive:option:: options: doctest flags\n      :type: comma separated list\n\n      A comma-separated list of doctest flags that apply to each example in the\n      tests.  (You still can give explicit flags per example, with doctest comments,\n      but they will show up in other builders too.)\n\n      Alternatively, you can give inline doctest options, like in doctest:\n\n      .. code-block:: pycon\n\n         >>> datetime.date.now()   # doctest: +SKIP\n         datetime.date(2008, 1, 1)\n\n      They will be respected when the test is run, but by default will be stripped from\n      presentation output. You can prevent stripping using the option\n      :rst:dir:`doctest:no-trim-doctest-flags`.\n\n   .. rst:directive:option:: pyversion\n      :type: text\n\n      Specify the required Python version for the example to be tested. For instance,\n      in the following case the example will be tested only for Python versions greater\n      than 3.14::\n\n         .. doctest::\n            :pyversion: > 3.14\n\n      The following operands are supported:\n\n      * ``~=``: Compatible release clause\n      * ``==``: Version matching clause\n      * ``!=``: Version exclusion clause\n      * ``<=``, ``>=``: Inclusive ordered comparison clause\n      * ``<``, ``>``: Exclusive ordered comparison clause\n      * ``===``: Arbitrary equality clause.\n\n      ``pyversion`` option is followed :pep:`PEP-440: Version Specifiers\n      <440#version-specifiers>`.\n\n      .. versionadded:: 1.6\n\n      .. versionchanged:: 1.7\n\n         Supported PEP-440 operands and notations\n\n   .. rst:directive:option:: trim-doctest-flags\n                             no-trim-doctest-flags\n\n      Whether to trim remove doctest flags (comments looking like\n      ``# doctest: FLAG, ...``) at the ends of lines and ``<BLANKLINE>`` markers\n      individually.  Default is ``trim-doctest-flags``.\n\n      Note that like with standard doctests, you have to use ``<BLANKLINE>`` to\n      signal a blank line in the expected output.  The ``<BLANKLINE>`` is removed\n      when building presentation output (HTML, LaTeX etc.).\n\n   .. rst:directive:option:: skipif: condition\n      :type: text\n\n      Skip the directive if the python expression *condition* is True.\n      See :ref:`skipping tests conditionally <doctest-skipif>`.\n\n.. rst:directive:: .. testcode:: [group]\n\n   A code block for a code-output-style test.\n\n   .. rubric:: Options\n\n   .. rst:directive:option:: hide\n\n      Hide the code block in other builders.\n      By default it is shown as a highlighted code block.\n\n   .. rst:directive:option:: trim-doctest-flags\n                             no-trim-doctest-flags\n\n      Whether to trim remove doctest flags (comments looking like\n      ``# doctest: FLAG, ...``) at the ends of lines and ``<BLANKLINE>`` markers\n      individually.  Default is ``trim-doctest-flags``.\n\n   .. rst:directive:option:: skipif: condition\n      :type: text\n\n      Skip the directive if the python expression *condition* is True.\n      See :ref:`skipping tests conditionally <doctest-skipif>`.\n\n   .. note::\n\n      Code in a ``testcode`` block is always executed all at once, no matter how\n      many statements it contains.  Therefore, output will *not* be generated\n      for bare expressions -- use ``print``.  Example::\n\n          .. testcode::\n\n             1+1         # this will give no output!\n             print(2+2)  # this will give output\n\n          .. testoutput::\n\n             4\n\n      Also, please be aware that since the doctest module does not support\n      mixing regular output and an exception message in the same snippet, this\n      applies to testcode/testoutput as well.\n\n\n.. rst:directive:: .. testoutput:: [group]\n\n   The corresponding output, or the exception message, for the last\n   :rst:dir:`testcode` block.\n\n   .. rst:directive:option:: hide\n\n     Hide the doctest block in other builders.\n     By default it is shown as a highlighted doctest block.\n\n   .. rst:directive:option:: options: doctest flags\n      :type: comma separated list\n\n      A comma-separated list of doctest flags.\n\n   .. rst:directive:option:: trim-doctest-flags\n                             no-trim-doctest-flags\n\n      Whether to trim remove doctest flags (comments looking like\n      ``# doctest: FLAG, ...``) at the ends of lines and ``<BLANKLINE>`` markers\n      individually.  Default is ``trim-doctest-flags``.\n\n   .. rst:directive:option:: skipif: condition\n      :type: text\n\n      Skip the directive if the python expression *condition* is True.\n      See :ref:`skipping tests conditionally <doctest-skipif>`.\n\n   Example::\n\n      .. testcode::\n\n         print('Output     text.')\n\n      .. testoutput::\n         :hide:\n         :options: -ELLIPSIS, +NORMALIZE_WHITESPACE\n\n         Output text.\n\nThe following is an example for the usage of the directives.  The test via\n:rst:dir:`doctest` and the test via :rst:dir:`testcode` and\n:rst:dir:`testoutput` are equivalent. ::\n\n   The parrot module\n   =================\n\n   .. testsetup:: *\n\n      import parrot\n\n   The parrot module is a module about parrots.\n\n   Doctest example:\n\n   .. doctest::\n\n      >>> parrot.voom(3000)\n      This parrot wouldn't voom if you put 3000 volts through it!\n\n   Test-Output example:\n\n   .. testcode::\n\n      parrot.voom(3000)\n\n   This would output:\n\n   .. testoutput::\n\n      This parrot wouldn't voom if you put 3000 volts through it!\n\n\n.. _doctest-skipif:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer describes implementation details (e.g., monkey-patching doctest.compile, _fakeout StringIO sharing, ns dict usage, _EXCEPTION_RE regex, fail-fast support) that are not present in the documentation. These appear to be code-level implementation details that could be plausible but cannot be verified from the provided documentation alone.",
      "The answer states that wildcard ('*') nodes 'can only be added to groups that were already created by non-wildcard nodes.' The documentation states that '*' means 'the block is assigned to all groups (including the default group)' but does not specify the ordering constraint described in the answer. This is a plausible implementation detail but is not supported by the doc.",
      "The answer's description of pyversion handling mentions packaging.specifiers.SpecifierSet and packaging.version.Version - the doc only says it follows PEP-440 version specifiers, not the specific implementation mechanism.",
      "The answer states 'DONT_ACCEPT_BLANKLINE is always enabled' for testcode/testoutput pairs - this is not mentioned in the documentation.",
      "The answer's claim about group parsing ('Arguments are split by comma and stripped. Empty arguments default to [\"default\"]') is consistent with the documentation's statement about the group argument interpretation.",
      "The answer correctly describes the skipif option behavior across directives, matching the documentation.",
      "The answer correctly describes that testsetup/testcleanup are not shown in output for other builders, matching the documentation.",
      "The answer correctly describes the no-trim-doctest-flags behavior matching the documentation.",
      "The answer correctly describes the testcode note about bare expressions needing print, matching the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rely on truncated code portions or documentation about external module behavior.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_06",
    "repo": "sphinx",
    "question": "Why does the C++ domain use both an `object_type` and a `directiveType` for class/struct declarations instead of just one type field?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `cpp:class` and `cpp:struct` directives are both handled by the same Python class `CPPClassObject` (in `sphinx/domains/cpp/__init__.py`), which has `object_type = 'class'`. These two directives are separated through a two-level type system:\n\n1. **`objectType`**: Always `'class'` for both directives. This is used for symbol lookup, ID generation, and determines how the parser processes the declaration (via `_parse_class()` in `_parser.py`).\n\n2. **`directiveType`**: Either `'class'` or `'struct'`, derived from `self.objtype` which is set at runtime from the actual directive name (e.g., `'cpp:struct'` \u2192 `self.objtype = 'struct'`). This is stored in `ASTDeclaration.directiveType` and used in two places:\n   - **Rendering**: In `ASTDeclaration.describe_signature()` in `_ast.py`, the `directiveType` value is used as the keyword prefix rendered before the class name (either 'class' or 'struct').\n   - **Index text**: `CPPClassObject.display_object_type` property returns `self.objtype` (not the fixed `object_type`), so the index entry says either 'C++ class' or 'C++ struct'.\n\nThe same pattern applies to enums: `directiveType` distinguishes `'enum'`, `'enum-class'`, and `'enum-struct'`, and critically controls the scoping behavior of enumerators \u2014 `_add_enumerator_to_parent()` only propagates enumerator names to the parent scope when `parent_decl.directiveType == 'enum'` (unscoped), not for `'enum-class'` or `'enum-struct'` (scoped).\n\nAdditionally, visibility (`public`, `private`, `protected`) is only rendered in the output when it is NOT `'public'` \u2014 both in `_stringify()` and `describe_signature()` the condition `self.visibility != 'public'` gates the rendering.\n\nThe C++ domain uses ID versioning with `_max_id = 4` and prefixes `('', '', '_CPPv2', '_CPPv3', '_CPPv4')` for backward compatibility. The `ASTClass` parser supports `final` keyword, virtual inheritance, parameter pack expansion (`...`) in base classes, and attributes.",
    "rubric": [
      "Explains that `object_type` is always 'class' for both cpp:class and cpp:struct directives, used for parsing/symbol lookup/ID generation",
      "Explains that `directiveType` captures the actual directive name ('class' or 'struct') and is stored in ASTDeclaration",
      "Mentions that `directiveType` controls the keyword prefix rendered in the signature output (showing 'class' or 'struct')",
      "Mentions that `display_object_type` uses `self.objtype` (not the fixed `object_type`) so index entries correctly say 'C++ class' or 'C++ struct'",
      "Notes that both directives are handled by the same CPPClassObject Python class",
      "Mentions a parallel pattern with enums where directiveType distinguishes scoped vs unscoped enums and controls enumerator propagation to parent scope"
    ],
    "key_files": [
      "sphinx/domains/cpp/__init__.py",
      "sphinx/domains/cpp/_ast.py",
      "sphinx/domains/cpp/_parser.py",
      "sphinx/domains/cpp/_ids.py"
    ],
    "source_doc": "[doc_file: doc/usage/domains/cpp.rst] Directives for Declaring Entities\nThe following directives are available. All declarations can start with a\nvisibility statement (``public``, ``private`` or ``protected``).\n\n.. rst:directive:: .. cpp:class:: class specifier\n                   .. cpp:struct:: class specifier\n\n   Describe a class/struct, possibly with specification of inheritance, e.g.,::\n\n      .. cpp:class:: MyClass : public MyBase, MyOtherBase\n\n   The difference between :rst:dir:`cpp:class` and :rst:dir:`cpp:struct` is\n   only cosmetic: the prefix rendered in the output, and the specifier shown\n   in the index.\n\n   The class can be directly declared inside a nested scope, e.g.,::\n\n      .. cpp:class:: OuterScope::MyClass : public MyBase, MyOtherBase\n\n   A class template can be declared::\n\n      .. cpp:class:: template<typename T, std::size_t N> std::array\n\n   or with a line break::\n\n      .. cpp:class:: template<typename T, std::size_t N> \\\n                     std::array\n\n   Full and partial template specialisations can be declared::\n\n      .. cpp:class:: template<> \\\n                     std::array<bool, 256>\n\n      .. cpp:class:: template<typename T> \\\n                     std::array<T, 42>\n\n   .. versionadded:: 2.0\n      The :rst:dir:`cpp:struct` directive.\n\n.. rst:directive:: .. cpp:function:: (member) function prototype\n\n   Describe a function or member function, e.g.,::\n\n      .. cpp:function:: bool myMethod(int arg1, std::string arg2)\n\n         A function with parameters and types.\n\n      .. cpp:function:: bool myMethod(int, double)\n\n         A function with unnamed parameters.\n\n      .. cpp:function:: const T &MyClass::operator[](std::size_t i) const\n\n         An overload for the indexing operator.\n\n      .. cpp:function:: operator bool() const\n\n         A casting operator.\n\n      .. cpp:function:: constexpr void foo(std::string &bar[2]) noexcept\n\n         A constexpr function.\n\n      .. cpp:function:: MyClass::MyClass(const MyClass&) = default\n\n         A copy constructor with default implementation.\n\n   Function templates can also be described::\n\n      .. cpp:function:: template<typename U> \\\n                        void print(U &&u)\n\n   and function template specialisations::\n\n      .. cpp:function:: template<> \\\n                        void print(int i)\n\n   .. rst:directive:option:: single-line-parameter-list\n      :type: no value\n\n      Ensures that the function's parameters will be emitted on a single logical\n      line, overriding :confval:`cpp_maximum_signature_line_length` and\n      :confval:`maximum_signature_line_length`.\n\n      .. versionadded:: 7.1\n\n.. rst:directive:: .. cpp:member:: (member) variable declaration\n                   .. cpp:var:: (member) variable declaration\n\n   Describe a variable or member variable, e.g.,::\n\n      .. cpp:member:: std::string MyClass::myMember\n\n      .. cpp:var:: std::string MyClass::myOtherMember[N][M]\n\n      .. cpp:member:: int a = 42\n\n   Variable templates can also be described::\n\n      .. cpp:member:: template<class T> \\\n                      constexpr T pi = T(3.1415926535897932385)\n\n.. rst:directive:: .. cpp:type:: typedef declaration\n                   .. cpp:type:: name\n                   .. cpp:type:: type alias declaration\n\n   Describe a type as in a typedef declaration, a type alias declaration, or\n   simply the name of a type with unspecified type, e.g.,::\n\n      .. cpp:type:: std::vector<int> MyList\n\n         A typedef-like declaration of a type.\n\n      .. cpp:type:: MyContainer::const_iterator\n\n         Declaration of a type alias with unspecified type.\n\n      .. cpp:type:: MyType = std::unordered_map<int, std::string>\n\n         Declaration of a type alias.\n\n   A type alias can also be templated::\n\n      .. cpp:type:: template<typename T> \\\n                    MyContainer = std::vector<T>\n\n   The example are rendered as follows.\n\n   .. cpp:type:: std::vector<int> MyList\n      :no-contents-entry:\n      :no-index-entry:\n\n      A typedef-like declaration of a type.\n\n   .. cpp:type:: MyContainer::const_iterator\n      :no-contents-entry:\n      :no-index-entry:\n\n      Declaration of a type alias with unspecified type.\n\n   .. cpp:type:: MyType = std::unordered_map<int, std::string>\n      :no-contents-entry:\n      :no-index-entry:\n\n      Declaration of a type alias.\n\n   .. cpp:type:: template<typename T> \\\n                 MyContainer = std::vector<T>\n      :no-contents-entry:\n      :no-index-entry:\n\n.. rst:directive:: .. cpp:enum:: unscoped enum declaration\n                   .. cpp:enum-struct:: scoped enum declaration\n                   .. cpp:enum-class:: scoped enum declaration\n\n   Describe a (scoped) enum, possibly with the underlying type specified.  Any\n   enumerators declared inside an unscoped enum will be declared both in the\n   enum scope and in the parent scope.  Examples::\n\n      .. cpp:enum:: MyEnum\n\n         An unscoped enum.\n\n      .. cpp:enum:: MySpecificEnum : long\n\n         An unscoped enum with specified underlying type.\n\n      .. cpp:enum-class:: MyScopedEnum\n\n         A scoped enum.\n\n      .. cpp:enum-struct:: protected MyScopedVisibilityEnum : std::underlying_type<MySpecificEnum>::type\n\n         A scoped enum with non-default visibility, and with a specified\n         underlying type.\n\n.. rst:directive:: .. cpp:enumerator:: name\n                   .. cpp:enumerator:: name = constant\n\n   Describe an enumerator, optionally with its value defined, e.g.,::\n\n      .. cpp:enumerator:: MyEnum::myEnumerator\n\n      .. cpp:enumerator:: MyEnum::myOtherEnumerator = 42\n\n.. rst:directive:: .. cpp:union:: name\n\n   Describe a union.\n\n   .. versionadded:: 1.8\n\n.. rst:directive:: .. cpp:concept:: template-parameter-list name\n\n   .. warning:: The support for concepts is experimental. It is based on the\n      current draft standard and the Concepts Technical Specification.\n      The features may change as they evolve.\n\n   Describe a concept. It must have exactly 1 template parameter list. The name\n   may be a nested name. Example::\n\n      .. cpp:concept:: template<typename It> std::Iterator\n\n         Proxy to an element of a notional sequence that can be compared,\n         indirected, or incremented.\n\n         **Notation**\n\n         .. cpp:var:: It r\n\n            An lvalue.\n\n         **Valid Expressions**\n\n         - :cpp:expr:`*r`, when :cpp:expr:`r` is dereferenceable.\n         - :cpp:expr:`++r`, with return type :cpp:expr:`It&`, when\n           :cpp:expr:`r` is incrementable.\n\n   This will render as follows:\n\n   .. cpp:concept:: template<typename It> std::Iterator\n      :no-contents-entry:\n      :no-index-entry:\n\n      Proxy to an element of a notional sequence that can be compared,\n      indirected, or incremented.\n\n      **Notation**\n\n      .. cpp:var:: It r\n\n         An lvalue.\n\n      **Valid Expressions**\n\n      - :cpp:expr:`*r`, when :cpp:expr:`r` is dereferenceable.\n      - :cpp:expr:`++r`, with return type :cpp:expr:`It&`, when :cpp:expr:`r`\n        is incrementable.\n\n   .. versionadded:: 1.5",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation-level details (Python class names like CPPClassObject, file paths like sphinx/domains/cpp/__init__.py, line numbers, _max_id = 4, ID versioning prefixes, etc.) that cannot be verified from the provided documentation. These appear to be code-level details that are plausible but go far beyond what the documentation states.",
      "The core claim that cpp:class and cpp:struct differ only cosmetically (prefix rendered in output and specifier shown in index) is directly supported by the documentation.",
      "The claim about enum scoping behavior \u2014 that enumerators in unscoped enums are declared in both enum scope and parent scope \u2014 is supported by the documentation: 'Any enumerators declared inside an unscoped enum will be declared both in the enum scope and in the parent scope.'",
      "The claim about visibility rendering only when not 'public' is not verifiable from the documentation. The documentation only says 'All declarations can start with a visibility statement (public, private or protected).' It does not describe rendering behavior for visibility.",
      "The claim about ASTClass supporting 'final' keyword, virtual inheritance, parameter pack expansion, and attributes is not verifiable from this documentation chunk.",
      "The enum directiveType distinction between 'enum', 'enum-class', and 'enum-struct' is consistent with the documentation's three directives: cpp:enum, cpp:enum-struct, cpp:enum-class."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor wording about 'design decisions' and specific line numbers needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_09",
    "repo": "sphinx",
    "question": "What is the full implementation chain for the `skipif` option in the Sphinx doctest extension, from directive parsing to skip evaluation?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `skipif` option in the Sphinx doctest extension is implemented through a chain of components in `sphinx/ext/doctest.py`. Here's how it works in detail:\n\n**Directive Layer:** All five directive classes (`TestsetupDirective`, `TestcleanupDirective`, `DoctestDirective`, `TestcodeDirective`, `TestoutputDirective`) inherit from `TestDirective`, which itself inherits from `SphinxDirective` (imported from `sphinx.util.docutils`). Each directive class declares `'skipif': directives.unchanged_required` in its `option_spec`. The base `TestDirective.run()` method stores the skipif value on the docutils node: `if 'skipif' in self.options: node['skipif'] = self.options['skipif']`.\n\n**Node Type Selection:** The `TestDirective.run()` method creates either a `nodes.literal_block` or `nodes.comment` node depending on the directive type. `testsetup` and `testcleanup` directives always produce `nodes.comment` nodes (hidden), while directives with the `hide` option also produce `nodes.comment` nodes.\n\n**Skip Evaluation in Builder:** The `DocTestBuilder.skipped()` method evaluates the skipif condition. Critically, it creates a fresh `context` dict for each evaluation, executes `doctest_global_setup` via `exec()` into that context, then uses `eval()` to evaluate the skipif condition string within that context. After evaluation, if `doctest_global_cleanup` is configured, it is also `exec()`'d in the same context. This means the global setup/cleanup code runs once for each node that has a skipif attribute, independently from the setup/cleanup that runs for the test groups themselves.\n\n**Filtering Location:** The `skipped()` method is called inside `test_doc()` during the node traversal loop: `if self.skipped(node): continue`. A skipped node is completely excluded from being added to any `TestGroup`, meaning it never becomes a `TestCode` object at all. This is a pre-filtering step that happens before test groups are assembled.\n\n**Node Filtering Conditions:** The `test_doc()` method uses one of two filter functions to find relevant nodes: `_condition_default` or `_condition_with_doctest`. The choice depends on whether `doctest_test_doctest_blocks` config is truthy.\n\n**Default doctest flags:** The `DocTestBuilder` uses `self.config.doctest_default_flags` as its option flags, stored in `self.opt` during `init()` and passed as `optionflags` to the `SphinxDocTestRunner` instances.\n\n**Test structure:** `TestsetupDirective` and `TestcleanupDirective` have the simplest `option_spec` with only `skipif`, while `DoctestDirective` additionally supports `hide`, `no-trim-doctest-flags`, `options`, `pyversion`, and `trim-doctest-flags`. The `TestcodeDirective` supports `hide`, `no-trim-doctest-flags`, `pyversion`, `skipif`, and `trim-doctest-flags` (but not `options`). The `TestoutputDirective` supports all of these including `options`.",
    "rubric": [
      "Mentions that all five directive classes (TestsetupDirective, TestcleanupDirective, DoctestDirective, TestcodeDirective, TestoutputDirective) inherit from TestDirective which inherits from SphinxDirective",
      "Explains that each directive declares 'skipif': directives.unchanged_required in its option_spec",
      "Explains that TestDirective.run() stores the skipif value on the node: node['skipif'] = self.options['skipif']",
      "Describes that testsetup/testcleanup directives produce nodes.comment nodes while others produce nodes.literal_block (unless 'hide' is specified)",
      "Explains how DocTestBuilder.skipped() creates a fresh context dict, exec's doctest_global_setup into it, then eval's the skipif condition in that context",
      "Notes that doctest_global_cleanup is also exec'd in the same context after evaluation if configured",
      "Explains that skipped() is called in test_doc() during node traversal and skipped nodes are completely excluded from TestGroup assembly (continue statement)",
      "Mentions the two filter functions _condition_default and _condition_with_doctest used to find relevant nodes",
      "Notes the differences in option_spec between the directives (e.g., TestsetupDirective only has skipif, while DoctestDirective has hide, options, pyversion, trim flags, etc.)"
    ],
    "key_files": [
      "sphinx/ext/doctest.py",
      "tests/roots/test-ext-doctest-skipif/skipif.txt",
      "tests/roots/test-ext-doctest-skipif/conf.py",
      "tests/test_extensions/test_ext_doctest.py"
    ],
    "source_doc": "[doc_file: doc/usage/extensions/doctest.rst] Skipping tests conditionally\n``skipif``, a string option, can be used to skip directives conditionally. This\nmay be useful e.g. when a different set of tests should be run depending on the\nenvironment (hardware, network/VPN, optional dependencies or different versions\nof dependencies). The ``skipif`` option is supported by all of the doctest\ndirectives. Below are typical use cases for ``skipif`` when used for different\ndirectives:\n\n- :rst:dir:`testsetup` and :rst:dir:`testcleanup`\n\n  - conditionally skip test setup and/or cleanup\n  - customize setup/cleanup code per environment\n\n- :rst:dir:`doctest`\n\n  - conditionally skip both a test and its output verification\n\n- :rst:dir:`testcode`\n\n  - conditionally skip a test\n  - customize test code per environment\n\n- :rst:dir:`testoutput`\n\n  - conditionally skip output assertion for a skipped test\n  - expect different output depending on the environment\n\nThe value of the ``skipif`` option is evaluated as a Python expression. If the\nresult is a true value, the directive is omitted from the test run just as if\nit wasn't present in the file at all.\n\nInstead of repeating an expression, the :confval:`doctest_global_setup`\nconfiguration option can be used to assign it to a variable which can then be\nused instead.\n\nHere's an example which skips some tests if Pandas is not installed:\n\n.. code-block:: py\n   :caption: conf.py\n\n   extensions = ['sphinx.ext.doctest']\n   doctest_global_setup = '''\n   try:\n       import pandas as pd\n   except ImportError:\n       pd = None\n   '''\n\n.. code-block:: rst\n   :caption: contents.rst\n\n   .. testsetup::\n      :skipif: pd is None\n\n      data = pd.Series([42])\n\n   .. doctest::\n      :skipif: pd is None\n\n      >>> data.iloc[0]\n      42\n\n   .. testcode::\n      :skipif: pd is None\n\n      print(data.iloc[-1])\n\n   .. testoutput::\n      :skipif: pd is None\n\n      42",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the specific default flag values and setup() return value needed minor rewrites due to truncation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_10",
    "repo": "sphinx",
    "question": "Why does Napoleon's docstring parser use a dict for `_sections`, `frozenset` for `_SINGLETONS`, `functools.partial` for admonitions, and a deque subclass for lines?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The Napoleon extension's docstring section parsing uses several deliberate performance optimizations:\n\n**1. Dictionary-based section dispatch (O(1) lookup instead of linear search):**\nThe `_sections` attribute in `GoogleDocstring.__init__` is a `dict[str, Callable[..., list[str]]]` mapping 34 lowercase section names to their parser callables. This enables O(1) average-time lookup when checking if a line is a section header, using `section.strip(':') in self._sections` rather than iterating through a list. `_is_section_header()` is called from `_consume_contiguous()`, `_is_section_break()`, `_consume_indented_block()`, and the main `_parse()` loop.\n\n**2. Module-level pre-compiled regex patterns:**\nAll regex patterns (`_google_section_regex`, `_numpy_section_regex`, `_directive_regex`, etc.) are compiled once at module import time rather than inside methods. These are used in the parsing methods \u2014 for example, `_google_section_regex` is used in section header detection.\n\n**3. `functools.partial` to avoid branching in section dispatch:**\nAdmonition-type sections (attention, caution, danger, error, hint, important, note, tip, todo, warning) are stored as `partial(self._parse_admonition, 'attention')` etc. in the `_sections` dict. When `_parse()` dispatches via `self._sections[section.lower()](section)`, the admonition name is already bound into the callable via `partial`, combining the dispatch lookup and parameter binding into a single dict lookup + call.\n\n**4. `frozenset` for singleton membership testing:**\n`_SINGLETONS = frozenset({'None', 'True', 'False', 'Ellipsis', '...'})` uses a `frozenset` for O(1) membership testing in `_convert_type_spec_obj()` (via `translation in _SINGLETONS`).\n\n**5. `hasattr` guard to prevent redundant dict rebuilding in subclasses:**\nThe `_sections` dict is only built if `not hasattr(self, '_sections')`. Similarly, `_directive_sections` is only set if `not hasattr(self, '_directive_sections')`. This pattern allows subclasses to pre-set these attributes before calling `super().__init__()` without the parent overwriting them.\n\n**6. `Deque` (collections.deque subclass) for O(1) line consumption:**\nLines are stored in a `Deque` (custom subclass of `collections.deque`) which provides O(1) `popleft()` via the `next()` method and O(1) length checking. The `get(n)` method provides safe indexed access with a sentinel fallback, used for peek-ahead operations.\n\n**7. Lazy annotation caching in `_lookup_annotation`:**\nThe `_lookup_annotation()` method uses a `hasattr(self, '_annotations')` check to cache type annotations \u2014 `get_type_hints()` is only called once per docstring object, and subsequent field lookups reuse the cached `_annotations` dict.",
    "rubric": [
      "Explains that _sections is a dict mapping section names to callables, enabling O(1) lookup for section header detection instead of linear search",
      "Mentions that _SINGLETONS is a frozenset for O(1) membership testing used in type spec conversion",
      "Explains that functools.partial pre-binds the admonition type name (e.g., 'attention', 'caution') into the callable, avoiding branching/conditionals during dispatch",
      "Explains the Deque subclass of collections.deque provides O(1) popleft via next() for efficient line-by-line consumption",
      "Mentions that module-level regex patterns are pre-compiled once at import time rather than per-call",
      "Notes the hasattr guard pattern on _sections and _directive_sections to avoid redundant rebuilding in subclasses",
      "Notes lazy annotation caching in _lookup_annotation using hasattr check so get_type_hints is called only once per docstring object"
    ],
    "key_files": [
      "sphinx/ext/napoleon/docstring.py",
      "sphinx/ext/napoleon/__init__.py"
    ],
    "source_doc": "[doc_file: doc/usage/extensions/napoleon.rst] Docstring Sections\nAll of the following section headers are supported:\n\n* ``Args`` *(alias of Parameters)*\n* ``Arguments`` *(alias of Parameters)*\n* ``Attention``\n* ``Attributes``\n* ``Caution``\n* ``Danger``\n* ``Error``\n* ``Example``\n* ``Examples``\n* ``Hint``\n* ``Important``\n* ``Keyword Args`` *(alias of Keyword Arguments)*\n* ``Keyword Arguments``\n* ``Methods``\n* ``Note``\n* ``Notes``\n* ``Other Parameters``\n* ``Parameters``\n* ``Return`` *(alias of Returns)*\n* ``Returns``\n* ``Raise`` *(alias of Raises)*\n* ``Raises``\n* ``References``\n* ``See Also``\n* ``Tip``\n* ``Todo``\n* ``Warning``\n* ``Warnings`` *(alias of Warning)*\n* ``Warn`` *(alias of Warns)*\n* ``Warns``\n* ``Yield`` *(alias of Yields)*\n* ``Yields``",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation chunk provided only lists supported docstring section headers. The generated answer discusses internal implementation details (line numbers, data structures, optimization strategies) that cannot be verified from this documentation alone.",
      "The answer claims 34 lowercase section names in the _sections dict, but the documentation lists 33 section headers (counting aliases as separate entries). This count discrepancy cannot be verified from the doc alone.",
      "All specific line number references, class names, method names, and implementation details in the answer are beyond what the documentation chunk covers and cannot be verified or contradicted from it."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from the stripped code structure; a few design rationale statements and references to truncated code needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_11",
    "repo": "sphinx",
    "question": "Where are the Google and NumPy docstring parsers defined, how do they relate to each other, and how are they integrated into Sphinx's autodoc pipeline?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The Google and NumPy docstring parsing classes are both located in `sphinx/ext/napoleon/docstring.py`. `GoogleDocstring` is defined as a class and `NumpyDocstring` is also defined there, where `NumpyDocstring` **inherits from** `GoogleDocstring`. The dispatch/integration point is in `sphinx/ext/napoleon/__init__.py`, in the `_process_docstring` function, which is connected to the `'autodoc-process-docstring'` event in the `setup()` function via `app.connect('autodoc-process-docstring', _process_docstring)`. Notably, in `_process_docstring`, NumPy parsing is applied **first**, then Google parsing is applied **second** \u2014 both are applied if both config flags are True.\n\nKey structural differences between the two parsers:\n\n1. **Section header detection**: `GoogleDocstring._is_section_header()` uses `_google_section_regex = re.compile(r'^(\\s|\\w)+:\\s*$')` \u2014 matching lines ending with a colon, and requires the next line to be indented more than the header. `NumpyDocstring._is_section_header()` checks if the next line (the underline) matches `_numpy_section_regex = re.compile(r'^[=\\-`:\\'\"~^_*+#<>]{2,}\\s*$')` \u2014 i.e., 2 or more underline characters.\n\n2. **Section header consumption**: `GoogleDocstring._consume_section_header()` consumes one line and strips trailing colons. `NumpyDocstring._consume_section_header()` consumes two lines \u2014 the header plus the underline.\n\n3. **Section break detection**: `NumpyDocstring._is_section_break()` adds a condition `(not line1 and not line2)` \u2014 two consecutive blank lines constitute a section break in NumPy style. `GoogleDocstring._is_section_break()` does not have this condition.\n\n4. **Field consumption**: `NumpyDocstring._consume_field()` parses type using colon separation (`name : type`), while `GoogleDocstring._consume_field()` extracts type from parentheses using `_google_typed_arg_regex = re.compile(r'(.+?)\\(\\s*(.*[^\\s]+)\\s*\\)')` (i.e., `name(type)`).\n\n5. **Returns section**: `NumpyDocstring._consume_returns_section()` delegates to `_consume_fields(prefer_type=True)`, supporting multiple return values. `GoogleDocstring._consume_returns_section()` parses a single type:description pair from colon-separated content.\n\n6. **See Also section**: `NumpyDocstring._parse_see_also_section()` has a special `_parse_numpydoc_see_also_section` parser that handles comma-separated function references. `GoogleDocstring._parse_see_also_section()` simply delegates to `_parse_admonition('seealso', section)`.\n\n7. **`NumpyDocstring.__init__`** sets `self._directive_sections = ['.. index::']` before calling super(), while `GoogleDocstring` initializes `_directive_sections` as an empty list.\n\n8. **`_escape_args_and_kwargs`**: `NumpyDocstring` overrides this to handle comma-separated parameter names (e.g., `x, y`), splitting on ', ' and escaping each individually.",
    "rubric": [
      "Both GoogleDocstring and NumpyDocstring are defined in sphinx/ext/napoleon/docstring.py",
      "NumpyDocstring inherits from GoogleDocstring",
      "Integration happens in _process_docstring in sphinx/ext/napoleon/__init__.py, connected via app.connect('autodoc-process-docstring', _process_docstring) in setup()",
      "In _process_docstring, NumPy parsing is applied first, then Google parsing second (both applied if both config flags are True)",
      "GoogleDocstring detects section headers via lines ending with a colon (with next line indented more), while NumpyDocstring uses underline characters (2+ of =, -, etc.)",
      "NumpyDocstring._consume_section_header() consumes two lines (header + underline) vs GoogleDocstring consuming one line and stripping the colon",
      "NumpyDocstring treats two consecutive blank lines as a section break, which GoogleDocstring does not",
      "GoogleDocstring extracts field types from parentheses (name(type)) while NumpyDocstring uses colon separation (name : type)",
      "NumpyDocstring has a specialized _parse_numpydoc_see_also_section for comma-separated references, while GoogleDocstring uses a simple admonition",
      "NumpyDocstring overrides _escape_args_and_kwargs to handle comma-separated parameter names"
    ],
    "key_files": [
      "sphinx/ext/napoleon/docstring.py",
      "sphinx/ext/napoleon/__init__.py"
    ],
    "source_doc": "[doc_file: doc/usage/extensions/napoleon.rst] Google vs NumPy\nNapoleon supports two styles of docstrings: `Google`_ and `NumPy`_. The\nmain difference between the two styles is that Google uses indentation to\nseparate sections, whereas NumPy uses underlines.\n\nGoogle style:\n\n.. code-block:: python\n\n    def func(arg1, arg2):\n        \"\"\"Summary line.\n\n        Extended description of function.\n\n        Args:\n            arg1 (int): Description of arg1\n            arg2 (str): Description of arg2\n\n        Returns:\n            bool: Description of return value\n\n        \"\"\"\n        return True\n\nNumPy style:\n\n.. code-block:: python\n\n    def func(arg1, arg2):\n        \"\"\"Summary line.\n\n        Extended description of function.\n\n        Parameters\n        ----------\n        arg1 : int\n            Description of arg1\n        arg2 : str\n            Description of arg2\n\n        Returns\n        -------\n        bool\n            Description of return value\n\n        \"\"\"\n        return True\n\nNumPy style tends to require more vertical space, whereas Google style\ntends to use more horizontal space. Google style tends to be easier to\nread for short and simple docstrings, whereas NumPy style tends be easier\nto read for long and in-depth docstrings.\n\nThe choice between styles is largely aesthetic, but the two styles should\nnot be mixed. Choose one style for your project and be consistent with it.\n\n.. seealso::\n\n   For complete examples:\n\n   * :ref:`example_google`\n   * :ref:`example_numpy`",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only exact line numbers needed removal as they may shift after stripping.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_12",
    "repo": "sphinx",
    "question": "How does the Napoleon extension process a NumPy-style `Yields` section from docstring detection through to final reStructuredText output?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `example_generator` function in `doc/usage/extensions/example_numpy.py` uses `yield from range(n)` to implement the generator.\n\n**Docstring Processing Pipeline**: When Sphinx processes docstrings, the `_process_docstring` function in `sphinx/ext/napoleon/__init__.py` is connected to the `'autodoc-process-docstring'` event (via `app.connect('autodoc-process-docstring', _process_docstring)` in `setup()`). It first passes the docstring through `NumpyDocstring`, then optionally through `GoogleDocstring`, modifying the lines list in place.\n\n**Section Detection**: The `NumpyDocstring._is_section_header()` method identifies section headers by checking if the section name (lowercased) exists in `self._sections` AND the next line matches `_numpy_section_regex` (pattern: `r'^[=\\-`:'\\\"~^_*+#<>]{2,}\\s*$'`), which matches underline characters like `------`.\n\n**Section Routing**: Both `'yield'` and `'yields'` map to `self._parse_yields_section` in the `_sections` dictionary.\n\n**Yields vs Returns Handling**: `_parse_yields_section` calls `self._consume_returns_section(preprocess_types=True)` followed by `self._format_fields(_('Yields'), fields)`. This differs from `_parse_returns_section` which uses `:returns:` and `:rtype:` docutils directives and respects `napoleon_use_rtype` config. The yields section always uses the generic `_format_fields` method which produces `:Yields:` field format.\n\n**NumpyDocstring._consume_returns_section Override**: In the `NumpyDocstring` subclass, `_consume_returns_section` delegates to `self._consume_fields(prefer_type=True)`, which iterates fields until a section break. With `prefer_type=True`, when no explicit type is found after a colon, the name is swapped into the type position (visible in `_consume_field`: `if prefer_type and not _type: _type, _name = _name, _type`).\n\n**Field Formatting**: `_format_field` handles the case where `_name` is empty but `_type` is set by formatting the type as italicized when there's no backtick in the type string.\n\n**NumpyDocstring** extends `GoogleDocstring` and sets `self._directive_sections = ['.. index::']` in its `__init__`. The `Deque` class wraps `collections.deque` with `.get(n)` and `.next()` methods used for line-by-line parsing.",
    "rubric": [
      "Explains that `_process_docstring` is connected to the `autodoc-process-docstring` event and passes the docstring through `NumpyDocstring` (then optionally `GoogleDocstring`), modifying lines in place.",
      "Describes that `NumpyDocstring._is_section_header()` identifies section headers by checking if the section name (lowercased) is in `self._sections` AND the next line matches `_numpy_section_regex` (pattern `r'^[=\\-`:'\"~^_*+#<>]{2,}\\s*$'`).",
      "Notes that both `'yield'` and `'yields'` keys map to `self._parse_yields_section` in the `_sections` dictionary.",
      "Explains that `_parse_yields_section` calls `self._consume_returns_section(preprocess_types=True)` and then `self._format_fields(_('Yields'), fields)`, differing from `_parse_returns_section` which uses `:returns:` and `:rtype:` directives.",
      "Describes that `NumpyDocstring._consume_returns_section` delegates to `self._consume_fields(prefer_type=True)`, and that `prefer_type=True` causes the name to be swapped into the type position when no explicit type is found (`if prefer_type and not _type: _type, _name = _name, _type`).",
      "Mentions that `_format_field` handles the case where `_name` is empty but `_type` is set by italicizing the type (when no backtick is present)."
    ],
    "key_files": [
      "doc/usage/extensions/example_numpy.py",
      "sphinx/ext/napoleon/docstring.py",
      "sphinx/ext/napoleon/__init__.py"
    ],
    "source_doc": "[docstring: doc/usage/extensions/example_numpy.py] doc.usage.extensions.example_numpy.example_generator\nGenerators have a ``Yields`` section instead of a ``Returns`` section.\n\n    Parameters\n    ----------\n    n : int\n        The upper limit of the range to generate, from 0 to `n` - 1.\n\n    Yields\n    ------\n    int\n        The next number in the range of 0 to `n` - 1.\n\n    Examples\n    --------\n    Examples should be written in doctest format, and should illustrate how\n    to use the function.\n\n    >>> print([i for i in example_generator(4)])\n    [0, 1, 2, 3]",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are about code structure/logic verifiable from stripped source; a few rely on docstring content or stated purposes that are documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_14",
    "repo": "sphinx",
    "question": "Why does CAliasObject.run() completely reimplement its logic instead of calling the parent ObjectDescription.run() method?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "CAliasObject.run() does not call ObjectDescription.run() (its parent class method) because the alias mechanism has fundamentally different behavior that is incompatible with the ObjectDescription flow:\n\n1. **One-to-many signature expansion**: When the 'noroot' option is used, a single alias signature can expand into multiple real signatures. ObjectDescription.run() creates one signode per signature in a 1:1 mapping (one `desc_signature` per sig in the `for sig in signatures:` loop).\n\n2. **Deferred resolution via AliasNode**: Instead of calling handle_signature() to parse and render each signature immediately, CAliasObject creates AliasNode placeholder elements that are resolved later by the AliasTransform. ReferencesResolver has `default_priority = 10`, and the AliasTransform runs before it.\n\n3. **No indexing or cross-reference targets needed**: CAliasObject sets node['no-index'] = True, never creates an indexnode, and never calls add_target_and_index().\n\n4. **No content parsing**: Unlike ObjectDescription.run() (which calls before_content(), parse_content_to_nodes(), transform_content(), DocFieldTransformer, and after_content()), CAliasObject.run() skips all content processing entirely and returns just [node].\n\n5. **No signature handling callbacks**: ObjectDescription.run() calls handle_signature() which can raise ValueError for error handling, and uses _object_hierarchy_parts/_toc_entry_name for TOC integration. CAliasObject bypasses all of this since AliasTransform handles the actual rendering.",
    "rubric": [
      "Explains that a single alias signature can expand into multiple real signatures (one-to-many) when 'noroot' is used, which is incompatible with ObjectDescription's 1:1 signature-to-signode loop",
      "Mentions that CAliasObject uses AliasNode placeholder elements that are resolved later by AliasTransform (deferred resolution), rather than calling handle_signature() to parse/render immediately",
      "Notes that AliasTransform runs at a priority just below ReferencesResolver (priority - 1) to resolve before cross-references",
      "Points out that CAliasObject sets no-index to True and skips all indexing/cross-reference target creation (no add_target_and_index call)",
      "Mentions that CAliasObject skips all content parsing (no before_content, parse_content_to_nodes, transform_content, DocFieldTransformer, or after_content calls)"
    ],
    "key_files": [
      "sphinx/domains/c/__init__.py",
      "sphinx/directives/__init__.py",
      "sphinx/transforms/post_transforms/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/domains/c/__init__.py] sphinx.domains.c.CAliasObject\nsphinx.domains.c.CAliasObject.run:\n  On purpose this doesn't call the ObjectDescription version, but is based on it.\n        Each alias signature may expand into multiple real signatures if 'noroot'.\n        The code is therefore based on the ObjectDescription version.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that run() 'on purpose doesn't call the ObjectDescription version' and that 'each alias signature may expand into multiple real signatures if noroot'. The answer provides extensive implementation details (specific line numbers, class names like AliasTransform, AliasNode, priority numbers, specific method calls like before_content/after_content, node['no-index'] = True at line 682, etc.) that go far beyond what the documentation states.",
      "Claims about specific line numbers (251, 562-563, 682), transform priorities (9 vs 10), and detailed implementation specifics like AliasTransform._render_symbol() with skip_this=True are not verifiable from the provided documentation and could be hallucinated or inaccurate.",
      "The comparison between C's CAliasObject and C++ CPPAliasObject regarding before_content()/after_content() is not mentioned in the documentation at all.",
      "The claim about AliasNode placeholder elements and deferred resolution is plausible but not directly supported by the provided documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code structure; a few design rationale statements and cross-domain comparisons rely on documentation or unavailable code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_15",
    "repo": "sphinx",
    "question": "Where does declaration data flow in the CPPDomain, from directive parsing through cross-reference resolution to inventory export?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "In the CPPDomain, data flows through a Symbol tree structure rooted at `self.data['root_symbol']`. Here is the complete data/control flow path:\n\n1. **Directive \u2192 Symbol Tree (writing):** When a C++ directive (e.g., `cpp:function`) is processed, `CPPObject.run()` is called (which overrides `ObjectDescription.run()`). It calls `handle_signature()`, which uses `DefinitionParser.parse_declaration(self.object_type, self.objtype)` to produce an `ASTDeclaration`. The `ASTDeclaration` is constructed with both `objectType` (e.g., 'function', 'class') and `directiveType` (e.g., 'struct', 'enum-class'). Then `parent_symbol.add_declaration(ast, docname, line)` adds the declaration to the symbol tree.\n\n2. **Symbol Tree \u2192 Cross-references (reading):** During xref resolution, `CPPDomain._resolve_xref_inner()` retrieves the `cpp:parent_key` from the pending_xref node (set there by `CPPXRefRole.process_link()` which copies `env.ref_context` attributes into the node). The parent_key is a `LookupKey` containing a sequence of `(ASTNestedNameElement, template_params, newest_id)` tuples. The root symbol does `direct_lookup(parent_key)` to find the parent context, then calls either `parent_symbol.find_name()` (for shorthand xrefs) or `parent_symbol.find_declaration()` (for full declaration xrefs).\n\n3. **Resolution result:** `_resolve_xref_inner()` returns a tuple of `(reference_node, declaration.objectType)`. The `objectType` is then used in `resolve_any_xref()` to map back to a role name via `self.role_for_objtype(objtype)` (which uses the `_type2role` dict built in `Domain.__init__()` from `object_types`).\n\n4. **Inventory export:** `get_objects()` iterates all symbols via `root_symbol.get_all_symbols()` (a recursive generator over `_children`) and yields tuples containing `symbol.declaration.objectType` as the type field, using `declaration.get_newest_id()` as the anchor.\n\n5. **Parent context propagation:** The `cpp:parent_key` is stored in `env.ref_context` and updated during `before_content()`/`after_content()` lifecycle methods. It's also set by namespace directives (`CPPNamespaceObject`, `CPPNamespacePushObject`, `CPPNamespacePopObject`). The `ASTIdentifier.describe_signature()` in `_ast.py` (line 126) also directly sets `pnode['cpp:parent_key'] = symbol.get_lookup_key()` on pending_xref nodes created for 'markType' mode signatures.",
    "rubric": [
      "Directives use DefinitionParser to produce ASTDeclaration objects which are added to a Symbol tree via parent_symbol.add_declaration()",
      "ASTDeclaration stores both objectType and directiveType attributes",
      "Cross-reference resolution uses CPPDomain._resolve_xref_inner() which retrieves cpp:parent_key from the pending_xref node",
      "The parent_key is a LookupKey used with direct_lookup() on the root symbol to find the parent context",
      "Resolution calls find_name() for shorthand xrefs or find_declaration() for full declaration xrefs on the parent symbol",
      "_resolve_xref_inner() returns a tuple containing the declaration's objectType, which resolve_any_xref() maps to a role name",
      "get_objects() iterates symbols via root_symbol.get_all_symbols() and yields declaration.objectType and declaration.get_newest_id()",
      "Parent context (cpp:parent_key) is propagated through env.ref_context and updated in before_content()/after_content() and namespace directives"
    ],
    "key_files": [
      "sphinx/domains/cpp/__init__.py",
      "sphinx/domains/cpp/_symbol.py",
      "sphinx/domains/cpp/_ast.py",
      "sphinx/domains/cpp/_parser.py",
      "sphinx/directives/__init__.py",
      "sphinx/domains/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/domains/cpp/__init__.py] sphinx.domains.cpp.CPPDomain\nsphinx.domains.cpp.CPPDomain:\n  C++ language domain.\n\n    There are two 'object type' attributes being used::\n\n    - Each object created from directives gets an assigned .objtype from ObjectDescription.run.\n      This is simply the directive name.\n    - Each declaration (see the distinction in the directives dict below) has a nested .ast of\n      type ASTDeclaration. That object has .objectType which corresponds to the keys in the\n      object_types dict below. They are the core different types of declarations in C++ that\n      one can document.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (Symbol tree, LookupKey, direct_lookup, find_name, find_declaration, get_all_symbols, _children, env.ref_context, ASTIdentifier.describe_signature, etc.) that go far beyond what the provided documentation chunk describes. These cannot be verified from the given documentation alone.",
      "The documentation only describes two 'object type' attributes: .objtype from ObjectDescription.run (directive name) and .objectType from ASTDeclaration (corresponding to object_types dict keys). The answer's claims about these two attributes are consistent with the documentation.",
      "The answer mentions DefinitionParser.parse_declaration taking 'self.object_type' and 'self.objtype' as parameters, and ASTDeclaration being constructed with 'objectType' and 'directiveType' - these are plausible but unverifiable from the given doc chunk.",
      "The detailed flow descriptions (steps 1-5) are plausible extrapolations based on Sphinx domain architecture knowledge but cannot be confirmed or denied from the provided documentation snippet alone."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code \u2014 no documentation leakage detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_16",
    "repo": "sphinx",
    "question": "How does the Python domain implement cross-reference resolution, including object registration, lookup priorities, and fallback chains?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `PythonDomain` class (name='py', label='Python') implements cross-referencing for Python objects through three key methods: `note_object`, `note_module`, and `find_obj`.\n\n## How `note_object` works:\n\n`note_object` stores objects in a dictionary (`self.objects`) keyed by fullname, mapping to `ObjectEntry` NamedTuples with fields `(docname, node_id, objtype, aliased)`. It implements priority logic for aliased vs. canonical names:\n1. If a name already exists and the existing entry is aliased but the new one is not (`aliased=False`), the new (canonical) definition overrides the old one.\n2. If the existing entry is canonical and the new one is aliased, the new aliased entry is silently skipped (returns early).\n3. If both have the same aliased status, a duplicate warning is logged.\n\nIt is called from `PyObject.add_target_and_index()` (in `_object.py`), which constructs the fullname by prepending the module name. If the directive has a `canonical` option (and is not a `type` directive), `note_object` is called a second time with `aliased=True` for the canonical name.\n\n## How `note_module` works:\n\n`note_module` stores module entries in `self.modules` as `ModuleEntry` NamedTuples with fields `(docname, node_id, synopsis, platform, deprecated)`. It is called from `PyModule.run()`, which also calls `note_object(modname, 'module', node_id)` so the module is findable via the general object lookup.\n\n## How `find_obj` works (the core cross-reference resolution algorithm):\n\n`find_obj` has two search modes controlled by `searchmode`:\n\n**searchmode=0 (exact match):** Object type is NOT considered. It checks in this priority order:\n1. Exact match on `name` itself\n2. If type is 'mod', return empty (only exact matches for modules)\n3. `classname + '.' + name`\n4. `modname + '.' + name`\n5. `modname + '.' + classname + '.' + name`\n\n**searchmode=1 (refspecific, triggered by leading dot in reference):** Object type IS considered via `objtypes_for_role()`. Priority order:\n1. `modname + '.' + classname + '.' + name` (most specific)\n2. `modname + '.' + name`\n3. `name` alone\n4. Fuzzy search: finds all objects whose name ends with `.{name}` (suffix match)\n\nThe method first strips trailing `()` from the name. When `type` is None (as in `:any:` role), all object types are considered.\n\n## How `resolve_xref` uses `find_obj`:\n\n`resolve_xref` determines `searchmode` from `node.hasattr('refspecific')` (set by `PyXRefRole.process_link()` when target starts with `.`). It implements fallback chains:\n- `:class:` falls back to `data`, then `attr`\n- `:attr:` falls back to `meth`\n- `:meth:` falls back to `_prop`\n\nWhen multiple matches exist, canonical (non-aliased) entries are preferred. If exactly one canonical match exists among multiple results, it's chosen; otherwise a warning is logged.\n\nThe `_prop` role is defined via `'property': ObjType(_('property'), 'attr', '_prop', 'obj')` in `object_types` - this means `_prop` role maps to the `property` object type, enabling the meth\u2192property fallback.\n\nIn `get_objects()`, aliased entries get search priority -1, while canonical entries get priority 1, and modules get priority 0.",
    "rubric": [
      "Explains that `note_object` stores entries in `self.objects` as ObjectEntry tuples (docname, node_id, objtype, aliased) keyed by fullname",
      "Describes the aliased vs canonical priority logic in `note_object`: canonical overrides aliased, aliased is skipped if canonical exists, duplicates produce a warning",
      "Explains that `note_module` stores entries in `self.modules` as ModuleEntry tuples and that `note_object` is also called for modules",
      "Describes `find_obj` searchmode=0 (exact match): checks name, classname.name, modname.name, modname.classname.name \u2014 does NOT consider object type",
      "Describes `find_obj` searchmode=1 (refspecific): checks modname.classname.name, modname.name, name, then fuzzy suffix search \u2014 DOES filter by object type",
      "Explains the fallback chains in `resolve_xref`: class\u2192data\u2192attr, attr\u2192meth, meth\u2192_prop",
      "Mentions that when multiple matches exist, canonical (non-aliased) entries are preferred \u2014 if exactly one canonical exists it is chosen",
      "Explains that searchmode is determined by `node.hasattr('refspecific')` which is set when the reference target starts with a dot"
    ],
    "key_files": [
      "sphinx/domains/python/__init__.py",
      "sphinx/domains/python/_object.py",
      "sphinx/domains/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/domains/python/__init__.py] sphinx.domains.python.PythonDomain\nsphinx.domains.python.PythonDomain:\n  Python language domain.\n\nsphinx.domains.python.PythonDomain.note_object:\n  Note a python object for cross reference.\n\n        .. versionadded:: 2.1\n\nsphinx.domains.python.PythonDomain.note_module:\n  Note a python module for cross reference.\n\n        .. versionadded:: 2.1\n\nsphinx.domains.python.PythonDomain.find_obj:\n  Find a Python object for \"name\", perhaps using the given module\n        and/or classname.  Returns a list of (name, object entry) tuples.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few parenthetical explanations about design rationale were documentation-only.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_13",
    "repo": "sphinx",
    "question": "What is the architecture of Sphinx's ANSI escape sequence stripping system \u2014 what patterns does it match, how are colour codes generated, and where is stripping used?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `strip_escape_sequences` function in `sphinx/_cli/util/errors.py` is a simple wrapper around a compiled regex substitution. It uses the module-level `_ANSI_CODES` regex pattern to remove ANSI CSI escape sequences from text by calling `_ANSI_CODES.sub('', text)`.\n\n**Architecture and regex details:**\n- The `_CSI` constant is defined as `re.escape('\\x1b[')`, though it's defined but not directly used in the `_ANSI_CODES` pattern.\n- The `_ANSI_CODES` regex pattern (compiled with `re.VERBOSE | re.ASCII` flags) matches two types of sequences:\n  1. Color/style codes: `\\x1b[` followed by up to two groups of `digits;` and then an optional digit sequence ending with `m` (e.g., `\\x1b[39;49;00m`, `\\x1b[0m`, `\\x1b[m`)\n  2. Erase sequences: `\\x1b[` followed by an optional `0`, `1`, or `2` and `K` (e.g., `\\x1b[2K`, `\\x1b[K`)\n\n**How colour codes are generated (what gets stripped):**\nIn `sphinx/_cli/util/colour.py`, the `_create_colour_func` factory creates colour functions that wrap text with `\\x1b[{escape_code}m` prefix and `\\x1b[39;49;00m` suffix (the reset sequence). Sphinx defines 22 colour functions this way (reset, bold, faint, standout, underline, blink, black, darkred, darkgreen, brown, darkblue, purple, turquoise, lightgray, darkgray, red, green, yellow, blue, fuchsia, teal, white).\n\n**Key consumers in the architecture:**\n1. `full_exception_context()` in the same module calls `strip_escape_sequences` to clean message log entries before writing them to error reports: `f'    {strip_escape_sequences(msg)}'.rstrip()`\n2. `TeeStripANSI` class in `sphinx/util/_io.py` uses it to write stripped text to a file stream while passing colored text to the terminal stream.\n3. `term_width_line()` in `sphinx/util/console.py` uses it to calculate the visible (non-escape) length of text for terminal padding: `len(text) - len(strip_escape_sequences(text))`.\n4. `sphinx/util/console.py` aliases it as `strip_colors = strip_escape_sequences`.\n\n**What is NOT stripped:**\nThe CURSOR_UP sequence (`\\x1b[2A`) is explicitly *not* stripped, as confirmed by the test file where `CURSOR_UP` is left intact after processing. Only color/style (`m`-terminated) and erase (`K`-terminated) sequences are removed.\n\n**The colour system:**\nColour functions use escape codes like '91' (red), '94' (blue), '39;49;00' (reset). On non-Windows platforms, there's also a `_create_input_mode_colour_func` that wraps escape codes in `\\1` and `\\2`. The colouring can be globally disabled via `_COLOURING_DISABLED` flag, toggled by `disable_colour()`/`enable_colour()`.",
    "rubric": [
      "Explains that strip_escape_sequences uses a compiled regex (_ANSI_CODES) to substitute matched sequences with empty string via _ANSI_CODES.sub('', text)",
      "Describes the _ANSI_CODES regex pattern matches two types: color/style codes (\\x1b[ followed by up to two digit; groups and optional digits ending with 'm') and erase-in-line sequences (\\x1b[ followed by optional 0/1/2 and 'K')",
      "Mentions the regex uses re.VERBOSE | re.ASCII flags",
      "Explains that colour functions are generated by _create_colour_func which wraps text with \\x1b[{escape_code}m prefix and \\x1b[39;49;00m reset suffix",
      "Identifies that TeeStripANSI in sphinx/util/_io.py uses strip_escape_sequences to write stripped text to a file stream while passing colored text to terminal",
      "Identifies that term_width_line in sphinx/util/console.py uses it to calculate visible text length for terminal padding",
      "Identifies that full_exception_context uses it to clean message log entries before writing error reports",
      "Notes that CURSOR_UP (\\x1b[2A) is NOT stripped \u2014 only 'm'-terminated and 'K'-terminated sequences are removed"
    ],
    "key_files": [
      "sphinx/_cli/util/errors.py",
      "sphinx/_cli/util/colour.py",
      "sphinx/util/_io.py",
      "sphinx/util/console.py",
      "tests/test__cli/test__cli_util_errors.py"
    ],
    "source_doc": "[docstring: sphinx/_cli/util/errors.py] sphinx._cli.util.errors.strip_escape_sequences\nRemove the ANSI CSI colors and \"erase in line\" sequences.\n\n    Other `escape sequences <https://en.wikipedia.org/wiki/ANSI_escape_code>`_\n    (e.g., VT100-specific functions) are not supported. Only control sequences\n    *natively* known to Sphinx (i.e., colour sequences used in Sphinx\n    and \"erase entire line\" (``'\\x1b[2K'``)) are stripped by this function.\n\n    .. warning:: This function only for use within Sphinx..\n\n    __ https://en.wikipedia.org/wiki/ANSI_escape_code",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: 'ANSI SGR', 'ANSI Erase in Line (EL)' labels from stripped comments, 'readline compatibility' purpose, and an unverifiable claim about sphinx/util/__init__.py.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_17",
    "repo": "sphinx",
    "question": "What is the architecture and behavior of the PyObject base class in Sphinx's Python domain?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `PyObject` class in `sphinx/domains/python/_object.py` is the central base class for describing Python language constructs in Sphinx's Python domain. Here is a detailed breakdown of its architecture and behavior based on the actual source code:\n\n**Class Hierarchy:**\n`PyObject` inherits from `ObjectDescription[tuple[str, str]]`, which itself inherits from `SphinxDirective` (which extends docutils' `Directive`). The generic type parameter `tuple[str, str]` represents `(fullname, classname_prefix)` \u2014 the return value of `handle_signature()`.\n\n**Key Design Details:**\n\n1. **`allow_nesting` defaults to `False`** in `PyObject`. Only `PyClasslike` overrides this to `True`, enabling classes/exceptions to nest other objects inside them (as seen in `before_content()`/`after_content()` logic).\n\n2. **`option_spec`** includes: `'no-index'`, `'no-index-entry'`, `'no-contents-entry'`, `'no-typesetting'`, `'noindex'`, `'noindexentry'`, `'nocontentsentry'`, `'single-line-parameter-list'`, `'single-line-type-parameter-list'`, `'module'`, `'canonical'`, and `'annotation'`.\n\n3. **`handle_signature()` uses a regex `py_sig_re`** that captures 5 groups: class name(s) with trailing dot, thing name, optional type parameters list (in square brackets), optional arguments (in parentheses), and optional return annotation (after `->`).\n\n4. **Multi-line parameter list logic**: The method calculates whether to use multi-line formatting by subtracting the type parameter list length from the total signature length and comparing against `python_maximum_signature_line_length` or `maximum_signature_line_length` config values.\n\n5. **`get_signature_prefix()` validation**: If the method returns a string instead of a list of nodes, a `TypeError` is raised with a descriptive message.\n\n6. **Fallback parsing**: When `_parse_arglist()` raises a `SyntaxError`, the code falls back to `_pseudo_parse_arglist()`. `NotImplementedError` and `ValueError` also trigger the fallback.\n\n7. **`doc_field_types`** defines 5 field types: `PyTypedField` for parameters (with names 'param', 'parameter', 'arg', 'argument', 'keyword', 'kwarg', 'kwparam'), `PyTypedField` for variables ('var', 'ivar', 'cvar'), `PyGroupedField` for exceptions ('raises', 'raise', 'exception', 'except'), a plain `Field` for return values, and `PyField` for return types.\n\n8. **`PyXrefMixin`**: Used by `PyField`, `PyGroupedField`, and `PyTypedField` to make cross-references behave like `:class:` links, supporting `\".\"` and `\"~\"` prefixes. It has a `_delimiters_re` pattern that splits on brackets, parentheses, commas, `or`, `of`, pipes (`|`), and ellipsis (`...`).\n\n9. **`before_content()` module handling**: If the directive has a `module` option, it pushes the current module onto a `'py:modules'` stack and sets the new module. `after_content()` pops it back.\n\n10. **Subclass directives registered in `PythonDomain`**: `PyFunction` (for 'function'), `PyVariable` (for 'data'), `PyClasslike` (for 'class' and 'exception'), `PyMethod` (for 'method'), `PyAttribute` (for 'attribute'), `PyProperty` (for 'property'), and `PyTypeAlias` (for 'type').\n\n11. **`_toc_entry_name()`** appends `'()'` to function/method entries when `add_function_parentheses` config is True, and respects `toc_object_entries_show_parents` with values 'domain', 'hide', or 'all'.\n\n12. **Config values** used throughout the code: `python_use_unqualified_type_names`, `python_maximum_signature_line_length`, `python_trailing_comma_in_multi_line_signatures`, and `python_display_short_literal_types`.",
    "rubric": [
      "Identifies that PyObject inherits from ObjectDescription[tuple[str, str]] which extends SphinxDirective",
      "Explains that allow_nesting defaults to False and only PyClasslike overrides it to True",
      "Lists key option_spec entries including 'single-line-parameter-list', 'module', 'canonical', 'annotation', and no-index variants",
      "Describes that handle_signature() uses py_sig_re regex to parse class prefix, name, type parameters, arguments, and return annotation",
      "Explains multi-line parameter list logic involving python_maximum_signature_line_length or maximum_signature_line_length config values",
      "Mentions fallback from _parse_arglist() to _pseudo_parse_arglist() on SyntaxError/NotImplementedError/ValueError",
      "Describes doc_field_types including typed fields for parameters and variables, grouped field for exceptions, and fields for return value/type",
      "Explains PyXrefMixin's role in making cross-references behave like :class: links with support for '.' and '~' prefixes and its delimiter regex",
      "Describes before_content()/after_content() module stack push/pop behavior",
      "Lists subclass directives like PyFunction, PyVariable, PyClasslike, PyMethod, PyAttribute, PyProperty, PyTypeAlias",
      "Explains _toc_entry_name() behavior with add_function_parentheses and toc_object_entries_show_parents config",
      "Mentions relevant config values like python_use_unqualified_type_names, python_maximum_signature_line_length, python_trailing_comma_in_multi_line_signatures"
    ],
    "key_files": [
      "sphinx/domains/python/_object.py",
      "sphinx/domains/python/__init__.py",
      "sphinx/directives/__init__.py",
      "sphinx/domains/python/_annotations.py"
    ],
    "source_doc": "[docstring: sphinx/domains/python/_object.py] sphinx.domains.python._object.PyObject\nsphinx.domains.python._object.PyObject:\n  Description of a general Python object.\n\n    :cvar allow_nesting: Class is an object that allows for nested namespaces\n    :vartype allow_nesting: bool\n\nsphinx.domains.python._object.PyObject.get_signature_prefix:\n  May return a prefix to put before the object name in the\n        signature.\n\nsphinx.domains.python._object.PyObject.needs_arglist:\n  May return true if an empty argument list is to be generated even if\n        the document contains none.\n\nsphinx.domains.python._object.PyObject.handle_signature:\n  Transform a Python signature into RST nodes.\n\n        Return (fully qualified name of the thing, classname if any).\n\n        If inside a class, the current class name is handled intelligently:\n        * it is stripped from the displayed name if present\n        * it is added to the full name (return value) if not present\n\nsphinx.domains.python._object.PyObject.get_index_text:\n  Return the text for the index entry of the object.\n\nsphinx.domains.python._object.PyObject.before_content:\n  Handle object nesting before content\n\n        :py:class:`PyObject` represents Python language constructs. For\n        constructs that are nestable, such as a Python classes, this method will\n        build up a stack of the nesting hierarchy so that it can be later\n        de-nested correctly, in :py:meth:`after_content`.\n\n        For constructs that aren't nestable, the stack is bypassed, and instead\n        only the most recent object is tracked. This object prefix name will be\n        removed with :py:meth:`after_content`.\n\nsphinx.domains.python._object.PyObject.after_content:\n  Handle object de-nesting after content\n\n        If this class is a nestable object, removing the last nested class prefix\n        ends further nesting in the object.\n\n        If this class is not a nestable object, the list of classes should not\n        be altered as we didn't affect the nesting levels in\n        :py:meth:`before_content`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer go far beyond what the provided documentation states, referencing specific line numbers, config values, regex patterns, option specs, and implementation details that cannot be verified from the given docstrings alone.",
      "The core claims that ARE verifiable against the documentation (allow_nesting is a bool class variable, handle_signature returns (fully qualified name, classname), before_content handles nesting, after_content handles de-nesting) are consistent with the documentation.",
      "Claims about specific line numbers (e.g., 'line 234', 'line 186 in __init__.py') cannot be verified from the documentation provided.",
      "The answer states allow_nesting defaults to False, which is plausible given the documentation says it controls nested namespaces, but this specific default value is not stated in the provided docs.",
      "Claims about 13 options, specific regex patterns, config values, etc. are neither supported nor contradicted by the documentation - they appear to come from actual source code inspection but cannot be verified here."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor adjustments needed for line number references and documentation-only details about bracket notation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_20",
    "repo": "sphinx",
    "question": "How does the `AutodocDirective.run()` method process its options, generate content, and parse the result into document nodes?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `AutodocDirective.run()` method implements the following algorithm:\n\n1. **Get source info**: Calls `self.get_source_info()` (inherited from `SphinxDirective`) to get the current file source and line number for debugging/logging.\n\n2. **Determine object type**: Strips the `'auto'` prefix from `self.name` (e.g., `'autoclass'` \u2192 `'class'`, `'automodule'` \u2192 `'module'`) by slicing `self.name[4:]`. It asserts `self.name.startswith('auto')`.\n\n3. **Set titles_allowed = True**: This flag is unconditionally set to True.\n\n4. **Process options**: Calls `_process_documenter_options()` with the object type, `autodoc_default_options` from config, and user-supplied `self.options`. This function:\n   - Looks up the correct `_OPTION_SPECS` mapping for the object type (e.g., module, class, function).\n   - Iterates through `AUTODOC_DEFAULT_OPTIONS` to merge defaults, handling negation (e.g., `:no-members:` pops the negated flag), extension with `+` prefix for extendable options (members, private-members, special-members, exclude-members), and removal of `+` when there's nothing to merge.\n   - Uses `docutils.utils.assemble_option_dict` to validate and transform options.\n   - Returns an `_AutoDocumenterOptions` instance (created via `from_directive_options()` which replaces hyphens with underscores in option keys).\n   - On error (KeyError, ValueError, TypeError), logs an error and returns an empty list.\n   - The `_tab_width` attribute on the options object is set to `self.state.document.settings.tab_width`.\n\n5. **Generate content**: Calls `_auto_document_object()` which:\n   - Calls `_load_object_by_name()` to load the object.\n   - If load fails (props is None), returns None.\n   - Creates a `StringList` result and calls `_generate_directives()` which:\n     - Gets or creates a `ModuleAnalyzer` and records source file dependencies.\n     - Checks `__module__` for imported-object filtering.\n     - Generates reST directive header lines and docstring content via `_add_directive_lines()`.\n     - Recursively documents members via `_document_members()` \u2192 `_gather_members()` \u2192 recursive `_generate_directives()` calls.\n\n6. **Parse generated content**: If content is non-empty, calls `parse_generated_content()` which:\n   - Uses `switch_source_input()` context manager that temporarily replaces the reporter's `get_source_and_line()` method with one backed by a new `StateMachine` whose `input_lines` is the generated content.\n   - If `titles_allowed` is True: calls `nested_parse_to_nodes(state, content)`.\n   - If False: creates a `nodes.paragraph()`, sets its document, and uses `state.nested_parse(content, 0, node, match_titles=False)`, returning the paragraph's children.\n\nThe `DummyOptionSpec` class (used as `option_spec`) is a dict subclass that always returns True for `__bool__` and returns `lambda x: x` for any key access via `__getitem__`.",
    "rubric": [
      "Mentions stripping the 'auto' prefix from self.name (e.g., self.name[4:]) to determine the object type",
      "Explains that _process_documenter_options() merges autodoc_default_options with user-supplied options, handling negation (no-X flags) and extendable options (+ prefix)",
      "Notes that assemble_option_dict is used to validate/transform options, and from_directive_options replaces hyphens with underscores in option keys",
      "Mentions that titles_allowed is unconditionally set to True",
      "Describes that _auto_document_object() is called to generate reST content as a StringList",
      "Explains that parse_generated_content() uses switch_source_input and branches on titles_allowed: nested_parse_to_nodes if True, or nested_parse into a paragraph node if False",
      "Mentions that on option processing errors (KeyError, ValueError, TypeError), an error is logged and an empty list is returned",
      "Notes that DummyOptionSpec is a dict subclass that always returns True for __bool__ and returns an identity lambda for any key access"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_directive.py",
      "sphinx/ext/autodoc/_directive_options.py",
      "sphinx/ext/autodoc/_generate.py",
      "sphinx/ext/autodoc/_shared.py",
      "sphinx/util/docutils.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_directive.py] sphinx.ext.autodoc._directive.AutodocDirective\nsphinx.ext.autodoc._directive.AutodocDirective:\n  A directive class for all autodoc directives.\n\n    It generates the directive lines for the given object,\n    then parses and returns the generated content.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extremely detailed implementation specifics (e.g., _process_documenter_options internals, _auto_document_object, _load_object_by_name, parse_generated_content details, DummyOptionSpec behavior) that are not supported by the provided documentation chunk, which only contains a brief docstring describing the class at a high level.",
      "While none of the claims directly contradict the documentation, the vast majority of the answer's content cannot be verified against the provided documentation - it appears to be based on source code knowledge rather than the given doc chunk.",
      "The documentation only states that AutodocDirective 'generates the directive lines for the given object, then parses and returns the generated content' - the answer's detailed algorithm is plausible but entirely unsupported by the provided text."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in purpose descriptions and details about unshown function internals.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_18",
    "repo": "sphinx",
    "question": "Why does ConfigurationValue subclass ObjectDescription directly instead of GenericObject, and what does its transform_content method accomplish?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `ConfigurationValue` class is a directive class for documenting configuration values in Sphinx's standard domain. It subclasses `ObjectDescription[str]`, and its `transform_content` method converts the `:type:` and `:default:` directive options into a docutils field list that is **prepended** (inserted at position 0) to the content node, ensuring structured metadata appears before any user-written description.\n\n**How `transform_content` works:**\n\n1. **Structured metadata injection via field lists**: `transform_content` automatically converts the `:type:` and `:default:` options into `nodes.field` entries within a `nodes.field_list`. The options are parsed as inline RST via `self.parse_inline`, which delegates to `self.state.inline_text(text, lineno)`, supporting cross-references, markup, and roles. The field names are internationalized using Sphinx's `_()` translation function (translated to `'Type'` and `'Default'`).\n\n2. **Ordering relative to event emission and DocFieldTransformer**: Looking at the parent class `ObjectDescription.run()` (in `sphinx/directives/__init__.py`), `transform_content` is called *after* content parsing but *before* the `object-description-transform` event and before `DocFieldTransformer(self).transform_all(content_node)`.\n\n3. **Relationship to `GenericObject`**: Unlike `GenericObject` (which `EnvVar` subclasses), `ConfigurationValue` directly subclasses `ObjectDescription[str]` rather than `GenericObject`. `ConfigurationValue` has custom `option_spec` entries (`:type:` and `:default:`) that `GenericObject` doesn't define. It also always uses `'pair'` as the index entry type (hardcoded in `add_target_and_index`), whereas `GenericObject` defaults to `'single'` and parses the index template to determine the type.\n\n4. **`option_spec` differences from parent**: `ConfigurationValue` defines its own `option_spec` with only the hyphenated option names (`no-index`, `no-index-entry`, `no-contents-entry`, `no-typesetting`) plus `type` and `default`, omitting the `noindex`, `noindexentry`, and `nocontentsentry` options that the parent `ObjectDescription` still includes.\n\n5. **`parse_inline` delegation**: The `format_type` and `format_default` methods both call `self.parse_inline(text, lineno=self.lineno)`, which is defined in `SphinxDirective` (in `sphinx/util/docutils.py`). This method delegates to `self.state.inline_text(text, lineno)`.",
    "rubric": [
      "Explains that ConfigurationValue subclasses ObjectDescription[str] rather than GenericObject",
      "Mentions that ConfigurationValue defines custom option_spec entries (:type: and :default:) not present in GenericObject",
      "Explains that transform_content converts :type: and :default: options into a docutils field_list prepended (inserted at position 0) to the content node",
      "Notes that the field names are internationalized using the _() translation function (translated to 'Type' and 'Default')",
      "Mentions that option values are parsed as inline RST via parse_inline, supporting cross-references and markup",
      "Describes the timing: transform_content is called after content parsing but before the object-description-transform event and before DocFieldTransformer runs",
      "Notes that ConfigurationValue hardcodes 'pair' as the index entry type, while GenericObject determines index type dynamically from its template"
    ],
    "key_files": [
      "sphinx/domains/std/__init__.py",
      "sphinx/directives/__init__.py",
      "sphinx/util/docutils.py"
    ],
    "source_doc": "[docstring: sphinx/domains/std/__init__.py] sphinx.domains.std.ConfigurationValue\nsphinx.domains.std.ConfigurationValue.transform_content:\n  Insert *type* and *default* as a field list.\n\nsphinx.domains.std.ConfigurationValue.format_type:\n  Formats the ``:type:`` option.\n\nsphinx.domains.std.ConfigurationValue.format_default:\n  Formats the ``:default:`` option.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that transform_content 'Insert *type* and *default* as a field list' \u2014 the answer's claim that it is 'prepended (inserted at position 0)' is a plausible extrapolation from code but not directly supported by the provided documentation.",
      "The documentation only states that format_type 'Formats the :type: option' and format_default 'Formats the :default: option' \u2014 the detailed claims about parse_inline, self.state.inline_text, and SphinxDirective line numbers are extrapolations from source code not provided in the documentation.",
      "Claims about GenericObject, EnvVar, ObjectDescription inheritance, option_spec contents, deprecated options, add_target_and_index behavior, StandardDomain registration details, and ObjType label translation are all beyond the scope of the provided documentation \u2014 they may be accurate based on source code but cannot be verified from the given doc chunk.",
      "The claim about ordering relative to 'object-description-transform' event and DocFieldTransformer, and specific line numbers in sphinx/directives/__init__.py, cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; some design rationale ('deliberately', 'convenience pattern', 'because') and line numbers constitute minor doc leakage.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_19",
    "repo": "sphinx",
    "question": "Where are environment collectors defined, registered, and how do they connect to Sphinx's event system?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `EnvironmentCollector` class is located in `sphinx/environment/collectors/__init__.py` and serves as a base class for environment collectors (with methods raising `NotImplementedError`). When registered via `app.add_env_collector()`, the collector is instantiated and its `enable()` method is called, which connects five event handlers to the Sphinx event system.\n\nThe `enable()` method connects these five events:\n1. `'doctree-read'` \u2192 `self.process_doc`\n2. `'env-merge-info'` \u2192 `self.merge_other`\n3. `'env-purge-doc'` \u2192 `self.clear_doc`\n4. `'env-get-updated'` \u2192 `self.get_updated_docs`\n5. `'env-get-outdated'` \u2192 `self.get_outdated_docs`\n\nThe collector IDs are stored in `self.listener_ids` (a `dict[str, int] | None` class variable initially set to `None`), which maps event names to listener IDs returned by `app.connect()`. The `disable()` method iterates over these listener IDs and calls `app.disconnect()` on each, then resets `listener_ids` to `None`.\n\nThe built-in collectors are registered as builtin extensions in `sphinx/application.py` in the `builtin_extensions` tuple, loaded in a specific order:\n1. `sphinx.environment.collectors.dependencies` \u2192 `DependenciesCollector`\n2. `sphinx.environment.collectors.asset` \u2192 `ImageCollector` and `DownloadFileCollector`\n3. `sphinx.environment.collectors.metadata` \u2192 `MetadataCollector`\n4. `sphinx.environment.collectors.title` \u2192 `TitleCollector`\n5. `sphinx.environment.collectors.toctree` \u2192 `TocTreeCollector`\n\nThe events are emitted from different locations:\n- `'env-purge-doc'` is emitted from `sphinx/builders/__init__.py` in the `read()` method\n- `'env-get-outdated'` is emitted from `sphinx/builders/__init__.py` in the `read()` method\n\nThe `app.connect()` method delegates to `self.events.connect()` on the `EventManager` (in `sphinx/events.py`), which creates an `EventListener` named tuple with `(id, handler, priority)` and appends it to `self.listeners[name]`. The default priority is 500. The `EventManager.emit()` method sorts listeners by priority (ascending) before invoking them.",
    "rubric": [
      "Identifies that `EnvironmentCollector` base class is in `sphinx/environment/collectors/__init__.py`",
      "Mentions the `enable()` method connects five events: doctree-read, env-merge-info, env-purge-doc, env-get-updated, env-get-outdated",
      "Explains that listener IDs are stored in `self.listener_ids` (a dict mapping event names to listener IDs) and `disable()` disconnects them",
      "Lists the built-in collectors registered in `sphinx/application.py`'s `builtin_extensions` (dependencies, asset, metadata, title, toctree)",
      "Explains that `app.connect()` delegates to `EventManager.connect()` which creates an `EventListener` namedtuple and appends it to `self.listeners[name]`",
      "Mentions that events are emitted from `sphinx/builders/__init__.py` (e.g., env-purge-doc and env-get-outdated in the `read()` method)"
    ],
    "key_files": [
      "sphinx/environment/collectors/__init__.py",
      "sphinx/application.py",
      "sphinx/events.py",
      "sphinx/environment/collectors/toctree.py",
      "sphinx/environment/collectors/asset.py",
      "sphinx/builders/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/environment/collectors/__init__.py] sphinx.environment.collectors.EnvironmentCollector\nsphinx.environment.collectors.EnvironmentCollector:\n  An EnvironmentCollector is a specific data collector from each document.\n\n    It gathers data and stores :py:class:`BuildEnvironment\n    <sphinx.environment.BuildEnvironment>` as a database.\n    Examples of specific data would be images, download files, section titles, metadatas, index\n    entries and toctrees, etc.\n\n    .. note::\n\n        This class essentially wraps a sub-set of :ref:`Sphinx event callbacks <events>`.\n\nsphinx.environment.collectors.EnvironmentCollector.clear_doc:\n  Remove specified data of a document.\n\n        This method is called on the removal of the document.\n\n        .. seealso:: :event:`env-purge-doc`\n\nsphinx.environment.collectors.EnvironmentCollector.merge_other:\n  Merge in specified data regarding docnames from a different `BuildEnvironment`\n        object which coming from a subprocess in parallel builds.\n\n        .. seealso:: :event:`env-merge-info`\n\nsphinx.environment.collectors.EnvironmentCollector.process_doc:\n  Process a document and gather specific data from it.\n\n        This method is called after the document is read.\n\n        .. seealso:: :event:`doctree-read`\n\nsphinx.environment.collectors.EnvironmentCollector.get_updated_docs:\n  Return a list of docnames to re-read.\n\n        This method is called after reading the whole of documents.\n\n        .. seealso:: :event:`env-get-updated`\n\nsphinx.environment.collectors.EnvironmentCollector.get_outdated_docs:\n  Return a list of docnames to re-read.\n\n        This method is called before reading the documents.\n\n        .. seealso:: :event:`env-get-outdated`",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, method signatures, class variables, built-in collector lists, event emission locations, EventManager internals) that are not present in the provided documentation chunk. These cannot be verified against the given documentation but are plausible code-level details.",
      "The documentation states get_updated_docs is called 'after reading the whole of documents' and maps to 'env-get-updated', while get_outdated_docs is called 'before reading the documents' and maps to 'env-get-outdated'. The answer correctly maps these events.",
      "The answer's claim about enable() connecting five specific events is consistent with the documentation's association of methods to events via seealso references.",
      "The answer mentions app.add_env_collector() at line 1744, listener_ids as dict[str, int] | None, disable() method, specific built-in collectors and their registration order, event emission locations with specific line numbers, EventManager internals, EventListener named tuple, and priority sorting \u2014 none of these can be verified from the provided documentation chunk, but they don't contradict it either."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims rely on line numbers, comments, and files not shown in the stripped code; most core structural claims are verifiable from the source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_22",
    "repo": "sphinx",
    "question": "Why does `_ensure_annotations_from_type_comments` use a WeakSet cache, and how does it prevent repeated work?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `_ensure_annotations_from_type_comments` function uses a `WeakSet` named `_objects_with_type_comment_annotations` to avoid redundantly processing the same object multiple times. This is important for performance because:\n\n1. **Early return on cached objects**: The function checks `if obj in _objects_with_type_comment_annotations: return` at the very top, ensuring that once an object (module or class) has had its annotations updated from type comments, the expensive analysis work is never repeated.\n\n2. **Avoiding redundant MRO traversal**: When called with a class type, the function iterates through the entire MRO (via `inspect.getmro(obj)`) and recursively calls itself for each parent class's module. Without the cache, documenting multiple classes from the same module hierarchy would repeatedly trigger full source code parsing for each module.\n\n3. **Avoiding repeated `ModuleAnalyzer` parsing**: The `_update_module_annotations_from_type_comments` function (called for module objects) invokes `ModuleAnalyzer.for_module(mod.__name__)` followed by `analyzer.analyze()`. While `ModuleAnalyzer` has its own cache (`cls.cache` dict keyed by `('module', modname)`) and `analyze()` has a short-circuit via `self._analyzed` flag, the `_update_module_annotations_from_type_comments` also does post-processing work (iterating annotations, resolving class attributes via `safe_getattr`, calling `dict(inspect.getannotations(cls))`, and assigning `__annotations__`). The WeakSet cache prevents all of this work from being repeated.\n\n4. **WeakSet chosen over regular set**: A `WeakSet` is used (imported from `weakref`) so that cached references don't prevent garbage collection of modules/classes that are no longer needed.\n\n5. **`contextlib.suppress(TypeError)` around `.add()`**: Some objects (e.g., unhashable ones) cannot be added to a WeakSet. The function suppresses the TypeError to handle this gracefully, meaning such objects will be re-processed on every call but won't crash.\n\nThe function is called from `_loader.py` in two explicit places\u2014for 'data' objects and 'attribute' objects\u2014and implicitly via the recursive MRO path, all gated by the `config.autodoc_use_type_comments` config flag. The `annotations.setdefault(attrname, annotation)` call ensures that inline annotations (which are already present) take priority over type comments, avoiding overwriting more precise type information.",
    "rubric": [
      "Explains that the WeakSet `_objects_with_type_comment_annotations` enables an early return check (`if obj in ...: return`) to avoid reprocessing the same object",
      "Mentions that without the cache, processing a class would repeatedly trigger MRO traversal and recursive calls for each parent class's module",
      "Explains that it avoids repeated invocations of `ModuleAnalyzer.for_module` / `analyzer.analyze()` and the subsequent annotation post-processing work in `_update_module_annotations_from_type_comments`",
      "Notes that WeakSet (rather than a regular set) is used so cached references don't prevent garbage collection of modules/classes",
      "Mentions that `contextlib.suppress(TypeError)` around `.add()` handles unhashable objects gracefully (they'll be re-processed but won't crash)"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_dynamic/_type_comments.py",
      "sphinx/ext/autodoc/_dynamic/_loader.py",
      "sphinx/pycode/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_dynamic/_type_comments.py] sphinx.ext.autodoc._dynamic._type_comments._ensure_annotations_from_type_comments\nEnsures `obj.__annotations__` includes type comment information.\n\n    Failures to assign to `__annotations__` are silently ignored.\n\n    If `obj` is a class type, this also ensures that type comment\n    information is incorporated into the `__annotations__` member of\n    all parent classes, if possible.\n\n    This mutates the `__annotations__` of existing imported objects,\n    in order to allow the existing `typing.get_type_hints` method to\n    take the modified annotations into account.\n\n    Modifying existing imported objects is unfortunate but avoids the\n    need to reimplement `typing.get_type_hints` in order to take into\n    account type comment information.\n\n    Note that this does not directly include type comment information\n    from parent classes, but `typing.get_type_hints` takes that into\n    account.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer are implementation details (line numbers, WeakSet usage, MRO traversal, ModuleAnalyzer, contextlib.suppress, config flags, setdefault behavior) that cannot be verified from the provided documentation alone - they appear to be plausible code-level extrapolations or knowledge from the actual source code, but are not directly supported by the docstring provided.",
      "The claim about MRO traversal is consistent with the documentation's statement that 'If obj is a class type, this also ensures that type comment information is incorporated into the __annotations__ member of all parent classes'.",
      "The claim about mutating existing objects is consistent with the documentation's statement 'This mutates the __annotations__ of existing imported objects'.",
      "The claim about typing.get_type_hints is consistent with the documentation's multiple references to that function.",
      "The specific line numbers, variable names, and implementation details (WeakSet, contextlib.suppress(TypeError), config.autodoc_use_type_comments, annotations.setdefault) cannot be verified from the docstring alone but are plausible implementation details"
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: the docstring-derived 'Cache of objects...' description and the default value of autodoc_use_type_comments are not in stripped code, but nearly all claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_21",
    "repo": "sphinx",
    "question": "What are the key components and design patterns in Sphinx autodoc's member discovery and filtering pipeline?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `_get_members_to_document` function in `sphinx/ext/autodoc/_dynamic/_member_finder.py` is a key component in Sphinx's autodoc member discovery pipeline. Here's what the code reveals:\n\n**Architecture and Dependencies:**\n\n1. **Two-phase pipeline**: The actual member documentation process is split into `_get_members_to_document()` (discovery phase returning `ObjectMember` list) and `_filter_members()` (filtering phase that applies skip logic and emits the `autodoc-skip-member` event). In the code these are separate functions called sequentially from `_gather_members()`.\n\n2. **`wanted_members` sentinel pattern**: When `want_all=True`, if the parent is a module with `__all__` defined (and `ignore_module_all` is False), `wanted_members` is set to `frozenset(props.all)`. Otherwise, it's set to the `ALL` sentinel (an instance of `_All` class from `sphinx/ext/autodoc/_sentinels.py`), whose `__contains__` method always returns `True`, allowing any name to pass the `name in wanted_members` check.\n\n3. **Class signature 'separated' mode**: When `want_all=False` and `class_signature == 'separated'` and the parent object type is 'class' or 'exception', the function automatically adds `'__new__'` and `'__init__'` to the `combined_members` set to ensure they are documented.\n\n4. **`ObjectMember` class**: Has `__slots__ = '__name__', 'object', 'docstring', 'class_'` (4 slots). It's initialized with `name`, `obj`, and optional keyword arguments `docstring` and `class_`. Despite declaring `skipped: bool` as a type annotation, it is NOT in `__slots__`, meaning it cannot actually be set as an instance attribute.\n\n5. **Member sources for classes**: For class-type objects, members are gathered from multiple sources in order: enum members (via `_filter_enum_dict`), `__slots__` (using `getslots`), `dir()` results, annotation-only members from MRO classes, and instance attributes discovered by `ModuleAnalyzer`.\n\n6. **Name unmangling**: The `unmangle()` function handles Python's name-mangling for private attributes (names starting with `__` but not ending with `__`). It checks if a name starts with `_ClassName__` prefix and unmangles it back to `__name`. If the mangled name originates from a parent class in the MRO, `unmangle()` returns `None` to skip it.\n\n7. **The `_filter_members` function emits `autodoc-skip-member`**: The event is emitted via `events.emit_firstresult()` with arguments: `props.obj_type`, `member_name`, `member_obj`, `not keep` (the default skip decision inverted), and `options`. The event handler protocol (`_AutodocSkipMemberListener`) returns `bool | None`.\n\n8. **Privacy detection via metadata**: In `_should_keep_member()`, the function calls `separate_metadata()` from `sphinx/util/docstrings.py`, which extracts `:meta X:` field list items from docstrings. The `separate_metadata` function parses field list markers and stores fields with a `meta ` prefix into a metadata dictionary.\n\n9. **Inherited member filtering**: When `want_all=True` and `inherited_members` is empty (no inherited-members option), only members where `m.class_ == props._obj` are returned, filtering out inherited members.\n\n10. **Caller**: `_gather_members()` is called from `sphinx/ext/autodoc/_generate.py`, which computes `want_all = bool(all_members or options.inherited_members or options.members is ALL)`.",
    "rubric": [
      "Identifies the two-phase pipeline: _get_members_to_document (discovery) and _filter_members (filtering) called sequentially from _gather_members",
      "Explains the ALL sentinel pattern: _All class with __contains__ always returning True, used when want_all=True without a module __all__ restriction",
      "Describes that when want_all=True and a module has __all__ defined (and ignore_module_all is False), wanted_members is set to frozenset(props.all)",
      "Notes that when want_all=False and class_signature=='separated' for class/exception types, __new__ and __init__ are added to combined_members",
      "Describes ObjectMember's __slots__ containing '__name__', 'object', 'docstring', 'class_' (4 slots), with 'skipped' declared as annotation but not in __slots__",
      "Explains the unmangle() function handles Python name-mangling (_ClassName__ prefix), returning None for mangled names from parent classes in the MRO",
      "Describes that _filter_members emits the 'autodoc-skip-member' event via emit_firstresult with the member details and inverted keep decision",
      "Notes that _should_keep_member uses separate_metadata() to detect :meta private: and :meta public: field list items in docstrings",
      "Explains inherited member filtering: when want_all=True and inherited_members is empty, only members where m.class_ == props._obj are returned",
      "Identifies that for class-type objects, members are gathered from multiple sources: enum members, __slots__, dir() results, annotation-only members from MRO, and instance attributes via ModuleAnalyzer"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_dynamic/_member_finder.py",
      "sphinx/ext/autodoc/_sentinels.py",
      "sphinx/ext/autodoc/_shared.py",
      "sphinx/ext/autodoc/_property_types.py",
      "sphinx/util/docstrings.py",
      "sphinx/ext/autodoc/_generate.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_dynamic/_member_finder.py] sphinx.ext.autodoc._dynamic._member_finder._get_members_to_document\nFind out which members are documentable\n\n    If *want_all* is True, return all members.  Else, only return those\n    members given by *self.options.members* (which may also be None).\n\n    Filter the given member list.\n\n    Members are skipped if\n\n    - they are private (except if given explicitly or the private-members\n      option is set)\n    - they are special methods (except if given explicitly or the\n      special-members option is set)\n    - they are undocumented (except if the undoc-members option is set)\n\n    The user can override the skipping decision by connecting to the\n    ``autodoc-skip-member`` event.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive implementation details (10 numbered points) that go far beyond what the documentation states. Since the documentation is just a docstring, these details cannot be verified from the provided text alone.",
      "Point 1 claims the docstring 'describes the combined behavior' of two separate functions, but the docstring specifically says members are 'skipped if' certain conditions are met and mentions the autodoc-skip-member event - this is consistent with the answer's claim but cannot be fully verified.",
      "The answer's description of filtering logic (private members, special methods, undocumented members, autodoc-skip-member event) is consistent with the documentation's description of skip conditions.",
      "All 10 points describe internal implementation details that are plausible for a Sphinx autodoc module but cannot be verified from the provided docstring alone - they could be accurate code-level observations or hallucinations."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in one claim about docstring describing combined behavior and partial leakage about :meta private:/:meta public: specifics; most claims are code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_23",
    "repo": "sphinx",
    "question": "Where is `cut_lines` defined, how is it re-exported, and how does its returned closure connect to the `autodoc-process-docstring` event?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `cut_lines` function is defined in `sphinx/ext/autodoc/_event_listeners.py` (line 80) and is re-exported through `sphinx/ext/autodoc/__init__.py` (line 19, imported from `_event_listeners`, and listed in `__all__` at line 78). It returns a closure named `process` with return type annotation `_AutodocProcessDocstringListener`, matching the protocol for `autodoc-process-docstring` event listeners. The event is registered in the `setup()` function of `sphinx/ext/autodoc/__init__.py` at line 200 via `app.add_event('autodoc-process-docstring')`. When the event is emitted, it happens in `sphinx/ext/autodoc/_dynamic/_docstrings.py` in the `_process_docstrings` function (line 121-128), which passes `props.obj_type`, `props.full_name`, `props._obj`, `options`, and `docstring_lines` to the listener. The `_AutodocObjType` type (used for the `what_` parameter in the inner `process` function) is defined in `sphinx/ext/autodoc/_property_types.py` as a `Literal` type with values: `'module'`, `'class'`, `'exception'`, `'function'`, `'decorator'`, `'method'`, `'property'`, `'attribute'`, `'data'`, `'type'`.",
    "rubric": [
      "States that `cut_lines` is defined in `sphinx/ext/autodoc/_event_listeners.py`",
      "States that it is re-exported via import in `sphinx/ext/autodoc/__init__.py` and listed in `__all__`",
      "Explains that `cut_lines` returns an inner closure named `process` with the `_AutodocProcessDocstringListener` return type",
      "Identifies that the `autodoc-process-docstring` event is registered in the `setup()` function of `sphinx/ext/autodoc/__init__.py` via `app.add_event`",
      "Identifies that the event is emitted in `sphinx/ext/autodoc/_dynamic/_docstrings.py` in the `_process_docstrings` function, passing `props.obj_type`, `props.full_name`, `props._obj`, `options`, and `docstring_lines`",
      "Mentions that the `_AutodocObjType` type (used for the `what_` parameter) is defined in `sphinx/ext/autodoc/_property_types.py` as a Literal with values including 'module', 'class', 'exception', 'function', 'decorator', 'method', 'property', 'attribute', 'data', 'type'"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_event_listeners.py",
      "sphinx/ext/autodoc/__init__.py",
      "sphinx/ext/autodoc/_dynamic/_docstrings.py",
      "sphinx/ext/autodoc/_property_types.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_event_listeners.py] sphinx.ext.autodoc._event_listeners.cut_lines\nReturn a listener that removes the first *pre* and last *post*\n    lines of every docstring.  If *what* is a sequence of strings,\n    only docstrings of a type in *what* will be processed.\n\n    Use like this (e.g. in the ``setup()`` function of :file:`conf.py`)::\n\n       from sphinx.ext.autodoc import cut_lines\n\n       app.connect('autodoc-process-docstring', cut_lines(4, what={'module'}))\n\n    This can (and should) be used in place of :confval:`automodule_skip_lines`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific claims about line numbers, file paths, and implementation details (e.g., 'line 80', 'line 19', 'line 78', 'line 200', 'line 121-128', '_AutodocObjType', '_property_types.py', '_dynamic/_docstrings.py') that cannot be verified from the provided documentation alone. These are plausible code-level details but could be hallucinated.",
      "The documentation states the function is in 'sphinx/ext/autodoc/_event_listeners.py' which matches the answer's claim about where it's defined.",
      "The documentation shows the import path as 'from sphinx.ext.autodoc import cut_lines' which is consistent with the answer's claim about re-exporting through __init__.py.",
      "The answer's description of functionality (returns a listener that removes first/last lines of docstrings, filtered by 'what') is consistent with the documentation.",
      "The list of _AutodocObjType literal values includes 'decorator' which is not mentioned in the documentation but cannot be contradicted by it either - this is an unverifiable claim."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only a minor phrasing about 'serves as an event listener' was slightly rewritten to reference the type annotation instead.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_24",
    "repo": "sphinx",
    "question": "How does the legacy class-based autodoc system dispatch directives to documenters and handle member documentation?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Documenter` class in `sphinx/ext/autodoc/_legacy_class_based/_documenters.py` serves as the base class for all autodoc documenters. Here's how it supports the autodoc framework:\n\n**Registration and Discovery:**\n- The `documenters` property accesses `self.env._registry.documenters`, which is a dict stored in `SphinxComponentRegistry` (registry.py) mapping objtype strings to Documenter classes.\n- Documenters are registered via `SphinxComponentRegistry`, which stores them in `self.documenters`.\n- The legacy class-based system is activated when `config.autodoc_use_legacy_class_based` is True, at which point `app.add_autodocumenter()` is called for each documenter in `_register_directives()` (autodoc/__init__.py).\n\n**Dispatching Mechanism:**\n- The `AutodocDirective.run()` method (directive.py) looks up the documenter class via `self.env._registry.documenters[objtype]` where objtype is derived from the directive name by stripping the 'auto' prefix (e.g., 'autoclass' \u2192 'class').\n- It creates a `DocumenterBridge` containing env, reporter, options, lineno, and state, then instantiates the documenter and calls `generate()`.\n\n**Member Documentation Flow:**\n- `document_members()` finds candidate documenter classes for each member by calling `can_document_member()` on all registered documenters, sorts them by `priority`, and picks the highest-priority one.\n- Members are constructed with name format `'{modname}::{path.to.member}'`.\n- All member documenters call `parse_name()` and `import_object()` before sorting, then directly call `_generate()` (not `generate()`) since those steps are already done.\n\n**`get_attr` and the attrgetter system:**\n- `get_attr()` delegates to `autodoc_attrgetter()`, which iterates `registry.autodoc_attrgetters` (a dict of type \u2192 callable) and calls the matching function for the object's type, falling back to `safe_getattr()`.\n\n**Options handling:**\n- The `Options` class (in `_directive_options.py`) is a dict subclass that converts attribute access to hyphenated key lookups (`name.replace('_', '-')`) and returns `None` for missing keys.\n\n**Sorting:**\n- The `sort_members()` method supports three orders: 'groupwise' (sorts by `member_order` class variable then name), 'bysource' (uses `ModuleAnalyzer.tagorder`), and alphabetical (default, sorts by name).\n\n**Sentinel objects:**\n- `ALL` is an instance of `_All` whose `__contains__` always returns True and whose `append` is a no-op.\n- `INSTANCEATTR`, `SLOTSATTR`, `UNINITIALIZED_ATTR`, and `SUPPRESS` are bare `object()` instances used as sentinels.\n\n**Class hierarchy:**\n- `ModuleLevelDocumenter` and `ClassLevelDocumenter` extend `Documenter` with different `resolve_name()` implementations.\n- `FunctionDocumenter` (priority default 0, member_order 30), `ClassDocumenter` (priority 15, member_order 20), `ExceptionDocumenter` (priority 20), `MethodDocumenter` (priority 1, member_order 50), `AttributeDocumenter` (priority 10, member_order 60), `PropertyDocumenter` (priority 11), `DataDocumenter` (priority -10, member_order 40), `DecoratorDocumenter` (priority -1).\n- `DocstringSignatureMixin` provides signature extraction from docstrings - it searches for lines matching `py_ext_sig_re` against valid names (the object name and MRO class names for ClassDocumenter).",
    "rubric": [
      "Explains that AutodocDirective.run() strips the 'auto' prefix from the directive name to get objtype and looks up the documenter class via self.env._registry.documenters[objtype]",
      "Mentions DocumenterBridge is created with env, reporter, options, lineno, and state, then the documenter is instantiated and generate() is called",
      "Describes how document_members() calls can_document_member() on all registered documenters to find which can handle each member, sorts by priority, and picks the highest",
      "Explains that member documenters call parse_name() and import_object() first, then _generate() is called directly (not generate()) since those steps are already done",
      "Describes sort_members() supporting 'groupwise' (sorts by member_order then name), 'bysource' (uses ModuleAnalyzer.tagorder), and alphabetical (default, sorts by name)",
      "Mentions how documenters are registered: when autodoc_use_legacy_class_based is True, app.add_autodocumenter() is called for each documenter class in _register_directives()",
      "Explains that the Documenter.documenters property accesses self.env._registry.documenters to get the mapping of objtype strings to Documenter classes"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_legacy_class_based/_documenters.py",
      "sphinx/ext/autodoc/directive.py",
      "sphinx/ext/autodoc/_legacy_class_based/_directive_options.py",
      "sphinx/ext/autodoc/_legacy_class_based/_sentinels.py",
      "sphinx/ext/autodoc/__init__.py",
      "sphinx/registry.py",
      "sphinx/ext/autodoc/importer.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_legacy_class_based/_documenters.py] sphinx.ext.autodoc._legacy_class_based._documenters.Documenter\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter:\n  A Documenter knows how to autodocument a single object type.  When\n    registered with the AutoDirective, it will be used to document objects\n    of that type when needed by autodoc.\n\n    Its *objtype* attribute selects what auto directive it is assigned to\n    (the directive name is 'auto' + objtype), and what directive it generates\n    by default, though that can be overridden by an attribute called\n    *directivetype*.\n\n    A Documenter has an *option_spec* that works like a docutils directive's;\n    in fact, it will be used to parse an auto directive's options that matches\n    the Documenter.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.get_attr:\n  getattr() override for types such as Zope interfaces.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.can_document_member:\n  Called to see if a member can be documented by this Documenter.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.documenters:\n  Returns registered Documenter classes\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.add_line:\n  Append one line of generated reST to the output.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.resolve_name:\n  Resolve the module and name of the object to document given by the\n        arguments and the current module/class.\n\n        Must return a pair of the module name and a chain of attributes; for\n        example, it would return ``('zipfile', ['ZipFile', 'open'])`` for the\n        ``zipfile.ZipFile.open`` method.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.parse_name:\n  Determine what module to import and what attribute to document.\n\n        Returns True and sets *self.modname*, *self.objpath*, *self.fullname*,\n        *self.args* and *self.retann* if parsing and resolving was successful.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.import_object:\n  Import the object given by *self.modname* and *self.objpath* and set\n        it as *self.object*.\n\n        Returns True if successful, False if an error occurred.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.get_real_modname:\n  Get the real module name of an object to document.\n\n        It can differ from the name of the module through which the object was\n        imported.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.check_module:\n  Check if *self.object* is really defined in the module given by\n        *self.modname*.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.format_args:\n  Format the argument signature of *self.object*.\n\n        Should return None if the object does not have a signature.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.format_name:\n  Format the name of *self.object*.\n\n        This normally should be something that can be parsed by the generated\n        directive, but doesn't need to be (Sphinx will display it unparsed\n        then).\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.format_signature:\n  Format the signature (arguments and return annotation) of the object.\n\n        Let the user process it via the ``autodoc-process-signature`` event.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.add_directive_header:\n  Add the directive header and options to the generated content.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.get_doc:\n  Decode and return lines of the docstring(s) for the object.\n\n        When it returns None, autodoc-process-docstring will not be called for this\n        object.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.process_doc:\n  Let the user process the docstrings before adding them.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.add_content:\n  Add content from docstrings, attribute documentation and user.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.get_object_members:\n  Return `(members_check_module, members)` where `members` is a\n        list of `(membername, member)` pairs of the members of *self.object*.\n\n        If *want_all* is True, return all members.  Else, only return those\n        members given by *self.options.members* (which may also be None).\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.filter_members:\n  Filter the given member list.\n\n        Members are skipped if\n\n        - they are private (except if given explicitly or the private-members\n          option is set)\n        - they are special methods (except if given explicitly or the\n          special-members option is set)\n        - they are undocumented (except if the undoc-members option is set)\n\n        The user can override the skipping decision by connecting to the\n        ``autodoc-skip-member`` event.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.document_members:\n  Generate reST for member documentation.\n\n        If *all_members* is True, document all members, else those given by\n        *self.options.members*.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.sort_members:\n  Sort the given member list.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.Documenter.generate:\n  Generate reST for the object given by *self.name*, and possibly for\n        its members.\n\n        If *more_content* is given, include that content. If *real_modname* is\n        given, use that module name to find attribute docs. If *check_module* is\n        True, only generate if the object is defined in the module name it is\n        imported from. If *all_members* is True, document all members.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, priority values, class hierarchies, sentinel objects, etc.) that are not present in the provided documentation chunk. These cannot be verified against the given documentation.",
      "The documentation describes `get_attr` as 'getattr() override for types such as Zope interfaces' - the answer's claim about `autodoc_attrgetter()` delegation and `registry.autodoc_attrgetters` cannot be verified from the provided doc.",
      "The answer mentions specific priority values (FunctionDocumenter priority 0, ClassDocumenter priority 15, etc.) that are not in the documentation and cannot be verified.",
      "The answer describes `Options` class behavior, `ALL` sentinel, `INSTANCEATTR`, `SLOTSATTR`, `UNINITIALIZED_ATTR`, `SUPPRESS` objects - none of these are mentioned in the provided documentation.",
      "The answer's description of `document_members()` calling `can_document_member()` on all registered documenters and sorting by priority is a plausible extrapolation from the documentation's description of `can_document_member` and `document_members`, but the specific mechanism isn't stated in the doc.",
      "The claim about `config.autodoc_use_legacy_class_based` and `_register_directives()` cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rationale phrases ('to avoid circular imports') relied on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_26",
    "repo": "sphinx",
    "question": "Why does AutodocDirective use a dispatcher pattern with DummyOptionSpec instead of having separate directive classes for each autodoc object type?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `AutodocDirective` (in `sphinx/ext/autodoc/directive.py`) is designed as a **dispatcher** rather than having separate directive classes for each documented object type because of a key architectural decision: **a single directive class is registered multiple times under different names** (e.g., `automodule`, `autoclass`, `autofunction`), and it determines which Documenter to invoke at runtime by stripping the `'auto'` prefix from `self.name` (`objtype = self.name[4:]`).\n\nThe design of this dispatcher pattern includes:\n\n1. **`DummyOptionSpec` enables the same directive class to handle all autodoc types**: `DummyOptionSpec` inherits from `dict` but overrides `__bool__` to always return `True` and `__getitem__` to return an identity lambda (`lambda x: x`) for any key, accepting any option without type-specific validation at the directive level. The actual option validation is deferred to `process_documenter_options()` which uses the specific Documenter's `option_spec`.\n\n2. **Separation of directive registration from documenter registration**: `app.add_autodocumenter()` both registers the Documenter class in `registry.documenters[objtype]` AND registers the same `AutodocDirective` class under the name `'auto' + objtype`.\n\n3. **Two parallel implementations exist**: There's a legacy class-based `AutodocDirective` in `sphinx/ext/autodoc/directive.py` that dispatches to `Documenter` subclasses via `env._registry.documenters[objtype]`, and a newer one in `sphinx/ext/autodoc/_directive.py` that calls `_auto_document_object()` directly without the `Documenter` class hierarchy. The choice is controlled by the `autodoc_use_legacy_class_based` config option (default: `False`). The new directive hardcodes `titles_allowed = True` rather than reading it from a Documenter class.\n\n4. **Error handling returns empty list**: If option processing fails (KeyError, ValueError, TypeError), both implementations log an error and return `[]` (empty node list).\n\n5. **Dependency tracking**: The legacy directive records file dependencies discovered during documentation generation (`params.record_dependencies`) into the document's dependency list (`self.state.document.settings.record_dependencies`).",
    "rubric": [
      "Explains that a single AutodocDirective class is registered multiple times under different names (automodule, autoclass, autofunction, etc.)",
      "Explains that the object type is determined at runtime by stripping the 'auto' prefix from self.name (self.name[4:])",
      "Explains DummyOptionSpec overrides __bool__ to return True and __getitem__ to return an identity lambda, allowing any option to pass through without type-specific validation at directive level",
      "Explains that actual option validation is deferred to process_documenter_options or _process_documenter_options which uses the specific documenter/object type's option_spec",
      "Mentions that add_autodocumenter registers both the documenter and the same AutodocDirective class under the 'auto' + objtype name",
      "Notes the existence of two parallel implementations (legacy class-based in directive.py and new one in _directive.py) controlled by autodoc_use_legacy_class_based config"
    ],
    "key_files": [
      "sphinx/ext/autodoc/directive.py",
      "sphinx/ext/autodoc/_directive.py",
      "sphinx/ext/autodoc/__init__.py",
      "sphinx/application.py",
      "sphinx/ext/autodoc/_legacy_class_based/_documenters.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/directive.py] sphinx.ext.autodoc.directive.AutodocDirective\nsphinx.ext.autodoc.directive.AutodocDirective:\n  A directive class for all autodoc directives. It works as a dispatcher of Documenters.\n\n    It invokes a Documenter upon running. After the processing, it parses and returns\n    the content generated by Documenter.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that AutodocDirective is 'a directive class for all autodoc directives' that 'works as a dispatcher of Documenters' and 'invokes a Documenter upon running' then 'parses and returns the content generated by Documenter'. The answer provides extensive implementation details (line numbers, DummyOptionSpec, two parallel implementations, autodoc_use_legacy_class_based config option, _auto_document_object function) that cannot be verified from the provided documentation.",
      "The claim about 'two parallel implementations' and 'autodoc_use_legacy_class_based' config option in point 3 appears to be a hallucination - this is not a well-known feature of Sphinx and cannot be verified from the documentation provided.",
      "The specific line number references (line 143, line 73) and code details like `self.name[4:]` cannot be verified from the documentation provided.",
      "The claim about `_auto_document_object()` and `sphinx/ext/autodoc/_directive.py` as a separate newer implementation is suspicious and likely fabricated."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in design rationale explanations (why patterns exist, what they enable), but the core factual claims about code structure are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_25",
    "repo": "sphinx",
    "question": "What is the architecture of FunctionDocumenter \u2014 its class hierarchy, key methods, and how does it differ from MethodDocumenter in handling singledispatch and overloads?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `FunctionDocumenter` class in `sphinx/ext/autodoc/_legacy_class_based/_documenters.py` is a specialized documenter for functions that inherits from both `DocstringSignatureMixin` and `ModuleLevelDocumenter` (which itself inherits from `Documenter`). Here are the key architectural details:\n\n**Class hierarchy:** `FunctionDocumenter` \u2192 `DocstringSignatureMixin` + `ModuleLevelDocumenter` \u2192 `Documenter`\n\n**Key attributes:**\n- `objtype = 'function'`\n- `member_order = 30`\n- Inherits `priority = 0` from `Documenter` (not overridden)\n\n**`can_document_member`** checks three conditions: `inspect.isfunction(member)` OR `inspect.isbuiltin(member)` OR (`inspect.isroutine(member)` AND parent is a `ModuleDocumenter`).\n\n**`format_args`** emits the `'autodoc-before-process-signature'` event with `False` as the second argument, then uses `sphinx.util.inspect.signature()` with `type_aliases` from config. It handles `autodoc_typehints` in `{'none', 'description'}` by setting `show_annotation=False`, and respects `autodoc_typehints_format == 'short'` and `python_display_short_literal_types`.\n\n**`format_signature`** handles three cases:\n1. Normal functions (delegates to `DocstringSignatureMixin.format_signature`)\n2. `singledispatch` functions - iterates `self.object.registry.items()`, skipping `object` type entries, and calls `annotate_to_first_argument` for each dispatch variant\n3. Overloaded functions - detected via `self.analyzer.overloads`, where each overload signature is processed through `merge_default_value` then `evaluate_signature`\n\nAll signatures are joined with `'\\n'.join(sigs)`.\n\n**`merge_default_value`** replaces overload parameter defaults that are `'...'` (ellipsis as a string) with the actual implementation's defaults, looking up by parameter name.\n\n**`annotate_to_first_argument`** (for FunctionDocumenter) annotates `params[0]` (the first parameter) if it has no annotation, returning a dummy function with the modified signature. This differs from `MethodDocumenter.annotate_to_first_argument` which annotates `params[1]` (skipping `self`) and returns `None` when `len(sig.parameters) == 1` instead of `== 0`.\n\n**`add_directive_header`** adds `:async:` option if the function is a coroutine or async generator function.\n\n**`document_members`** is a no-op.\n\n**`DecoratorDocumenter`** subclasses `FunctionDocumenter` with `priority = -1` and its `format_args` only returns args if they contain a comma.\n\nThe overloads dictionary in `ModuleAnalyzer` maps dotted qualified names (e.g., `'MyClass.my_method'`) to lists of `Signature` objects, parsed from AST via `signature_from_ast` in `sphinx/util/inspect.py`.\n\nThere's also a newer non-legacy implementation in `sphinx/ext/autodoc/_dynamic/_signatures.py` with a standalone `_merge_default_value` function that has identical logic.",
    "rubric": [
      "Identifies FunctionDocumenter inherits from DocstringSignatureMixin and ModuleLevelDocumenter (which inherits from Documenter)",
      "Describes can_document_member checking isfunction OR isbuiltin OR (isroutine AND parent is ModuleDocumenter)",
      "Explains format_args emits 'autodoc-before-process-signature' with False and handles autodoc_typehints and typehints_format config",
      "Describes format_signature handling three cases: normal functions, singledispatch functions (iterating registry, skipping object type), and overloaded functions (using analyzer.overloads with merge_default_value and evaluate_signature)",
      "Explains merge_default_value replaces overload parameter defaults that are '...' (ellipsis string) with actual implementation defaults",
      "Explains annotate_to_first_argument annotates params[0] for FunctionDocumenter vs params[1] for MethodDocumenter, and the guard condition differs (len == 0 vs len == 1)",
      "Mentions add_directive_header adds :async: for coroutine or async generator functions",
      "Mentions document_members is a no-op",
      "Mentions DecoratorDocumenter subclasses FunctionDocumenter with priority -1 and only shows args if they contain a comma"
    ],
    "key_files": [
      "sphinx/ext/autodoc/_legacy_class_based/_documenters.py",
      "sphinx/ext/autodoc/__init__.py",
      "sphinx/util/inspect.py",
      "sphinx/ext/autodoc/_dynamic/_signatures.py",
      "sphinx/pycode/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autodoc/_legacy_class_based/_documenters.py] sphinx.ext.autodoc._legacy_class_based._documenters.FunctionDocumenter\nsphinx.ext.autodoc._legacy_class_based._documenters.FunctionDocumenter:\n  Specialized Documenter subclass for functions.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.FunctionDocumenter.merge_default_value:\n  Merge default values of actual implementation to the overload variants.\n\nsphinx.ext.autodoc._legacy_class_based._documenters.FunctionDocumenter.annotate_to_first_argument:\n  Annotate type hint to the first argument of function if needed.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only mentions FunctionDocumenter as a 'Specialized Documenter subclass for functions' with two methods: merge_default_value and annotate_to_first_argument. The vast majority of the generated answer's claims (class hierarchy, attributes like objtype/member_order/priority, can_document_member logic, format_args details, format_signature behavior, add_directive_header, document_members, DecoratorDocumenter, ModuleAnalyzer details, non-legacy implementation) are not supported by the provided documentation.",
      "The claims about merge_default_value and annotate_to_first_argument are consistent with the documentation's brief descriptions but go far beyond what the documentation states.",
      "No claims directly contradict the documentation, but most are unsupported extrapolations that could be hallucinated or could be accurate based on source code inspection."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor documentation-style explanations were trimmed from a few claims.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_27",
    "repo": "sphinx",
    "question": "Where does data flow through the Autosummary directive, from parsing content names to generating table nodes and toctrees?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data flows through the `Autosummary` directive as follows:\n\n1. **`Autosummary.run()`** is the entry point. It extracts names from `self.content` by filtering lines that start with `~`, a letter, or `_` (regex `r'^[~a-zA-Z_]'`). It calls `get_items(names)` to produce a list of `(display_name, signature, summary, real_name)` tuples, then passes those to `get_table(items)` which returns docutils nodes.\n\n2. **`get_items()`** first resolves import prefixes via `get_import_prefixes_from_env(self.env)`, which builds a list from `env.ref_context` looking at `'py:module'` and `'py:class'` keys. For each name, it calls `self.import_by_name()` which wraps `import_by_name()` and falls back to `import_ivar_by_name()` (for instance attributes). The imported object is then typed via `_get_documenter(obj, parent)` which calls `_best_object_type_for_member` from `sphinx.ext.autodoc._dynamic._member_finder` and defaults to `'data'` if no better type is found. Then `_load_object_by_name()` from `sphinx/ext/autodoc/_dynamic/_loader.py` is called to get `_ItemProperties` including signatures and docstring lines.\n\n3. **Signature handling in `get_items()`**: The `signatures` option supports three values: `'none'`, `'short'`, and `'long'`. The legacy `nosignatures` flag maps to `'none'`. When `'short'`, signatures become `'()'` if the signature is exactly `('()',)`, otherwise `'(\u2026)'` (with an ellipsis character U+2026). When `'long'`, signatures go through `mangle_signature()` with `max_chars = max(10, 50 - len(display_name))` (the hardcoded `max_item_chars` is 50).\n\n4. **`mangle_signature()`** first calls `_cleanup_signature()` which uses `signature_from_str()` to parse, removes type annotations and replaces defaults with `None`. It then strips return type annotations, literals, and complex nested objects. It splits into positional `args` and optional `opts` (parameters with defaults). It calls `limited_join()` which truncates with an overflow marker.\n\n5. **Summary extraction**: `extract_summary()` takes the first non-blank stanza, parses it via `_parse_summary()` using a fresh `RSTStateMachine` with `NullReporter()`. It handles sections, non-paragraph starts, and multi-sentence paragraphs split by `periods_re = re.compile(r'\\.(?:\\s+)')`. It skips splitting at `WELL_KNOWN_ABBREVIATIONS = ('et al.', 'e.g.', 'i.e.', 'vs.')`. Trailing `::` is replaced with `.` via `literal_re`.\n\n6. **`get_table()`** creates a 2-column table with colwidths 10 and 90. The table gets CSS classes `['autosummary', 'longtable']` plus any user-specified classes. Each item's first column uses the RST role `:py:obj:` (hardcoded qualifier `'obj'`), linking `display_name` to `real_name`. The signature is escaped with `rst.escape()` and appended with `\\\\ ` (backslash-space for non-breaking). The second column is the summary string. Each cell is parsed via `nested_parse_to_nodes()` with `allow_section_headings=False`.\n\n7. **Toctree generation**: After `get_table()`, if the `toctree` option is set, `run()` iterates through items using their `real_name`, looks up `autosummary_filename_map` for remapping, constructs docnames joined with the tree prefix, and checks `self.env.found_docs`. Valid docnames are added to an `addnodes.toctree` node with `maxdepth=-1` and `glob=None`, wrapped in an `autosummary_toc` node.\n\n8. **HTML rendering**: `autosummary_toc_visit_html` raises `SkipNode` to hide the toctree in HTML. `autosummary_table_visit_html` replaces spaces with `\\u00a0` (non-breaking space) in the first column's text nodes.\n\n9. **Instance attribute fallback**: `import_ivar_by_name()` is called when normal import fails. It uses `ModuleAnalyzer` to check `attr_docs` and `annotations` for the attribute, and returns `INSTANCE_ATTR` sentinel (a `_Sentinel` instance from `sphinx.ext.autodoc._sentinels`) as the object.\n\n10. **Registration**: `setup()` registers the directive as `'autosummary'`.",
    "rubric": [
      "run() is the entry point that extracts names from self.content using regex r'^[~a-zA-Z_]', then calls get_items(names) and get_table(items)",
      "get_items() resolves import prefixes via get_import_prefixes_from_env() which looks at env.ref_context for 'py:module' and 'py:class' keys",
      "import_by_name() is called for each name and falls back to import_ivar_by_name() for instance attributes",
      "_get_documenter() determines the object type using _best_object_type_for_member and defaults to 'data'",
      "_load_object_by_name() is called to get item properties including signatures and docstring lines",
      "The signatures option supports 'none', 'short', and 'long'; 'short' produces '()' or '(\u2026)'; 'long' goes through mangle_signature() with max_chars = max(10, 50 - len(display_name))",
      "mangle_signature() calls _cleanup_signature() which uses signature_from_str() to parse, removes annotations and replaces defaults, then uses limited_join() to truncate",
      "extract_summary() finds the first non-blank stanza, uses _parse_summary() with RSTStateMachine and NullReporter, splits sentences using periods_re while respecting WELL_KNOWN_ABBREVIATIONS",
      "get_table() creates a 2-column table with colwidths 10 and 90, uses ':py:obj:' role for linking, and adds CSS classes ['autosummary', 'longtable']",
      "If toctree option is set, run() builds an addnodes.toctree with maxdepth=-1 and glob=None, wrapped in autosummary_toc node",
      "import_ivar_by_name() uses ModuleAnalyzer to check attr_docs and annotations, returning INSTANCE_ATTR sentinel as the object"
    ],
    "key_files": [
      "sphinx/ext/autosummary/__init__.py",
      "sphinx/ext/autodoc/_dynamic/_loader.py",
      "sphinx/ext/autodoc/_sentinels.py",
      "sphinx/util/docutils.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autosummary/__init__.py] sphinx.ext.autosummary.Autosummary\nsphinx.ext.autosummary.Autosummary:\n  Pretty table containing short signatures and summaries of functions etc.\n\n    autosummary can also optionally generate a hidden toctree:: node.\n\nsphinx.ext.autosummary.Autosummary.get_items:\n  Try to import the given names, and return a list of\n        ``[(name, signature, summary_string, real_name), ...]``.\n\n        signature is already formatted and is None if :nosignatures: option was given.\n\nsphinx.ext.autosummary.Autosummary.get_table:\n  Generate a proper list of table nodes for autosummary:: directive.\n\n        *items* is a list produced by :meth:`get_items`.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, regex patterns, specific function names, internal module paths, etc.) that go far beyond what the documentation provides. While these may be accurate based on the actual source code, they cannot be verified from the provided documentation alone.",
      "The documentation states that get_items returns '[(name, signature, summary_string, real_name), ...]' while the answer says it returns '(display_name, signature, summary, real_name)' tuples - this is a minor naming difference but not a contradiction.",
      "The documentation states 'signature is already formatted and is None if :nosignatures: option was given' - the answer claims signatures option supports 'none', 'short', and 'long' values, with nosignatures mapping to 'none'. This is an extrapolation that cannot be verified from the doc but doesn't directly contradict it.",
      "The documentation says get_table takes '*items* is a list produced by :meth:`get_items`' which is consistent with the answer's description of the data flow."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only line numbers and a few details about the truncated setup() function needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_28",
    "repo": "sphinx",
    "question": "How does the autosummary extension split a fully-qualified Python name into module and qualname, and when is this splitting used?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `_split_full_qualified_name` function in `sphinx/ext/autosummary/generate.py` uses a left-to-right greedy import algorithm to split a fully-qualified Python name into a `(modname, qualname)` tuple. Here's how it works:\n\n**Algorithm:**\n1. The input `name` string is split on `'.'` into `parts`.\n2. It iterates through the parts left-to-right using `enumerate(parts, 1)` (1-based indexing).\n3. At each step `i`, it constructs a candidate module name from `parts[:i]` and attempts `importlib.import_module(modname)` on it.\n4. **If the import succeeds**, it continues to the next part, greedily consuming as much as possible into the module name.\n5. **If an `ImportError` is raised**: it checks whether any previous parts successfully imported (`parts[:i-1]` is non-empty). If yes, it returns `('.'.join(parts[:i-1]), '.'.join(parts[i-1:]))` \u2014 the last successfully imported module as modname, and the rest starting from the failing part as qualname. If no previous part succeeded, it returns `(None, '.'.join(parts))`, signaling the entire name couldn't be resolved to any module.\n6. **If an `IndexError` is raised**, it silently passes (an edge case guard).\n7. If the entire loop completes without any `ImportError` (meaning the full name is itself a module), it returns `(name, '')` \u2014 the whole name is the module, with an empty qualname.\n\n**Return type:** `tuple[str | None, str]` \u2014 the first element can be `None` if no module component could be imported.\n\n**How it fits into the system:**\n- It is called as a **fallback** inside `generate_autosummary_content()` only when `modname is None or qualname is None`. The `generate_autosummary_content` function receives `modname` and `qualname` as parameters, which are computed upstream.\n- After `_split_full_qualified_name` returns, the results populate the Jinja2 template namespace: `ns['module']` is set to `modname`, `ns['objname']` to `qualname`, and `ns['name']` to a `shortname` derived from `qualname.rsplit('.', 1)[-1]` (or the full `qualname` for classes). For methods/attributes/properties, `ns['class']` is extracted from `qualname.rsplit('.', 1)[0]`.\n- `_split_full_qualified_name` only cares about finding the module boundary, searching **forward** (shortest to longest module prefix), and stops at the first `ImportError`.",
    "rubric": [
      "Explains that the name is split on '.' into parts and iterated left-to-right with 1-based indexing",
      "Mentions that each candidate module prefix is tested via importlib.import_module",
      "Describes greedy behavior: the loop continues consuming parts into the module name as long as imports succeed",
      "Explains that on ImportError, it returns the last successfully imported prefix as modname and the remainder as qualname",
      "Explains that if no prefix imports successfully, it returns None as modname",
      "Explains that if the entire loop completes without ImportError, the whole name is returned as modname with empty qualname",
      "Mentions the IndexError except clause as a silent pass / edge case guard",
      "Notes that this function is called as a fallback when modname or qualname is None in generate_autosummary_content",
      "Describes how the returned modname and qualname populate template namespace variables (module, objname, name/shortname)"
    ],
    "key_files": [
      "sphinx/ext/autosummary/generate.py",
      "sphinx/ext/autosummary/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autosummary/generate.py] sphinx.ext.autosummary.generate._split_full_qualified_name\nSplit full qualified name to a pair of modname and qualname.\n\n    A qualname is an abbreviation for \"Qualified name\" introduced at PEP-3155\n    (https://peps.python.org/pep-3155/).  It is a dotted path name\n    from the module top-level.\n\n    A \"full\" qualified name means a string containing both module name and\n    qualified name.\n\n    .. note:: This function actually imports the module to check its existence.\n              Therefore you need to mock 3rd party modules if needed before\n              calling this function.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The detailed algorithm description (lines 170-183, left-to-right greedy import, enumerate with 1-based indexing, specific error handling logic) is not present in the provided documentation and appears to be derived from reading the actual source code rather than the docstring. These claims cannot be verified from the documentation alone, but they are plausible code-level details.",
      "The description of how the function fits into the system (generate_autosummary_content, generate_autosummary_docs, Jinja2 template namespace, etc.) goes well beyond what the documentation states. These are plausible but unverifiable from the provided doc.",
      "The comparison with _import_by_name() searching in reverse is not mentioned in the documentation and cannot be verified from it.",
      "The documentation explicitly states the function 'actually imports the module to check its existence' and requires mocking 3rd party modules - the answer's description of using importlib.import_module is consistent with this.",
      "The core claim that the function splits a full qualified name into (modname, qualname) is directly supported by the documentation.",
      "The description of qualname as defined by PEP-3155 is consistent with the documentation."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Almost all claims are directly verifiable from stripped code; minor adjustments needed for line number references and claims about truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_29",
    "repo": "sphinx",
    "question": "What does `find_autosummary_in_lines` return, what regex patterns does it use, and how does it handle module prefixes, tilde prefixes, and nested automodule directives?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `find_autosummary_in_lines` function scans input lines to extract items listed in `.. autosummary::` directives. It returns a list of `AutosummaryEntry` named tuples (defined at line 102 of `generate.py`), which is a `NamedTuple` with four fields: `name` (str), `path` (str | None), `template` (str), and `recursive` (bool). \n\nThe function uses 7 compiled regular expressions to parse the input:\n1. `autosummary_re` - matches `.. autosummary::` directives and captures the leading indentation\n2. `automodule_re` - matches `.. automodule::` directives\n3. `module_re` - matches both `.. module::` and `.. currentmodule::` directives (using `(current)?module::` pattern)\n4. `autosummary_item_re` - matches individual item entries (names starting with `~` or `_` or letter)\n5. `recursive_arg_re` - matches the `:recursive:` option\n6. `toctree_arg_re` - matches the `:toctree:` option\n7. `template_arg_re` - matches the `:template:` option\n\nKey implementation details:\n- When a `~` prefix is found on an item name, it is stripped using `str.removeprefix('~')` (line 836)\n- If a `current_module` is set and the item name does not already start with that module prefix, the module name is prepended as `f'{current_module}.{name}'`\n- When `.. automodule::` is encountered, the function recursively calls `find_autosummary_in_docstring()`, which uses `pydoc.getdoc(obj).splitlines()` to obtain the docstring of the module and then calls `find_autosummary_in_lines` again on those lines\n- The `:toctree:` path is resolved relative to the file's parent directory using `Path(filename).parent / toctree`\n- Entries without a `:toctree:` option get `None` for their `path` field and are explicitly skipped in `generate_autosummary_docs` (line 655-658)\n- The `recursive` field controls whether sub-modules of a package are automatically included in the generated content (checked in `generate_autosummary_content` at line 335: `if ispackage and recursive:`)\n- Unknown options (lines starting with `:` that don't match known patterns) are silently skipped (line 831-832)\n- The end of an autosummary block is detected when a non-empty line that doesn't continue the expected indentation is encountered",
    "rubric": [
      "States that the function returns a list of AutosummaryEntry named tuples with fields: name (str), path (str | None), template (str), and recursive (bool)",
      "Identifies all 7 compiled regular expressions: autosummary_re, automodule_re, module_re, autosummary_item_re, recursive_arg_re, toctree_arg_re, template_arg_re",
      "Explains that the tilde (~) prefix is stripped from item names using removeprefix('~')",
      "Explains that if current_module is set and the name doesn't already start with it, the module name is prepended as '{current_module}.{name}'",
      "Explains that when .. automodule:: is encountered, the function recursively calls find_autosummary_in_docstring which obtains the module's docstring via pydoc.getdoc() and calls find_autosummary_in_lines on those lines",
      "Mentions that the :toctree: path is resolved relative to the file's parent directory using Path(filename).parent / toctree",
      "Notes that entries without a :toctree: option get None for their path field and are skipped during generation",
      "Mentions that unknown options (lines starting with ':') are silently skipped"
    ],
    "key_files": [
      "sphinx/ext/autosummary/generate.py",
      "sphinx/ext/autosummary/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/autosummary/generate.py] sphinx.ext.autosummary.generate.find_autosummary_in_lines\nFind out what items appear in autosummary:: directives in the\n    given lines.\n\n    Returns a list of (name, toctree, template) where *name* is a name\n    of an object and *toctree* the :toctree: path of the corresponding\n    autosummary directive (relative to the root of the file name), and\n    *template* the value of the :template: option. *toctree* and\n    *template* ``None`` if the directive does not have the\n    corresponding options set.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states the function returns a list of (name, toctree, template) tuples, but the answer claims it returns AutosummaryEntry named tuples with four fields: name, path, template, and recursive. This could be a version difference or the answer may reflect newer code, but it contradicts the documentation's explicit description of 3-tuple returns.",
      "The documentation says 'toctree' is the field name for the path, but the answer claims the field is called 'path' instead of 'toctree'.",
      "Many implementation details (line numbers, specific regex patterns, str.removeprefix usage, pydoc.getdoc calls, etc.) cannot be verified from the provided documentation and may be hallucinated or based on a specific version of the code.",
      "The documentation explicitly states returns are '(name, toctree, template)' with toctree and template being None when not set, but the answer describes a 4-field NamedTuple with 'recursive' as an additional field - this directly contradicts the documented return format."
    ],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped code; only minor descriptive wording was adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_30",
    "repo": "sphinx",
    "question": "Why does `is_allowed_version` only actually take effect for `doctest` directives even though `testcode` and `testoutput` also accept the `pyversion` option?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `is_allowed_version` function implements a Python version check for doctest directives, conditionally skipping doctest blocks based on the Python version running the tests.\n\n**Behavior:** When a `.. doctest::` directive includes the `:pyversion:` option (e.g., `:pyversion: >=3.6`), `is_allowed_version` checks whether the current Python interpreter's version satisfies the given version specifier (using the `packaging` library's PEP-440 specifier support). If the version check fails, the doctest is marked with the `SKIP` flag so it won't actually be executed during builds.\n\n**Implementation details:**\n1. The function delegates entirely to the `packaging` library: it creates a `Version` object from the version string and checks membership in a `SpecifierSet` created from the spec string (line 60: `return Version(version) in SpecifierSet(spec)`).\n\n2. The function is called at **directive parse time** (in `TestDirective.run()` at line 130-136), NOT at test execution time. The current Python version is computed as `'.'.join(map(str, sys.version_info[:3]))` \u2014 i.e., the first three components of `sys.version_info`.\n\n3. When version check fails, the `doctest.OPTIONFLAGS_BY_NAME['SKIP']` flag is set to `True` in the node's options dict, which then propagates to the `DocTestRunner` through the `TestCode.options` attribute. During test execution (lines 575-579), these options are merged into each example's options before running.\n\n4. **Inconsistency in scope:** Although `pyversion` is declared as an option in `DoctestDirective`, `TestcodeDirective`, AND `TestoutputDirective` option_specs (lines 167, 177, 188), the version-check logic in `TestDirective.run()` is guarded by `if self.name == 'doctest'` (line 130), meaning it only takes effect for `.. doctest::` directives, not `.. testcode::` or `.. testoutput::` directives despite those accepting the option.\n\n5. Invalid specifiers are caught via `except InvalidSpecifier` and reported as a docutils warning rather than raising an error (lines 137-140).",
    "rubric": [
      "Explains that the pyversion check in TestDirective.run() is guarded by a condition checking self.name == 'doctest', so it only fires for doctest directives",
      "Notes that TestcodeDirective and TestoutputDirective both declare pyversion in their option_spec, creating an inconsistency where the option is accepted but never acted upon",
      "Describes what is_allowed_version does: uses the packaging library's Version and SpecifierSet to check if the current Python version satisfies a PEP-440 specifier",
      "Explains that when the version check fails, the SKIP doctest flag is set in the node's options, causing the test to be skipped at execution time",
      "Mentions that invalid specifiers are caught and reported as warnings rather than raising errors"
    ],
    "key_files": [
      "sphinx/ext/doctest.py",
      "tests/test_extensions/test_ext_doctest.py"
    ],
    "source_doc": "[docstring: sphinx/ext/doctest.py] sphinx.ext.doctest.is_allowed_version\nCheck `spec` satisfies `version` or not.\n\n    This obeys PEP-440 specifiers:\n    https://peps.python.org/pep-0440/#version-specifiers\n\n    Some examples:\n\n        >>> is_allowed_version('<=3.5', '3.3')\n        True\n        >>> is_allowed_version('<=3.2', '3.3')\n        False\n        >>> is_allowed_version('>3.2, <4.0', '3.3')\n        True",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claims about specific line numbers (e.g., line 60, 130-136, 575-579, 167, 177, 188, 137-140) cannot be verified from the provided documentation and may be fabricated",
      "Claims about implementation details (packaging library usage, TestDirective.run(), SKIP flag behavior, InvalidSpecifier handling) go far beyond what the documentation states and cannot be verified",
      "The claim about inconsistency in scope regarding pyversion option is unverifiable from the provided documentation",
      "The core claim about PEP-440 compliance is supported by the documentation"
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are directly verifiable from the stripped source code; minor wording about 'design purpose' and 'PEP-440' was softened.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_32",
    "repo": "sphinx",
    "question": "How does the intersphinx `_get_safe_url` function obscure passwords in URLs, and how does it differ from `_strip_basic_auth`?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `_get_safe_url` function (in `sphinx/ext/intersphinx/_load.py`) implements URL password obscuring using the following algorithm:\n\n1. It uses `urllib.parse.urlsplit(url)` to parse the URL into its components (scheme, netloc, path, query, fragment).\n2. It checks if `parts.username` is `None`. If so, it returns the original URL unchanged.\n3. If a username exists, it converts the `SplitResult` to a mutable list via `list(parts)` (resulting in a 5-element list corresponding to scheme, netloc, path, query, fragment).\n4. It then reconstructs the netloc (index 1 of the list):\n   - If `parts.port` is truthy (a port is specified), it sets `frags[1]` to `f'{parts.username}@{parts.hostname}:{parts.port}'` \u2014 keeping the username and port but dropping the password.\n   - If no port is present, it sets `frags[1]` to `f'{parts.username}@{parts.hostname}'` \u2014 keeping only the username.\n5. Finally, it calls `urllib.parse.urlunsplit(frags)` to reassemble the URL string.\n\nThis contrasts with the related `_strip_basic_auth` function (also in the same file), which removes BOTH the username and password entirely by splitting the netloc on `'@'` and keeping only the part after it (the hostname/port).\n\nThe function is called in `_fetch_inventory_group` when logging the inventory location being loaded. Both `urlsplit` and `urlunsplit` are imported from `urllib.parse` at the top of the module.",
    "rubric": [
      "Mentions that `_get_safe_url` uses `urlsplit` to parse the URL into components",
      "Explains that if no username is present (`parts.username is None`), the original URL is returned unchanged",
      "Describes converting the SplitResult to a mutable list and reconstructing the netloc (index 1) without the password",
      "Explains the port-handling branching: if a port exists, the netloc includes username@hostname:port; otherwise just username@hostname",
      "Mentions that `urlunsplit` is used to reassemble the final URL",
      "Contrasts with `_strip_basic_auth`, which removes both username and password by splitting on '@' and keeping only the hostname/port portion"
    ],
    "key_files": [
      "sphinx/ext/intersphinx/_load.py",
      "tests/test_ext_intersphinx/test_ext_intersphinx.py",
      "sphinx/ext/intersphinx/_shared.py"
    ],
    "source_doc": "[docstring: sphinx/ext/intersphinx/_load.py] sphinx.ext.intersphinx._load._get_safe_url\nGets version of *url* with basic auth passwords obscured. This function\n    returns results suitable for printing and logging.\n\n    E.g.: https://user:12345@example.com => https://user@example.com\n\n    :param url: a url\n    :type url: ``str``\n\n    :return: *url* with password removed\n    :rtype: ``str``",
    "verification_verdict": "warn",
    "verification_issues": [
      "The detailed algorithm (steps 1-5) describing the internal implementation using urlsplit, list conversion, netloc reconstruction, and urlunsplit is not present in the documentation \u2014 these are plausible implementation details but cannot be verified from the provided doc.",
      "The description of `_strip_basic_auth` function behavior is not mentioned anywhere in the documentation and cannot be verified.",
      "The claim about being called in `_fetch_inventory_group` is not supported by the documentation.",
      "The claim about specific import line numbers (line 12) is not supported by the documentation.",
      "The core documented behavior \u2014 that the function obscures passwords in URLs and the example transformation (https://user:12345@example.com => https://user@example.com) \u2014 is correctly represented in the answer."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are directly verifiable from the stripped source code; only minor explanatory phrases about intent were slightly adjusted.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_31",
    "repo": "sphinx",
    "question": "Where is the intersphinx extension's functionality split across modules, and where do inventory creation, fetching, caching, and reference resolution happen?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The intersphinx extension is located in `sphinx/ext/intersphinx/`, split across six modules: `__init__.py` (setup and public API), `_shared.py` (shared data structures), `_load.py` (inventory fetching/caching), `_resolve.py` (reference resolution), `_cli.py` (CLI tool, imported as `inspect_main`), and `__main__.py` (entry point).\n\n**Where inventory files are created:** The `objects.inv` file is written by calling `InventoryFile.dump()` (defined in `sphinx/util/inventory.py`). The constant `INVENTORY_FILENAME = 'objects.inv'` is defined in `sphinx/builders/html/__init__.py`.\n\n**Where inventories are fetched and loaded:** In `_load.py`, `load_mappings()` uses `concurrent.futures.ThreadPoolExecutor` to fetch inventories in parallel. Each inventory is fetched by `_fetch_inventory_group()`, which tries each location in order. When a location is `None`, it defaults to `posixpath.join(project.target_uri, INVENTORY_FILENAME)`. Remote inventory data is cached on disk at `app.doctreedir / '__intersphinx_cache__' / f'{project.name}_{INVENTORY_FILENAME}'`. Cache expiry is calculated as `now - config.intersphinx_cache_limit * 86400`.\n\n**Where references are resolved:** The `missing_reference()` function in `_resolve.py` is connected to Sphinx's `'missing-reference'` event in `__init__.py`'s `setup()`. It delegates to `resolve_reference_detect_inventory()`, which first tries the main (combined) inventory, then attempts to split the target on `:` to extract an explicit inventory name (`inv_name:target`). The `intersphinx_resolve_self` config value enables self-referential resolution by setting `node['intersphinx_self_referential'] = True` and modifying the reftarget.\n\n**Where the `:external:` role is dispatched:** `IntersphinxDispatcher` (a `CustomReSTDispatcher` subclass) is installed on every `'source-read'` event via `install_dispatcher()`. It intercepts role names starting with `'external:'` or `'external+'` (checking `len(role_name) > 9`) and returns an `IntersphinxRole` instance. The `IntersphinxRole.run()` method sets `node['intersphinx'] = True` and `node['inventory'] = inventory` on pending_xref nodes.\n\n**Where intersphinx pending_xrefs are post-processed:** `IntersphinxRoleResolver` extends `ReferencesResolver` (from `sphinx.transforms.post_transforms`) and is added as a post-transform in `setup()`. It is registered via `app.add_post_transform(IntersphinxRoleResolver)`.\n\n**Where inventory data is parsed:** `InventoryFile.loads()` in `sphinx/util/inventory.py` handles both v1 and v2 inventory formats. V2 inventories use zlib compression and a regex `r'(.+?)\\s+(\\S+)\\s+(-?\\d+)\\s?(\\S*)\\s+(.*)'` to parse each line. Case-insensitive matching is specifically handled for `std:label` and `std:term` types during both loading (detecting ambiguities) and resolution.\n\n**Where the setup registers config values:** The `setup()` function registers five config values: `intersphinx_mapping` (dict), `intersphinx_resolve_self` (str, default `''`), `intersphinx_cache_limit` (int, default 5), `intersphinx_timeout` (float/int/None, default None), and `intersphinx_disabled_reftypes` (list, default `['std:doc']`).",
    "rubric": [
      "Identifies the six modules in sphinx/ext/intersphinx/: __init__.py, _shared.py, _load.py, _resolve.py, _cli.py, and __main__.py",
      "States that objects.inv is written by InventoryFile.dump() in sphinx/util/inventory.py and the INVENTORY_FILENAME constant is defined in sphinx/builders/html/__init__.py",
      "Explains that inventory fetching happens in _load.py using concurrent.futures.ThreadPoolExecutor for parallel loading, with _fetch_inventory_group trying each location in order",
      "Notes that when a location is None, it defaults to posixpath.join(project.target_uri, INVENTORY_FILENAME)",
      "Describes the on-disk cache path as app.doctreedir / '__intersphinx_cache__' / f'{project.name}_{INVENTORY_FILENAME}' and cache expiry as now - config.intersphinx_cache_limit * 86400",
      "Explains that missing_reference() in _resolve.py is connected to the 'missing-reference' event and delegates to resolve_reference_detect_inventory()",
      "Describes how resolve_reference_detect_inventory splits the target on ':' to extract an explicit inventory name",
      "Notes that IntersphinxDispatcher intercepts roles starting with 'external:' or 'external+' and returns IntersphinxRole instances that set node['intersphinx'] = True",
      "Explains that IntersphinxRoleResolver is a post-transform (subclass of ReferencesResolver) registered via app.add_post_transform",
      "Lists the five config values registered in setup(): intersphinx_mapping, intersphinx_resolve_self, intersphinx_cache_limit, intersphinx_timeout, intersphinx_disabled_reftypes"
    ],
    "key_files": [
      "sphinx/ext/intersphinx/__init__.py",
      "sphinx/ext/intersphinx/_load.py",
      "sphinx/ext/intersphinx/_resolve.py",
      "sphinx/ext/intersphinx/_shared.py",
      "sphinx/util/inventory.py",
      "sphinx/builders/html/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/intersphinx/__init__.py] sphinx.ext.intersphinx\nInsert links to objects documented in remote Sphinx documentation.\n\nThis works as follows:\n\n* Each Sphinx HTML build creates a file named \"objects.inv\" that contains a\n  mapping from object names to URIs relative to the HTML set's root.\n\n* Projects using the Intersphinx extension can specify links to such mapping\n  files in the `intersphinx_mapping` config value.  The mapping will then be\n  used to resolve otherwise missing references to objects into links to the\n  other documentation.\n\n* By default, the mapping file is assumed to be at the same location as the\n  rest of the documentation; however, the location of the mapping file can\n  also be specified individually, e.g. if the docs should be buildable\n  without Internet access.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (line numbers, function names, module structures, config defaults, regex patterns, etc.) that go far beyond what the provided documentation chunk describes. These cannot be verified from the given documentation alone.",
      "The documentation states the mapping file is 'assumed to be at the same location as the rest of the documentation' by default. The answer says 'When a location is None, it defaults to posixpath.join(project.target_uri, INVENTORY_FILENAME)' which is a plausible code-level equivalent but cannot be verified from the doc.",
      "The documentation mentions 'intersphinx_mapping' config value. The answer mentions this plus four additional config values (intersphinx_resolve_self, intersphinx_cache_limit, intersphinx_timeout, intersphinx_disabled_reftypes) which cannot be verified from the provided doc chunk.",
      "The core claims that are verifiable \u2014 objects.inv is created during HTML builds, it maps object names to URIs, intersphinx_mapping is used to specify links to mapping files, and the mapping resolves missing references \u2014 are all consistent with the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with line numbers, truncated code sections, and one interpretive comment about days-to-seconds conversion.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_33",
    "repo": "sphinx",
    "question": "What does `_strip_basic_auth` do, how does it differ from `_get_safe_url`, and where is each one called in the intersphinx loading flow?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`_strip_basic_auth` is a private function in `sphinx/ext/intersphinx/_load.py` that uses `urlsplit` and `urlunsplit` from `urllib.parse` (imported at line 12) to remove basic authentication credentials from URLs. The function works by splitting the URL into its 5 components via `urlsplit`, converting the result to a list (`frags`), then checking if `'@'` is present in `frags[1]` (the netloc component). If so, it splits the netloc on `'@'` and keeps only the part after the `@` (i.e., `frags[1].split('@')[1]`), effectively removing the `user:password@` prefix. Finally, it reconstructs the URL with `urlunsplit(frags)` and returns it as a single string.\n\nThe actual implementation only returns a `str` \u2014 the cleaned URL. It does NOT return the extracted credentials; they are discarded.\n\n`_strip_basic_auth` is called exactly once in production code: in `_fetch_inventory_data()` (line 364), which calls `_strip_basic_auth(target_uri)` only when the `target_uri` contains `'://'` (i.e., it's a remote URL). This strips auth from the target URI before it's used for inventory fetching. Notably, the `inv_location` parameter is NOT stripped of auth by `_strip_basic_auth` \u2014 it's passed as-is to `_fetch_inventory_url`, which sends it directly to `requests.get()`, meaning the library's HTTP client can still use the credentials in the inventory location URL.\n\nA companion function `_get_safe_url` (line 439) serves a different purpose: it only removes the *password* but keeps the *username* in the URL. It uses `urlsplit`'s parsed `parts.username` and `parts.hostname` properties rather than raw string splitting on `@`. `_get_safe_url` is called in `_fetch_inventory_group` when logging which inventory is being loaded.\n\nThe `_get_tls_cacert` function in `sphinx/util/requests.py` also has a similar pattern: it handles `@` in the netloc by using `hostname.partition('@')[-1]` to extract the hostname for TLS certificate lookup.\n\nThe test class `TestStripBasicAuth` in `tests/test_ext_intersphinx/test_ext_intersphinx.py` tests three cases: URL with auth (`https://user:12345@domain.com/project/objects.inv` \u2192 `https://domain.com/project/objects.inv`), URL without auth (unchanged), and URL with auth and a port (`https://user:12345@domain.com:8080/project/objects.inv` \u2192 `https://domain.com:8080/project/objects.inv`).",
    "rubric": [
      "_strip_basic_auth removes both username and password from the URL by splitting the netloc on '@' and keeping only the part after it",
      "_strip_basic_auth uses urlsplit/urlunsplit and raw string splitting on '@' in frags[1] (the netloc component)",
      "_strip_basic_auth returns only the cleaned URL string \u2014 it does NOT return the extracted credentials despite the docstring suggesting otherwise",
      "_get_safe_url removes only the password but keeps the username in the URL, using urlsplit's parsed properties (parts.username, parts.hostname, parts.port)",
      "_strip_basic_auth is called in _fetch_inventory_data on target_uri (only when target_uri contains '://'), but NOT on inv_location",
      "inv_location is passed as-is (with credentials intact) to _fetch_inventory_url / requests.get, so the HTTP client can still authenticate",
      "_get_safe_url is called in _fetch_inventory_group when logging which inventory is being loaded, to produce a safe-to-print URL",
      "Tests cover three cases for _strip_basic_auth: URL with auth, URL without auth, and URL with auth plus a port number"
    ],
    "key_files": [
      "sphinx/ext/intersphinx/_load.py",
      "sphinx/ext/intersphinx/_shared.py",
      "sphinx/util/requests.py",
      "tests/test_ext_intersphinx/test_ext_intersphinx.py"
    ],
    "source_doc": "[docstring: sphinx/ext/intersphinx/_load.py] sphinx.ext.intersphinx._load._strip_basic_auth\nReturns *url* with basic auth credentials removed. Also returns the\n    basic auth username and password if they're present in *url*.\n\n    E.g.: https://user:pass@example.com => https://example.com\n\n    *url* need not include basic auth credentials.\n\n    :param url: url which may or may not contain basic auth credentials\n    :type url: ``str``\n\n    :return: *url* with any basic auth creds removed\n    :rtype: ``str``",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 'the actual implementation only returns a str \u2014 the cleaned URL. It does NOT return the extracted credentials; they are discarded.' However, the docstring says 'Also returns the basic auth username and password if they're present in *url*' while the :rtype: says 'str'. There is a contradiction between the docstring's prose and its :rtype:, and the answer takes the position that the implementation only returns a string. This may be correct based on code inspection, but it contradicts part of the docstring.",
      "Many implementation details (urlsplit/urlunsplit usage, line numbers, companion functions like _get_safe_url, _get_tls_cacert, test cases, _fetch_inventory_data call site) cannot be verified from the provided documentation alone \u2014 they are plausible extrapolations from code but unverifiable from the doc chunk."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "One claim about a docstring's content was removed as it references documentation not present in stripped code; all other claims are verifiable from code logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_34",
    "repo": "sphinx",
    "question": "Why does intersphinx's `resolve_reference_detect_inventory` use a two-phase lookup strategy with a pre-merged `main_inventory` instead of iterating over each named inventory?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `resolve_reference_detect_inventory` function uses a two-phase resolution strategy that exploits a pre-merged `main_inventory` data structure.\n\n**Key design: The `main_inventory` avoids iterating over all named inventories.**\n\nDuring inventory loading (`_load.py`), all named inventories are merged into a single `main_inventory` dictionary (keyed by `objtype` \u2192 `target` \u2192 `_InventoryItem`). This means the first lookup (`resolve_reference_any_inventory`) performs a single O(1) dictionary lookup per objtype in this merged inventory, rather than iterating over every named inventory separately.\n\nThe function tries the direct lookup first:\n1. **The direct lookup in `main_inventory`** handles targets in O(1) per objtype via `target in inventory[objtype]` (in `_resolve_reference_in_domain_by_target`).\n2. **The string splitting and inventory existence check are avoided** when unnecessary \u2014 if `:` isn't in the target (`if ':' not in target: return None`), the function returns immediately without the overhead of `target.partition(':')` or `inventory_exists()` checks.\n3. **When no colon is present, the function exits early**, avoiding the second resolution pass entirely.\n\nThe second phase (splitting into `inv_name:new_target`) only triggers when the fast path fails AND the target contains a colon. This involves looking up a specific named inventory (`InventoryAdapter(env).named_inventory[inv_name]`).\n\nAdditionally, `_resolve_reference` forces `honor_disabled_refs = honor_disabled_refs and inv_name is None`, meaning when looking up in a specific named inventory, disabled reftypes are NOT checked \u2014 eliminating an extra set membership test per objtype in the targeted lookup path.\n\n`ReferencesResolver` has `default_priority = 10`. The `IntersphinxRoleResolver` is intended to run before it (at `ReferencesResolver.default_priority - 1 = 9`), ensuring explicit `:external:` roles are resolved before the general `ReferencesResolver`.",
    "rubric": [
      "Explains that the main_inventory merges all named inventories into a single dict (keyed by objtype \u2192 target), enabling O(1) lookups instead of iterating over all inventories",
      "Describes the first phase: direct lookup in main_inventory via resolve_reference_any_inventory, which avoids per-inventory iteration",
      "Describes the early-exit optimization: if the target has no colon, the function returns None immediately without attempting the second phase",
      "Describes the second phase: splitting target on ':' to form inv_name:new_target, then looking up in a specific named inventory only if the first phase failed",
      "Mentions that honor_disabled_refs is forced to False when inv_name is not None, skipping disabled reftype checks in the targeted inventory lookup",
      "Notes IntersphinxRoleResolver runs at priority ReferencesResolver.default_priority - 1, ensuring explicit :external: roles resolve before the general missing-reference handler"
    ],
    "key_files": [
      "sphinx/ext/intersphinx/_resolve.py",
      "sphinx/ext/intersphinx/_shared.py",
      "sphinx/ext/intersphinx/_load.py",
      "sphinx/transforms/post_transforms/__init__.py"
    ],
    "source_doc": "[docstring: sphinx/ext/intersphinx/_resolve.py] sphinx.ext.intersphinx._resolve.resolve_reference_detect_inventory\nAttempt to resolve a missing reference via intersphinx references.\n\n    Resolution is tried first with the target as is in any inventory.\n    If this does not succeed, then the target is split by the first ``:``,\n    to form ``inv_name:new_target``. If ``inv_name`` is a named inventory, then resolution\n    is tried in that inventory with the new target.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer introduces many specific implementation details (line numbers, O(1) complexity claims, `main_inventory` merging, `honor_disabled_refs`, priority values, etc.) that are not present in the documentation and cannot be verified from it alone.",
      "The core two-phase resolution strategy described in the answer (try target as-is first, then split by first colon to form inv_name:new_target) is consistent with the documentation.",
      "Claims about `_load.py` lines 205-208, line 318, line 324, line 94, line 204 are specific code references that cannot be verified from the provided documentation but are plausible extrapolations from source code.",
      "The characterization of the function as a 'performance optimization' with 'key performance design' framing goes beyond what the documentation states \u2014 the doc simply describes the resolution strategy without mentioning performance motivations.",
      "The claim about `IntersphinxRoleResolver` running at priority 9 is not mentioned in the documentation but is a plausible code-level detail."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code structure and control flow; a few claims about design rationale ('common case', 'performance optimization', 'relatively rare') rely on unstated intent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_35",
    "repo": "sphinx",
    "question": "Where is `IntersphinxRole` defined and how does it get invoked during document processing?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `IntersphinxRole` class is defined in `sphinx/ext/intersphinx/_resolve.py` (line 369). It inherits from `SphinxRole` (defined in `sphinx/util/docutils.py`, line 608). The `IntersphinxDispatcher` class (line 350 of `_resolve.py`) is also in the same file and acts as the entry point \u2014 its `role()` method creates `IntersphinxRole` instances when a role name starts with `'external:'` or `'external+'` and has length > 9.\n\nKey location details:\n\n1. **`IntersphinxDispatcher`** is installed via `install_dispatcher()` (line 572), which is connected to the `'source-read'` event in `sphinx/ext/intersphinx/__init__.py` (line 81: `app.connect('source-read', install_dispatcher)`). The dispatcher's `enable()` method monkey-patches `docutils.parsers.rst.roles.role` via the parent class `CustomReSTDispatcher.enable()` (in `sphinx/util/docutils.py`, line 265-270).\n\n2. **`IntersphinxRole`** is created by `IntersphinxDispatcher.role()` at line 364 with the original role name passed to the constructor (`self.orig_name = orig_name`). When called, `SphinxRole.__call__` (line 633 of docutils.py) lowercases the name into `self.name` and then calls `self.run()`. The `run()` method asserts `self.name == self.orig_name.lower()` at line 379.\n\n3. **`IntersphinxRole.run()`** delegates to a found `role_func` (line 465), passing `f'{domain_name}:{role_name}'` as the first argument. After the role function returns result nodes, if not self-referential, it marks each `pending_xref` node with `node['intersphinx'] = True` and `node['inventory'] = inventory` (lines 481-482).\n\n4. **`IntersphinxRoleResolver`** (line 536) is a post-transform that resolves these marked nodes. It extends `ReferencesResolver` (from `sphinx/transforms/post_transforms/__init__.py`, line 62) and has `default_priority = ReferencesResolver.default_priority - 1`, which equals **9** (since `ReferencesResolver.default_priority = 10`). Since `IntersphinxRoleResolver` has a lower priority number, it runs before `ReferencesResolver` in the transform pipeline.\n\n5. The `_re_inv_ref` regex at line 373 (`re.compile(r'(\\+([^:]+))?:(.*)')`) is defined as a class attribute but is never actually used in the current code \u2014 the `get_inventory_and_name_suffix` method uses string slicing and `partition(':')` instead.\n\n6. When no domain is specified in the role name, `IntersphinxRole.run()` checks both the default domain and the standard domain (lines 429-435). If the default domain is the same as the standard domain, only the standard domain is checked.\n\n7. The `intersphinx_resolve_self` config value (registered in `__init__.py` line 68 as an empty string default) controls self-referential resolution. When `resolve_self == inventory`, the role skips intersphinx attribute injection, allowing normal Sphinx resolution to handle the reference instead.",
    "rubric": [
      "IntersphinxRole is defined in sphinx/ext/intersphinx/_resolve.py and inherits from SphinxRole",
      "IntersphinxDispatcher.role() creates IntersphinxRole instances when a role name starts with 'external:' or 'external+'",
      "IntersphinxDispatcher is installed via install_dispatcher() connected to the 'source-read' event, and it monkey-patches docutils.parsers.rst.roles.role through CustomReSTDispatcher.enable()",
      "IntersphinxRoleResolver is a post-transform that resolves pending_xref nodes marked with intersphinx=True, running at priority 9 (one before ReferencesResolver's priority of 10)",
      "IntersphinxRole.run() marks resulting pending_xref nodes with node['intersphinx'] = True and node['inventory'] = inventory"
    ],
    "key_files": [
      "sphinx/ext/intersphinx/_resolve.py",
      "sphinx/util/docutils.py",
      "sphinx/ext/intersphinx/__init__.py",
      "sphinx/transforms/post_transforms/__init__.py",
      "sphinx/ext/intersphinx/_shared.py"
    ],
    "source_doc": "[docstring: sphinx/ext/intersphinx/_resolve.py] sphinx.ext.intersphinx._resolve.IntersphinxRole\nsphinx.ext.intersphinx._resolve.IntersphinxRole.get_inventory_and_name_suffix:\n  Extract an inventory name (if any) and ``domain+name`` suffix from a role *name*.\n        and the domain+name suffix.\n\n        The role name is expected to be of one of the following forms:\n\n        - ``external+inv:name`` -- explicit inventory and name, any domain.\n        - ``external+inv:domain:name`` -- explicit inventory, domain and name.\n        - ``external:name`` -- any inventory and domain, explicit name.\n        - ``external:domain:name`` -- any inventory, explicit domain and name.\n\nsphinx.ext.intersphinx._resolve.IntersphinxRole._get_domain_role:\n  Convert the *name* string into a domain and a role name.\n\n        - If *name* contains no ``:``, return ``(None, name)``.\n        - If *name* contains a single ``:``, the domain/role is split on this.\n        - If *name* contains multiple ``:``, return ``(None, None)``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific line numbers, implementation details, and behavioral claims that cannot be verified from the provided documentation snippet alone. These could be accurate code-level details or hallucinations - the documentation is too limited to confirm or deny most of them.",
      "The claim about `_re_inv_ref` regex being 'never actually used' cannot be verified from the documentation provided.",
      "The claim about `intersphinx_resolve_self` config value cannot be verified from the documentation provided.",
      "The description of `get_inventory_and_name_suffix` is consistent with the documentation - it mentions extracting inventory name and domain+name suffix, and the role name forms match.",
      "The description of `_get_domain_role` behavior is consistent with the documentation - the rules about colons match exactly.",
      "The claim that `get_inventory_and_name_suffix` uses 'string slicing and partition' rather than the regex is unverifiable from the docs but doesn't contradict anything stated."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from the stripped source code; only a minor interpretive claim about transform ordering was slightly reworded.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_36",
    "repo": "sphinx",
    "question": "How does Napoleon's setup function integrate the extension into Sphinx, and what does it register?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `sphinx.ext.napoleon.setup()` function integrates the Napoleon extension into Sphinx through the following mechanism:\n\n1. **Guard against non-Sphinx callers**: If `app` is not a `Sphinx` instance, it immediately returns a metadata dict with `version` and `parallel_read_safe: True` without doing any registration.\n\n2. **Patches the Python domain**: Calls `_patch_python_domain()` which modifies `PyObject.doc_field_types`. Specifically, it finds the existing 'parameter' field (which originally has names `('param', 'parameter', 'arg', 'argument', 'keyword', 'kwarg', 'kwparam')`) and truncates its names to `('param', 'parameter', 'arg', 'argument')`. Then it appends a new `PyTypedField` named 'keyword' with label 'Keyword Arguments', names `('keyword', 'kwarg', 'kwparam')`, typerolename `'class'`, typenames `('paramtype', 'kwtype')`, and `can_collapse=True`.\n\n3. **Sets up autodoc dependency**: Calls `app.setup_extension('sphinx.ext.autodoc')`.\n\n4. **Connects two event handlers**:\n   - `'autodoc-process-docstring'` \u2192 `_process_docstring`: Processes docstrings by first passing them through `NumpyDocstring` (if `napoleon_numpy_docstring` is enabled), then through `GoogleDocstring` (if `napoleon_google_docstring` is enabled). The result of NumpyDocstring processing becomes the input to GoogleDocstring processing.\n   - `'autodoc-skip-member'` \u2192 `_skip_member`: Controls whether private/special/init members with docstrings are included based on config settings.\n\n5. **Registers 16 configuration values**: Iterates over `Config._config_values` and calls `app.add_config_value()` for each, all with rebuild type `'env'`. These include settings like `napoleon_google_docstring` (default True), `napoleon_numpy_docstring` (default True), `napoleon_use_param` (default True), etc.\n\n6. **Returns metadata**: Returns `{'version': sphinx.__display_version__, 'parallel_read_safe': True}`.\n\nThe `NumpyDocstring` class extends `GoogleDocstring`. `GoogleDocstring.__init__` builds a `_sections` dictionary with 32 built-in section mappings (args, arguments, attributes, example, examples, parameters, returns, raises, etc.) and then calls `_load_custom_sections()` and finally calls `_parse()`.",
    "rubric": [
      "Mentions the guard clause that returns early with metadata if app is not a Sphinx instance",
      "Describes _patch_python_domain() splitting keyword args from param field into a separate PyTypedField",
      "Mentions setup_extension('sphinx.ext.autodoc') is called",
      "Identifies the two event handlers connected: autodoc-process-docstring to _process_docstring and autodoc-skip-member to _skip_member",
      "Explains that _process_docstring runs NumpyDocstring first then GoogleDocstring (with numpy output feeding into google)",
      "Mentions that config values from Config._config_values are registered with rebuild type 'env'",
      "States the returned metadata dict contains version and parallel_read_safe: True"
    ],
    "key_files": [
      "sphinx/ext/napoleon/__init__.py",
      "sphinx/domains/python/_object.py",
      "sphinx/ext/napoleon/docstring.py"
    ],
    "source_doc": "[docstring: sphinx/ext/napoleon/__init__.py] sphinx.ext.napoleon.setup\nSphinx extension setup function.\n\n    When the extension is loaded, Sphinx imports this module and executes\n    the ``setup()`` function, which in turn notifies Sphinx of everything\n    the extension offers.\n\n    Parameters\n    ----------\n    app : sphinx.application.Sphinx\n        Application object representing the Sphinx process\n\n    See Also\n    --------\n    `The Sphinx documentation on Extensions\n    <https://www.sphinx-doc.org/extensions.html>`_\n\n    `The Extension Tutorial <https://www.sphinx-doc.org/extdev/tutorial.html>`_\n\n    `The Extension API <https://www.sphinx-doc.org/extdev/appapi.html>`_",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the function's purpose at a high level (setup function that notifies Sphinx of everything the extension offers) and its parameter. The generated answer provides extremely detailed implementation specifics (16 config values, specific field names, version numbers like '9.1.0', 32 built-in section mappings, etc.) that are not verifiable from the provided documentation alone.",
      "The claim about __version__ being '9.1.0' is a specific version number that could become outdated and is not mentioned in the documentation.",
      "The detailed implementation claims (patch_python_domain behavior, specific field names like 'kwarg'/'kwparam', NumpyDocstring extending GoogleDocstring, 32 built-in section mappings) are plausible code-level details but entirely unsupported by the provided documentation chunk.",
      "None of the claims directly contradict the documentation - they are extrapolations from the actual source code rather than from the provided doc chunk."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor purpose descriptions and version-specific details were rewritten or removed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_37",
    "repo": "sphinx",
    "question": "What is the architecture of the SphinxPostTransform system \u2014 its class hierarchy, filtering mechanism, registration, and execution flow?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `SphinxPostTransform` class is a base class defined in `sphinx/transforms/post_transforms/__init__.py` that inherits from `SphinxTransform` (defined in `sphinx/transforms/__init__.py`), which in turn inherits from docutils' `Transform` class.\n\n**Class hierarchy**: `docutils.transforms.Transform` \u2192 `SphinxTransform` \u2192 `SphinxPostTransform`\n\n**Key architecture details:**\n\n1. **Filtering mechanism**: `SphinxPostTransform` has two class-level tuple attributes for filtering: `builders` (defaults to empty tuple `()`) and `formats` (defaults to empty tuple `()`). The `is_supported()` method checks `self.env._builder_cls.name` against `self.builders` and `self.env._builder_cls.format` against `self.formats`. When both tuples are empty, the transform runs for all builders. The logic is: if `builders` is non-empty, the builder's `name` must be in the tuple; if `formats` is non-empty, the builder's `format` must be in the tuple.\n\n2. **apply() wraps run()**: The `apply()` method first calls `is_supported()`, and only calls `run()` if supported. Subclasses override `run()` rather than `apply()`. The `run()` method raises `NotImplementedError` by default.\n\n3. **Registration and execution**: Post transforms are registered via `app.add_post_transform()`, which stores them in `self.registry.post_transforms` (a list in `SphinxExtensionRegistry`). They are applied during the write phase via `BuildEnvironment.apply_post_transforms()`, which creates a `SphinxTransformer`, adds all registered post transforms, and calls `apply_transforms()`. After post-transforms complete, the `'doctree-resolved'` event is emitted.\n\n4. **Parent class properties**: `SphinxTransform` provides three properties: `app` (deprecated, scheduled for removal in version 11.0, emits a deprecation warning and returns `self.env._app`), `env` (returns `self.document.settings.env`), and `config` (returns `self.env.config`).\n\n5. **Concrete subclasses in the module**: Four post transforms are registered in `setup()`: `ReferencesResolver` (priority 10), `OnlyNodeTransform` (priority 50), `SigElementFallbackTransform` (priority 200), and `PropagateDescDomain` (priority 200). None of these set `builders` or `formats`, so they run for all builders.\n\n6. **Format-specific examples**: `SubstitutionDefinitionsRemover` and `ShowUrlsTransform` in `sphinx/builders/latex/transforms.py` set `formats = ('latex',)`. `HyperlinkCollector` in `sphinx/builders/linkcheck.py` sets `builders = ('linkcheck',)`. The `ViewcodeAnchorTransform` in `sphinx/ext/viewcode.py` neither sets `builders` nor `formats` but does its own builder check inside `run()`.\n\n7. **Builder attributes checked**: The `Builder` class in `sphinx/builders/__init__.py` defines `name: ClassVar[str] = ''` and `format: ClassVar[str] = ''` as class variables, which are what `is_supported()` checks against.\n\n8. **Environment state during post-transforms**: `apply_post_transforms()` temporarily sets `self.current_document` to a deep-copied object with the docname set, then restores it in a `finally` block. The `_builder_cls` attribute is excluded from pickling (removed in `__getstate__`).",
    "rubric": [
      "Identifies the class hierarchy: docutils Transform \u2192 SphinxTransform \u2192 SphinxPostTransform",
      "Explains the builders and formats tuple attributes used for filtering, defaulting to empty tuples meaning the transform runs for all builders",
      "Describes how is_supported() checks env._builder_cls.name against builders and env._builder_cls.format against formats",
      "Explains that apply() delegates to run() only if is_supported() returns True, and subclasses override run() (which raises NotImplementedError by default)",
      "Describes how post transforms are registered via app.add_post_transform() which stores them in registry.post_transforms list",
      "Explains that apply_post_transforms() in BuildEnvironment creates a SphinxTransformer, adds registered post transforms, applies them, then emits 'doctree-resolved' event",
      "Mentions SphinxTransform provides app (deprecated), env, and config properties",
      "Lists concrete subclasses registered in setup(): ReferencesResolver (priority 10), OnlyNodeTransform (priority 50), SigElementFallbackTransform (priority 200), PropagateDescDomain (priority 200)",
      "Gives format-specific examples like SubstitutionDefinitionsRemover with formats=('latex',) or HyperlinkCollector with builders=('linkcheck',)"
    ],
    "key_files": [
      "sphinx/transforms/post_transforms/__init__.py",
      "sphinx/transforms/__init__.py",
      "sphinx/environment/__init__.py",
      "sphinx/builders/__init__.py",
      "sphinx/registry.py"
    ],
    "source_doc": "[docstring: sphinx/transforms/post_transforms/__init__.py] sphinx.transforms.post_transforms.SphinxPostTransform\nsphinx.transforms.post_transforms.SphinxPostTransform:\n  A base class of post-transforms.\n\n    Post transforms are invoked to modify the document to restructure it for outputting.\n    They resolve references, convert images, do special transformation for each output\n    formats and so on.  This class helps to implement these post transforms.\n\nsphinx.transforms.post_transforms.SphinxPostTransform.is_supported:\n  Check this transform working for current builder.\n\nsphinx.transforms.post_transforms.SphinxPostTransform.run:\n  Main method of post transforms.\n\n        Subclasses should override this method instead of ``apply()``.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most claims in the answer go far beyond what the provided documentation states, drawing on source code details not present in the doc chunk. These are plausible code-level extrapolations but cannot be verified from the given documentation alone.",
      "The core claims that are verifiable from the doc \u2014 SphinxPostTransform is a base class for post-transforms, is_supported() checks if the transform works for the current builder, run() is the main method that subclasses should override instead of apply() \u2014 are all correctly stated.",
      "Claims about class hierarchy (inheriting from SphinxTransform), filtering mechanisms (builders/formats tuples), specific subclasses, priority numbers, and other implementation details cannot be verified from the provided documentation but are presented as factual."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code; only a minor naming correction was needed (SphinxComponentRegistry vs SphinxExtensionRegistry).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_38",
    "repo": "sphinx",
    "question": "Why does `SigElementFallbackTransform` skip its fallback logic for `SphinxTranslator` subclasses but convert `desc_sig_*` nodes to plain inlines for other translators?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `SigElementFallbackTransform` handles the difference between Sphinx's custom signature nodes (like `desc_sig_space`, `desc_sig_name`, etc.) and translators that don't inherit from `SphinxTranslator`. The code shows it performs fallback conversion for non-SphinxTranslator translators while returning early for SphinxTranslator subclasses.\n\n1. **MRO-based dispatch in SphinxTranslator**: `SphinxTranslator.dispatch_visit()` (in `sphinx/util/docutils.py`) walks the `__mro__` of a node class to find a visitor method. Since `desc_sig_element` inherits from `nodes.inline` (in `sphinx/addnodes.py`: `class desc_sig_element(nodes.inline, _desc_classes_injector)`), SphinxTranslator subclasses can handle all `desc_sig_*` nodes through MRO resolution to `visit_inline`. This is why the transform immediately returns when `issubclass(translator, SphinxTranslator)` is True.\n\n2. **Fallback for non-Sphinx translators**: The fallback transform converts `desc_sig_element` nodes to standard `nodes.inline` instances that any translator can handle. In the `fallback` method: `newnode = nodes.inline()` is created, attributes are copied via `newnode.update_all_atts(node)`, children are moved with `newnode.extend(node)`, and the original node is replaced with `node.replace_self(newnode)`.\n\n3. **Two-tier fallback logic for desc_sig_element nodes**: The transform checks two conditions before triggering the fallback: (a) the translator must NOT have individual visitors for ALL nodes in `SIG_ELEMENTS`, AND (b) it must NOT have a generic `visit_desc_sig_element` method. If either condition is satisfied (all per-node visitors exist, or a generic visitor exists), no fallback occurs. The code structure allows third-party translators to implement either per-node or generic visitors.\n\n4. **Preserving identity via `_sig_node_type`**: The `node.tagname` (e.g., `'desc_sig_space'`) is stored in a `_sig_node_type` attribute on the replacement `nodes.inline` using `setdefault` (`newnode.setdefault('_sig_node_type', node.tagname)`). This preserves the original node type after conversion. The `setdefault` ensures user-defined values aren't overwritten.\n\n5. **HTML5Translator's `supported_inline_tags`**: The HTML5Translator sets `supported_inline_tags: set[str] = set()` (in `sphinx/writers/html5.py`). The `desc_sig_element` subclasses use short class names like `classes = ['s']` (for `desc_sig_literal_string`), `['w']` (for `desc_sig_space`), `['n']` (for `desc_sig_name`), which are visible in `sphinx/addnodes.py`.\n\n6. **Graceful handling of dummy builders**: The `try/except AttributeError` around `get_translator_class` handles the case where the builder has no `default_translator_class` attribute, as `get_translator_class` in the registry (`sphinx/registry.py`) raises `AttributeError` if no translator is found \u2014 the method tries `builder.default_translator_class` in a `try` block and re-raises `AttributeError` with a descriptive message.",
    "rubric": [
      "Explains that SphinxTranslator.dispatch_visit() walks the node class's __mro__ to find visitor methods, so desc_sig_* nodes naturally resolve to visit_inline since desc_sig_element inherits from nodes.inline",
      "Describes the fallback conversion: creating a nodes.inline, copying attributes with update_all_atts, moving children with extend, and replacing the original node via replace_self",
      "Explains the two-tier condition: fallback only triggers when the translator lacks BOTH individual visitors for ALL SIG_ELEMENTS AND a generic visit_desc_sig_element method",
      "Mentions that the original node type is preserved via setdefault('_sig_node_type', node.tagname) so the node identity is not lost after conversion",
      "Notes the try/except AttributeError around get_translator_class to gracefully handle dummy builders that lack a default_translator_class",
      "Mentions that desc_sig_element inherits from nodes.inline (and _desc_classes_injector), which is the key inheritance that enables MRO-based dispatch in SphinxTranslator"
    ],
    "key_files": [
      "sphinx/transforms/post_transforms/__init__.py",
      "sphinx/util/docutils.py",
      "sphinx/addnodes.py",
      "sphinx/writers/html5.py",
      "sphinx/registry.py"
    ],
    "source_doc": "[docstring: sphinx/transforms/post_transforms/__init__.py] sphinx.transforms.post_transforms.SigElementFallbackTransform\nsphinx.transforms.post_transforms.SigElementFallbackTransform:\n  Fallback various desc_* nodes to inline if translator does not support them.\n\nsphinx.transforms.post_transforms.SigElementFallbackTransform.fallback:\n  Translate nodes of type *node_type* to docutils inline nodes.\n\n        The original node type name is stored as a string in a private\n        ``_sig_node_type`` attribute if the latter did not exist.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states that `_sig_node_type` is stored 'if the latter did not exist', while the answer says it uses `setdefault` to ensure 'user-defined values aren't overwritten' - this is a plausible implementation detail but goes beyond the documentation.",
      "The documentation says the original 'node type name' is stored as a string, and the answer says 'node.tagname' is stored - this is a plausible extrapolation but not explicitly stated in the documentation.",
      "Most of the answer's claims about line numbers, MRO dispatch, HTML5 translator behavior, SIG_ELEMENTS checking, and specific implementation details are not present in the provided documentation and appear to be extrapolations from source code not included in the doc chunk.",
      "The core claim about `_sig_node_type` preserving identity is consistent with the documentation's statement about storing the 'original node type name' in that attribute."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed docutils behavioral details and design rationale about Pygments-inspired names and visit_inline HTML tag conversion.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_39",
    "repo": "sphinx",
    "question": "Where does a log message flow after calling `sphinx.util.logging.getLogger()` and issuing a warning?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "When `sphinx.util.logging.getLogger(name)` is called, the data flows through the following path:\n\n1. **Namespace prefixing**: The `getLogger` function (line 59-78 of `sphinx/util/logging.py`) prepends the `NAMESPACE` constant (which is `'sphinx'`) to the provided name, creating a logger named `'sphinx.' + name`. It then forces `logger.disabled = False` and wraps it in a `SphinxLoggerAdapter`.\n\n2. **SphinxLoggerAdapter.process()**: When a log method (e.g., `warning()`, `info()`) is called, the `process()` method extracts special keywords (`'type'`, `'subtype'`, `'location'`, `'nonl'`, `'color'`, `'once'`) from kwargs and moves them into the `extra` dict, making them attributes on the LogRecord.\n\n3. **Handler routing via setup()**: The `setup()` function (called in `Sphinx.__init__` at line 249 of `application.py`) installs three handlers on the `'sphinx'` namespace logger:\n   - `NewLineStreamHandler` (for info/status, level set by VERBOSITY_MAP)\n   - `WarningStreamHandler` (for warnings/errors, level WARNING)\n   - A `messagelog_handler` (stores last 10 messages in `app.messagelog` deque)\n\n4. **Warning filter chain**: The `WarningStreamHandler` applies filters in this order:\n   - `_RaiseOnWarningFilter` (only if `app._exception_on_warning` is True) \u2014 raises `SphinxWarning` exception\n   - `WarningSuppressor` \u2014 checks `app.config.suppress_warnings` and increments `app._warncount`\n   - `WarningLogRecordTranslator` \u2014 converts the record class to `SphinxWarningLogRecord` and resolves location (tuple/Node/string) to a path string; also appends `[type.subtype]` if `show_warning_types` is True\n   - `OnceFilter` \u2014 deduplicates messages that have `once=True`\n\n5. **Info filter chain**: The `NewLineStreamHandler` applies `InfoFilter` (passes only records with levelno < WARNING) and `InfoLogRecordTranslator` (converts record class to `SphinxInfoLogRecord`).\n\n6. **Parallel processing**: In `ParallelTasks._process()` (sphinx/util/parallel.py), log records are collected via `LogCollector`, serialized using `convert_serializable()` (which extracts args into msg and converts Node locations to strings), sent through a pipe, and then replayed via `logger.handle(log)` in the parent process (`_join_one()`).\n\n7. **Output formatting**: `ColorizeFormatter` applies colour based on the `color` attribute on the record or falls back to `COLOR_MAP` (ERROR\u2192'darkred', WARNING\u2192'red', DEBUG\u2192'darkgray'). `SafeEncodingWriter` handles UnicodeEncodeError by encoding/replacing non-encodable characters.",
    "rubric": [
      "Explains that getLogger prepends the 'sphinx' namespace to the logger name and wraps it in a SphinxLoggerAdapter",
      "Describes how SphinxLoggerAdapter.process() extracts special keywords (type, subtype, location, nonl, color, once) into the extra dict",
      "Identifies that setup() installs handlers on the 'sphinx' namespace logger including WarningStreamHandler and NewLineStreamHandler",
      "Describes the WarningStreamHandler filter chain: WarningSuppressor checks suppress_warnings, WarningLogRecordTranslator resolves location and converts record class, OnceFilter deduplicates",
      "Mentions _RaiseOnWarningFilter which raises SphinxWarning if exception_on_warning is enabled",
      "Explains how ColorizeFormatter applies color to output based on record attributes or COLOR_MAP defaults",
      "Describes the parallel processing path where LogCollector captures records, convert_serializable prepares them for serialization, and they are replayed in the parent process"
    ],
    "key_files": [
      "sphinx/util/logging.py",
      "sphinx/application.py",
      "sphinx/util/parallel.py",
      "sphinx/errors.py"
    ],
    "source_doc": "[docstring: sphinx/util/logging.py] sphinx.util.logging.getLogger\nGet logger wrapped by :class:`sphinx.util.logging.SphinxLoggerAdapter`.\n\n    Sphinx logger always uses ``sphinx.*`` namespace to be independent from\n    settings of root logger.  It ensures logging is consistent even if a\n    third-party extension or imported application resets logger settings.\n\n    Example usage::\n\n        >>> from sphinx.util import logging\n        >>> logger = logging.getLogger(__name__)\n        >>> logger.info('Hello, this is an extension!')\n        Hello, this is an extension!",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_40",
    "repo": "sphinx",
    "question": "How does Sphinx's HTML theme system discover, load, and resolve theme inheritance?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "Sphinx's HTML theme system is implemented primarily in `sphinx/theming.py` and works through a multi-layered architecture involving theme discovery, inheritance resolution, configuration parsing, and template/static file integration.\n\n## Theme Discovery and Loading\n\nThe `HTMLThemeFactory` class manages theme loading through three sources, loaded in this order:\n\n1. **Built-in themes**: `_load_builtin_themes()` scans `sphinx/themes/` directory.\n\n2. **Additional theme paths**: `_load_additional_themes()` resolves paths from the `html_theme_path` config relative to `confdir`.\n\n3. **Entry points**: `_load_entry_point_themes()` discovers themes registered under the `sphinx.html_themes` entry point group. These are loaded lazily \u2014 the factory stores a closure that calls `app.setup_extension(theme_module)` only when the theme is actually requested via `create()`.\n\nThe `_find_themes()` static method detects themes in two forms: directories containing either `theme.toml` or `theme.conf`, and `.zip` files (validated by `_is_archived_theme()` which checks if the archive contains either config file at its root).\n\n## Dual Configuration Format\n\nThemes support two configuration file formats:\n- **`theme.toml`** (TOML format parsed with `tomllib`): Uses `[theme]` table with `inherit`, `stylesheets` (list), `sidebars` (list), and `pygments_style` (a sub-table with `default` and `dark` keys).\n- **`theme.conf`** (INI format parsed with `configparser.RawConfigParser`): Uses `[theme]` section with `inherit`, `stylesheet` (comma-separated), `sidebars` (comma-separated), `pygments_style`, and `pygments_dark_style`.\n\nBoth are converted to a common `_ConfigFile` object (with `__slots__`) that holds: `stylesheets`, `sidebar_templates`, `pygments_style_default`, `pygments_style_dark`, and `options`.\n\n## Theme Inheritance Resolution\n\nThe `_load_theme_with_ancestors()` function walks the inheritance chain. It iterates up to **10 ancestors** (hardcoded limit \u2014 exceeding it raises `ThemeError` with \"too many ancestors\"). The chain terminates when `inherit = \"none\"`. Circular inheritance is detected by checking if an ancestor name already appeared in the resolved themes dict.\n\nZipped themes are extracted to a temporary directory created with `tempfile.mkdtemp('sxt')`, and the `Theme._cleanup()` method removes these temp dirs.\n\n## Configuration Merging in Theme.__init__\n\nThe `Theme.__init__` iterates through ancestor configs in **reverse order** (base theme first, derived theme last). Options are merged with `|=` (so derived theme options override base), while `stylesheets`, `sidebar_templates`, and pygments styles are simply overwritten by the most-derived theme that defines them.\n\n## Integration with HTML Builder\n\n`StandaloneHTMLBuilder.init_templates()` creates the `HTMLThemeFactory`, calls `create()` to get a `Theme`, and passes it to `BuiltinTemplateLoader.init()`. The template loader builds a search chain from all theme directories (and their parents), prepending user-configured `templates_path` entries. Templates prefixed with `!` skip user template directories and search only theme directories.\n\nThe `get_options()` method logs a warning for any option key in overrides that isn't defined in the theme's options.\n\nThe `basic` theme defines default sidebars: `localtoc.html`, `relations.html`, `sourcelink.html`, `searchbox.html`.",
    "rubric": [
      "Explains three discovery sources: built-in themes (sphinx/themes directory), additional theme paths (html_theme_path config relative to confdir), and entry points (sphinx.html_themes group)",
      "Mentions that entry point themes are loaded lazily via a closure that calls app.setup_extension only when the theme is actually requested",
      "Describes the two config formats supported: theme.toml (TOML) and theme.conf (INI), both converted to a common _ConfigFile object",
      "Explains inheritance resolution walks up to 10 ancestors (hardcoded limit), terminates at inherit='none', and detects circular inheritance",
      "Describes how configs are merged in reverse order (base first, derived last) with options using |= and stylesheets/sidebars/pygments being overwritten by the most-derived theme",
      "Mentions that zipped themes are extracted to temp directories (tempfile.mkdtemp('sxt')) and cleaned up via _cleanup()"
    ],
    "key_files": [
      "sphinx/theming.py",
      "sphinx/builders/html/__init__.py",
      "sphinx/jinja2glue.py",
      "sphinx/registry.py",
      "sphinx/themes/basic/theme.toml"
    ],
    "source_doc": "[doc_file: doc/development/html_themes/index.rst] HTML theme development\n.. versionadded:: 0.6\n\n.. note::\n\n   This document provides information about creating your own theme. If you\n   simply wish to use a pre-existing HTML themes, refer to\n   :doc:`/usage/theming`.\n\nSphinx supports changing the appearance of its HTML output via *themes*.  A\ntheme is a collection of HTML templates, stylesheet(s) and other static files.\nAdditionally, it has a configuration file which specifies from which theme to\ninherit, which highlighting style to use, and what options exist for customizing\nthe theme's look and feel.\n\nThemes are meant to be project-unaware, so they can be used for different\nprojects without change.\n\n.. note::\n\n   See :ref:`dev-extensions` for more information that may\n   be helpful in developing themes.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (e.g., exact number of built-in themes, ancestor limit of 10, `_ConfigFile` dataclass, `tempfile.mkdtemp('sxt')`, migration tool, etc.) that go far beyond what the provided documentation covers. The documentation only describes themes at a high level: they are collections of HTML templates, stylesheets, and static files with a configuration file specifying inheritance, highlighting style, and customization options.",
      "While none of the claims directly contradict the provided documentation, they cannot be verified against it either \u2014 they appear to be sourced from actual code inspection rather than from this documentation chunk.",
      "The documentation states themes have 'a configuration file' (singular), while the answer describes two configuration formats (theme.toml and theme.conf). This is not contradicted but is a significant extrapolation beyond what the doc says.",
      "The answer's claims about 13 specific built-in themes, entry point discovery, TOML/INI parsing, inheritance chain limits, template loader behavior, and static file copying are all plausible code-level details but are entirely unsupported by the provided documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few claims about specific built-in theme lists, inheritance chains, and truncated builder methods rely on documentation or unseen files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_41",
    "repo": "sphinx",
    "question": "What is the structure of Sphinx's theming system, including theme configuration, inheritance resolution, and discovery mechanisms?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "In Sphinx's theming system, a theme is a structured package (directory or zipfile) that provides configuration for rendering documentation output. The system is implemented primarily in `sphinx/theming.py` through two main classes: `Theme` and `HTMLThemeFactory`.\n\n**Theme Configuration Structure (theme.toml):**\nThe `theme.toml` file has two main TOML tables:\n- `[theme]` (required): Must contain an `inherit` key (string, required), and optionally `stylesheets` (list of strings), `sidebars` (list of strings), and `pygments_style` (a table with optional `default` and `dark` keys).\n- `[options]` (optional): A dictionary of theme-specific options with string values.\n\nThe `inherit` field is mandatory\u2014if set to `\"none\"`, the theme has no parent (only the `basic` theme does this). Otherwise it names a parent theme to inherit from.\n\n**Theme Resolution and Ancestor Chain:**\nThe function `_load_theme_with_ancestors()` resolves the full inheritance chain. It enforces a hard limit of 10 ancestor themes (using `for _ in range(10)`, raising `ThemeError` if exceeded). It also detects circular inheritance and raises a `ThemeError`.\n\n**Theme Discovery (`_find_themes`):**\nThe `HTMLThemeFactory._find_themes()` static method searches a directory for themes in two ways:\n1. Subdirectories containing either a `theme.toml` or `theme.conf` file\n2. Zip files (`.zip` extension) that are validated by `_is_archived_theme()`, which checks if the zip contains either `theme.toml` or `theme.conf` at the root level\n\n**Internal Configuration Representation:**\nBoth `theme.toml` and `theme.conf` are parsed into a `_ConfigFile` dataclass-like object (uses `__slots__`) with five attributes: `stylesheets`, `sidebar_templates`, `pygments_style_default`, `pygments_style_dark`, and `options`.\n\n**Integration with HTML Builder:**\nThe `StandaloneHTMLBuilder.init_templates()` method creates an `HTMLThemeFactory` and calls `create(theme_name)` to instantiate the theme. The builder then uses `theme.stylesheets` for CSS files and `theme.pygments_style_default`/`theme.pygments_style_dark` for syntax highlighting configuration.\n\n**Entry Point Loading:**\nExternal themes can register via the `sphinx.html_themes` entry point group. These are loaded lazily\u2014only when actually needed\u2014by calling `app.setup_extension(theme_module)` on demand.\n\n**Migration Tool:**\nThe module includes a `_migrate_conf_to_toml()` function that converts legacy `theme.conf` (INI format) files to `theme.toml` format.",
    "rubric": [
      "Mentions the two main classes: Theme and HTMLThemeFactory",
      "Describes theme.toml structure with [theme] table (inherit required, stylesheets, sidebars, pygments_style) and optional [options] table",
      "Explains the inherit field is mandatory; 'none' means no parent theme",
      "Describes _load_theme_with_ancestors enforcing a limit of 10 ancestors and detecting circular inheritance",
      "Explains theme discovery via _find_themes: subdirectories with theme.toml/theme.conf, and zip files validated by _is_archived_theme",
      "Mentions the _ConfigFile internal representation with its five slots (stylesheets, sidebar_templates, pygments_style_default, pygments_style_dark, options)",
      "Describes entry point loading via 'sphinx.html_themes' group with lazy/deferred loading through app.setup_extension",
      "Mentions the _migrate_conf_to_toml function for converting legacy theme.conf to theme.toml"
    ],
    "key_files": [
      "sphinx/theming.py",
      "sphinx/builders/html/__init__.py",
      "sphinx/registry.py",
      "sphinx/themes/basic/theme.toml"
    ],
    "source_doc": "[doc_file: doc/development/html_themes/index.rst] Creating themes\nThemes take the form of either a directory or a zipfile (whose name is the\ntheme name), containing the following:\n\n* Either a :file:`theme.toml` file (preferred) or a :file:`theme.conf` file.\n* HTML templates, if needed.\n* A ``static/`` directory containing any static files that will be copied to the\n  output static directory on build.  These can be images, styles, script files.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for a comment reference and a partially verifiable description.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_42",
    "repo": "sphinx",
    "question": "Why does Sphinx use a `TemplateBridge` abstraction instead of directly coupling builders to Jinja2 for template rendering?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `TemplateBridge` class serves as an abstraction layer between Sphinx's builders and template engine implementations. The `create_template_bridge()` mechanism allows swapping out the default Jinja2 templating engine with an alternative engine specified via configuration.\n\n**How the TemplateBridge pattern works in the code:**\n\n1. **Indirection via configuration**: The `create_template_bridge()` method in `sphinx/builders/__init__.py` checks `self.config.template_bridge`. If set, it uses `import_object()` to dynamically import and instantiate the user-specified class. If not set (default is `None` as shown in `config.py`'s `config_values`), it falls back to `BuiltinTemplateLoader` from `sphinx.jinja2glue`. The Jinja2 dependency is only imported when actually needed \u2014 the import `from sphinx.jinja2glue import BuiltinTemplateLoader` is inside the `else` branch, not at the top of the file.\n\n2. **Interface definition**: `TemplateBridge` (defined in `sphinx/application.py` and imported in `sphinx/jinja2glue.py`) defines the methods `init()`, `newest_template_mtime()`, `render()`, and `render_string()` that template implementations must provide.\n\n3. **Multiple inheritance in the default implementation**: `BuiltinTemplateLoader` inherits from both `TemplateBridge` AND Jinja2's `BaseLoader` (`class BuiltinTemplateLoader(TemplateBridge, BaseLoader)`). This dual inheritance allows it to function as both a Sphinx template bridge and a Jinja2 template loader. The `get_source()` method implements a `!` prefix convention: templates starting with `!` skip the first `self.templatepathlen` loaders (corresponding to user-specified template paths) and search only from the remaining loaders (theme directories).\n\n4. **Sandboxed environment**: The default `BuiltinTemplateLoader` uses Jinja2's `SandboxedEnvironment` rather than a plain `Environment`.\n\n5. **Flexible initialization**: The `BuiltinTemplateLoader.init()` method accepts optional `theme` and `dirs` parameters, allowing different builders to initialize templates differently \u2014 with a theme's directory chain, with explicit directories, or with neither.",
    "rubric": [
      "Explains that TemplateBridge allows swapping the default Jinja2 engine for an alternative template engine via configuration (template_bridge config value)",
      "Describes how create_template_bridge() checks config.template_bridge and dynamically imports a user-specified class, falling back to BuiltinTemplateLoader only when unset",
      "Notes that the Jinja2 import (BuiltinTemplateLoader from sphinx.jinja2glue) is deferred inside the else branch, not at the top of the file, so Jinja2 is only needed when actually used",
      "Mentions that TemplateBridge defines an interface with methods like init(), render(), render_string(), and newest_template_mtime() that implementations must provide",
      "Explains that BuiltinTemplateLoader uses dual inheritance from both TemplateBridge and Jinja2's BaseLoader, serving as both a Sphinx template bridge and a Jinja2 loader",
      "Notes that BuiltinTemplateLoader uses SandboxedEnvironment rather than a plain Jinja2 Environment"
    ],
    "key_files": [
      "sphinx/application.py",
      "sphinx/builders/__init__.py",
      "sphinx/jinja2glue.py",
      "sphinx/config.py"
    ],
    "source_doc": "[doc_file: doc/development/html_themes/templating.rst] Do I need to use Sphinx's templates to produce HTML?\nNo.  You have several other options:\n\n* You can write a :class:`~sphinx.application.TemplateBridge` subclass that\n  calls your template engine of choice, and set the :confval:`template_bridge`\n  configuration value accordingly.\n\n* You can :ref:`write a custom builder <writing-builders>` that derives from\n  :class:`~sphinx.builders.html.StandaloneHTMLBuilder` and calls your template\n  engine of choice.\n\n* You can use the :class:`~sphinxcontrib.serializinghtml.PickleHTMLBuilder` that\n  produces pickle files with the page contents, and postprocess them using a\n  custom tool, or use them in your Web application.\n\n.. _templating-primer:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (line numbers, method signatures, inheritance hierarchies, security details) that go far beyond what the provided documentation chunk discusses. These claims cannot be verified from the given documentation alone.",
      "The documentation only mentions TemplateBridge as one of three options for avoiding Sphinx's default templates, while the answer elaborates extensively on its internal architecture with specific claims about line numbers and implementation details that cannot be verified from this doc chunk.",
      "The answer's claims about specific line numbers (line 168, line 1852, line 161) and specific implementation details (SandboxedEnvironment, '!' prefix convention, NotImplementedError messages) are not verifiable from the provided documentation but appear to be plausible code-level details.",
      "The documentation does confirm that TemplateBridge exists in sphinx.application and that template_bridge is a configuration value, which is consistent with the answer's claims."
    ],
    "strip_verify_leakage": 0.3,
    "strip_verify_summary": "Several claims about design rationale and purpose rely on documentation rather than code, while the structural and behavioral claims are verifiable from stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_43",
    "repo": "sphinx",
    "question": "Where is the template inheritance mechanism with the `!` prefix implemented, and how does it interact with the template search chain?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The exclamation mark (`!`) prefix for template inheritance is implemented in the `BuiltinTemplateLoader.get_source()` method in `sphinx/jinja2glue.py`. When a template name starts with `!`, the method skips the user-configured template paths by slicing the loaders list: `loaders = loaders[self.templatepathlen:]`. The `self.templatepathlen` is set to `len(builder.config.templates_path)` during initialization, so the exclamation mark causes the search to begin from the theme's directories, bypassing any user overrides.\n\nThe template search chain is constructed in `BuiltinTemplateLoader.init()`. When a theme is present, the `loaderchain` is built as the theme's directories (from `theme.get_theme_dirs()`) PLUS the parent directories of each theme dir (`[p.parent for p in pathchain]`).\n\nThe user's `templates_path` directories are prepended to both `pathchain` and `loaderchain` at index 0 (`pathchain[0:0] = cfg_templates_path`), giving them highest priority during normal template resolution.\n\nThe environment uses `jinja2.sandbox.SandboxedEnvironment` with the `BuiltinTemplateLoader` itself as the loader (since it extends both `TemplateBridge` and Jinja2's `BaseLoader`). Custom Jinja2 filters (`tobool`, `toint`, `todim`, `slice_index`) and globals (`debug`, `warning`, `accesskey`, `idgen`) are registered on the environment.\n\nThe `SphinxFileSystemLoader` also supports a template naming fallback: if a template name ends with `.jinja`, it will also search for the same name with a `_t` suffix instead.",
    "rubric": [
      "Identifies that the `!` prefix is handled in `BuiltinTemplateLoader.get_source()` in `sphinx/jinja2glue.py`",
      "Explains that the `!` prefix skips user template paths by slicing loaders with `loaders[self.templatepathlen:]`",
      "Explains that `templatepathlen` is set to `len(builder.config.templates_path)` during init",
      "Describes the template search chain construction: theme dirs + parent dirs of theme dirs form `loaderchain`",
      "Notes that user `templates_path` directories are prepended at index 0, giving them highest priority in normal resolution",
      "Mentions that `SphinxFileSystemLoader` also supports a `.jinja` to `_t` suffix fallback for template names"
    ],
    "key_files": [
      "sphinx/jinja2glue.py",
      "sphinx/theming.py",
      "sphinx/builders/__init__.py",
      "sphinx/themes/basic/layout.html"
    ],
    "source_doc": "[doc_file: doc/development/html_themes/templating.rst] Jinja/Sphinx Templating Primer\nThe default templating language in Sphinx is Jinja.  It's Django/Smarty inspired\nand easy to understand.  The most important concept in Jinja is :dfn:`template\ninheritance`, which means that you can overwrite only specific blocks within a\ntemplate, customizing it while also keeping the changes at a minimum.\n\nTo customize the output of your documentation you can override all the templates\n(both the layout templates and the child templates) by adding files with the\nsame name as the original filename into the template directory of the structure\nthe Sphinx quickstart generated for you.\n\nSphinx will look for templates in the folders of :confval:`templates_path`\nfirst, and if it can't find the template it's looking for there, it falls back\nto the selected theme's templates.\n\nA template contains **variables**, which are replaced with values when the\ntemplate is evaluated, **tags**, which control the logic of the template and\n**blocks** which are used for template inheritance.\n\nSphinx's *basic* theme provides base templates with a couple of blocks it will\nfill with data.  These are located in the :file:`themes/basic` subdirectory of\nthe Sphinx installation directory, and used by all builtin Sphinx themes.\nTemplates with the same name in the :confval:`templates_path` override templates\nsupplied by the selected theme.\n\nFor example, to add a new link to the template area containing related links all\nyou have to do is to add a new template called ``layout.html`` with the\nfollowing contents::\n\n    {% extends \"!layout.html\" %}\n    {% block rootrellink %}\n        <li><a href=\"https://project.invalid/\">Project Homepage</a> &raquo;</li>\n        {{ super() }}\n    {% endblock %}\n\nBy prefixing the name of the overridden template with an exclamation mark,\nSphinx will load the layout template from the underlying HTML theme.\n\n.. important::\n   If you override a block, call ``{{ super() }}`` somewhere to render the\n   block's original content in the extended template -- unless you don't want\n   that content to show up.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in design rationale for parent dirs and characterization of '_t' as legacy naming convention; line numbers also unverifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_44",
    "repo": "sphinx",
    "question": "How does the recipe domain extension implement its custom Sphinx domain, from directive processing through cross-reference resolution and index generation?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The recipe domain extension implements a custom Sphinx domain through several interconnected classes. Here is how the implementation works in detail:\n\n**RecipeDirective (extends ObjectDescription \u2192 SphinxDirective \u2192 docutils.parsers.rst.Directive)**:\n- `handle_signature(sig, signode)` simply appends an `addnodes.desc_name(text=sig)` node to the signature node and returns the raw `sig` string. The base `ObjectDescription.handle_signature()` raises `ValueError` by default, meaning every subclass must override it.\n- `add_target_and_index(name_cls, sig, signode)` sets the signode's `ids` to `'recipe-' + sig`, splits the `contains` option on commas (stripping whitespace), then calls `self.env.get_domain('recipe').add_recipe(sig, ingredients)`.\n- The parent class `ObjectDescription.run()` does the heavy lifting: it creates an `addnodes.desc` node, iterates over signatures from `get_signatures()`, calls `handle_signature()` inside a try/except\u2014if `ValueError` is raised, the signode is cleared and replaced with a plain `desc_name` node, and `add_target_and_index` is skipped. If successful and the name is not a duplicate, `add_target_and_index` is called only when `no-index` is not set. After processing signatures, it calls `before_content()`, parses nested content via `parse_content_to_nodes(allow_section_headings=True)`, calls `transform_content()`, emits an `'object-description-transform'` event, runs `DocFieldTransformer(self).transform_all()`, then calls `after_content()`. The method returns `[self.indexnode, node]` normally, or just `[self.indexnode]` / `[self.indexnode, target_node]` if `no-typesetting` is set.\n- The `ObjectDescription` base class also supports `_toc_parts` and `_toc_name` private attributes for table of contents generation, set in the `finally` block.\n\n**Domain.__init__()** performs `copy.deepcopy(self.initial_data)` and adds a `'version'` key set to `self.data_version`. If the domain data already exists in `env.domaindata` and the version doesn't match, it raises `OSError('data of %r domain out of date')`. It also converts class-level `object_types`, `directives`, `roles`, and `indices` to instance-level copies (dict/list).\n\n**Domain.directive()** method creates a `DirectiveAdapter` inner class that subclasses the registered directive and overrides `run()` to set `self.name = fullname` (e.g., `'recipe:recipe'`). `ObjectDescription.run()` splits on `:` to extract `self.domain` and `self.objtype`.\n\n**RecipeDomain.add_recipe()** stores recipes as 6-tuples `(name, signature, 'Recipe', docname, anchor, 0)` where name is `f'recipe.{signature}'`, anchor is `f'recipe-{signature}'`, and priority is `0`. It also stores ingredients in `self.data['recipe_ingredients']` keyed by the canonical name.\n\n**RecipeDomain.resolve_xref()** iterates over `get_objects()` and matches where `sig == target`. It calls `make_refnode()` from `sphinx.util.nodes`, which creates a `nodes.reference` node. When no match is found, the recipe domain prints `'Awww, found nothing'` to stdout and returns `None`.\n\n**Index classes**: Both `IngredientIndex` and `RecipeIndex` extend `Index` (from `sphinx.domains._index`), which is an ABC requiring `generate()`. The `Index.__init__()` validates that `self.name` is truthy and `self.localname is not None`, raising `SphinxError` otherwise. Both generate methods return `(content, True)`. `IngredientIndex.generate()` inverts the recipe\u2192ingredients mapping to ingredient\u2192recipes before producing entries. `RecipeIndex.generate()` groups recipes by the first letter (lowercased) of the display name.\n\n**Domain.setup()** registers hyperlink targets for each index by calling `std.note_hyperlink_target(docname, docname, '', index.localname)` where `docname = f'{self.name}-{index.name}'`, e.g., `'recipe-recipe'` and `'recipe-ingredient'`.\n\n**XRefRole** (used as `RecipeDomain.roles['ref']`) creates `addnodes.pending_xref` nodes with `nodes.literal` inner nodes by default. Its `run()` method splits the role name on `:` to determine `refdomain` and `reftype`, and sets CSS classes like `['xref', 'recipe', 'recipe-ref']`.\n\n**setup() function** returns metadata with `version='0.1'`, `parallel_read_safe=True`, `parallel_write_safe=True`.",
    "rubric": [
      "Explains that RecipeDirective extends ObjectDescription and its handle_signature appends a desc_name node and returns the sig string",
      "Describes how add_target_and_index sets the signode ids to 'recipe-' + sig and splits the 'contains' option on commas to get ingredients",
      "Explains the ObjectDescription.run() flow: creates desc node, iterates signatures, calls handle_signature in try/except (ValueError causes fallback), calls add_target_and_index if not no-index, parses content, runs DocFieldTransformer, and returns [indexnode, node]",
      "Describes Domain.__init__() performing copy.deepcopy of initial_data, adding a 'version' key, and raising OSError if data version doesn't match",
      "Explains Domain.directive() creates a DirectiveAdapter inner class that sets self.name to the full 'domain:name' string",
      "Describes RecipeDomain.add_recipe() storing 6-tuples (name, signature, 'Recipe', docname, anchor, 0) where name is f'recipe.{signature}' and anchor is f'recipe-{signature}'",
      "Explains resolve_xref iterates get_objects matching sig == target, returns make_refnode on match or prints 'Awww, found nothing' and returns None",
      "Describes how IngredientIndex.generate() inverts recipe\u2192ingredients to ingredient\u2192recipes, and RecipeIndex.generate() groups by first letter lowercased",
      "Mentions that setup() returns metadata with version='0.1' and both parallel_read_safe and parallel_write_safe set to True"
    ],
    "key_files": [
      "doc/development/tutorials/examples/recipe.py",
      "sphinx/directives/__init__.py",
      "sphinx/domains/__init__.py",
      "sphinx/domains/_index.py",
      "sphinx/roles.py",
      "sphinx/util/nodes.py",
      "sphinx/registry.py"
    ],
    "source_doc": "[doc_file: doc/development/tutorials/adding_domain.rst] Writing the extension\nOpen :file:`recipe.py` and paste the following code in it, all of which we will\nexplain in detail shortly:\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n\nLet's look at each piece of this extension step-by-step to explain what's going\non.\n\n.. rubric:: The directive class\n\nThe first thing to examine is the ``RecipeDirective`` directive:\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n   :pyobject: RecipeDirective\n\nUnlike :ref:`tutorial-extending-syntax` and :ref:`tutorial-extend-build`,\nthis directive doesn't derive from\n:class:`docutils.parsers.rst.Directive` and doesn't define a ``run`` method.\nInstead, it derives from :class:`sphinx.directives.ObjectDescription` and\ndefines  ``handle_signature`` and ``add_target_and_index`` methods. This is\nbecause ``ObjectDescription`` is a special-purpose directive that's intended\nfor describing things like classes, functions, or, in our case, recipes. More\nspecifically, ``handle_signature`` implements parsing the signature of the\ndirective and passes on the object's name and type to its superclass, while\n``add_target_and_index`` adds a target (to link to) and an entry to the index\nfor this node.\n\nWe also see that this directive defines ``has_content``, ``required_arguments``\nand ``option_spec``. Unlike the ``TodoDirective`` directive added in the\n:ref:`previous tutorial <tutorial-extend-build>`,\nthis directive takes a single argument,\nthe recipe name, and an option, ``contains``,\nin addition to the nested reStructuredText in the body.\n\n.. rubric:: The index classes\n\n.. currentmodule:: sphinx.domains\n\n.. todo:: Add brief overview of indices\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n   :pyobject: IngredientIndex\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n   :pyobject: RecipeIndex\n\nBoth ``IngredientIndex`` and ``RecipeIndex`` are derived from :class:`Index`.\nThey implement custom logic to generate a tuple of values that define the\nindex. Note that ``RecipeIndex`` is a simple index that has only one entry.\nExtending it to cover more object types is not yet part of the code.\n\nBoth indices use the method :meth:`Index.generate` to do their work. This\nmethod combines the information from our domain, sorts it, and returns it in a\nlist structure that will be accepted by Sphinx. This might look complicated but\nall it really is is a list of tuples like ``('tomato', 'TomatoSoup', 'test',\n'rec-TomatoSoup',...)``. Refer to the :doc:`domain API guide\n</extdev/domainapi>` for more information on this API.\n\nThese index pages can be referenced with the :rst:role:`ref` role by combining\nthe domain name and the index ``name`` value. For example, ``RecipeIndex`` can be\nreferenced with ``:ref:`recipe-recipe``` and ``IngredientIndex`` can be referenced\nwith ``:ref:`recipe-ingredient```.\n\n.. rubric:: The domain\n\nA Sphinx domain is a specialized container that ties together roles,\ndirectives, and indices, among other things. Let's look at the domain we're\ncreating here.\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n   :pyobject: RecipeDomain\n\nThere are some interesting things to note about this ``recipe`` domain and domains\nin general. Firstly, we actually register our directives, roles and indices\nhere, via the ``directives``, ``roles`` and ``indices`` attributes, rather than\nvia calls later on in ``setup``. We can also note that we aren't actually\ndefining a custom role and are instead reusing the\n:class:`sphinx.roles.XRefRole` role and defining the\n:class:`sphinx.domains.Domain.resolve_xref` method. This method takes two\narguments, ``typ`` and ``target``, which refer to the cross-reference type and\nits target name. We'll use ``target`` to resolve our destination from our\ndomain's ``recipes`` because we currently have only one type of node.\n\nMoving on, we can see that we've defined ``initial_data``. The values defined in\n``initial_data`` will be copied to ``env.domaindata[domain_name]`` as the\ninitial data of the domain, and domain instances can access it via\n``self.data``. We see that we have defined two items in ``initial_data``:\n``recipes`` and ``recipe_ingredients``. Each contains a list of all objects\ndefined (i.e. all recipes) and a hash that maps a canonical ingredient name to\nthe list of objects. The way we name objects is common across our extension and\nis defined in the ``get_full_qualified_name`` method. For each object created,\nthe canonical name is ``recipe.<recipename>``, where ``<recipename>`` is the\nname the documentation writer gives the object (a recipe). This enables the\nextension to use different object types that share the same name. Having a\ncanonical name and central place for our objects is a huge advantage. Both our\nindices and our cross-referencing code use this feature.\n\n.. rubric:: The ``setup`` function\n\n.. currentmodule:: sphinx.application\n\n:ref:`As always <tutorial-extend-build>`,\nthe ``setup`` function is a requirement and is used to\nhook the various parts of our extension into Sphinx. Let's look at the\n``setup`` function for this extension.\n\n.. literalinclude:: examples/recipe.py\n   :language: python\n   :linenos:\n   :pyobject: setup\n\nThis looks a little different to what we're used to seeing. There are no calls\nto :meth:`~Sphinx.add_directive` or even :meth:`~Sphinx.add_role`. Instead, we\nhave a single call to :meth:`~Sphinx.add_domain` followed by some\ninitialization of the :doc:`standard domain </usage/domains/standard>`.\nThis is because we had already registered our directives,\nroles and indexes as part of the directive itself.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (e.g., DirectiveAdapter inner class, Domain.__init__() performing copy.deepcopy, OSError('data of %r domain out of date'), XRefRole CSS classes like ['xref', 'recipe', 'recipe-ref'], etc.) that go far beyond what the documentation describes. These appear to be plausible code-level details derived from reading the actual source code rather than the documentation, but cannot be verified from the documentation alone.",
      "The answer states resolve_xref prints 'Awww, found nothing' to stdout - this is not mentioned in the documentation and is an implementation detail that could be a hallucination or could be from reading the actual source code.",
      "The answer states setup() returns metadata with 'version=0.1', 'parallel_read_safe=True', 'parallel_write_safe=True' - not explicitly stated in the documentation snippet provided, though plausible.",
      "The documentation says resolve_xref takes two arguments 'typ' and 'target', while the answer describes it as iterating over get_objects() and matching where sig == target. The doc doesn't describe the internal implementation of resolve_xref in that level of detail.",
      "The answer says add_recipe stores recipes as 6-tuples with specific fields - this level of detail is not in the provided documentation.",
      "The documentation says 'initial_data' contains 'recipes' (a list) and 'recipe_ingredients' (a hash/dict). The answer's description of these is consistent with the documentation.",
      "The documentation says RecipeDirective derives from ObjectDescription and defines handle_signature and add_target_and_index - this is correctly reflected in the answer.",
      "The documentation says directives, roles, and indices are registered in the domain class rather than in setup() - correctly reflected.",
      "The documentation says XRefRole is reused rather than defining a custom role - correctly reflected.",
      "The answer's description of index referencing with :ref:`recipe-recipe` and :ref:`recipe-ingredient` matches the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rely on documentation for semantic meaning (priority=0 meaning, True meaning collapsed, enabling :ref: references).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_45",
    "repo": "sphinx",
    "question": "What classes and functions does the recipe domain tutorial example depend on from Sphinx's core, and how do those base classes work internally?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The recipe domain tutorial example in `doc/development/tutorials/examples/recipe.py` depends on a chain of classes and functions from Sphinx's core domain infrastructure. The `RecipeDomain` class inherits from `sphinx.domains.Domain` (defined in `sphinx/domains/__init__.py`), which on initialization deep-copies `initial_data` into `env.domaindata[domain_name]` and adds a 'version' key set to `data_version`. The `RecipeDirective` inherits from `sphinx.directives.ObjectDescription` (defined in `sphinx/directives/__init__.py`), which itself extends `SphinxDirective` (from `sphinx/util/docutils.py`). `ObjectDescription` is a generic class (`ObjectDescription[ObjDescT]`) that provides the full `run()` method implementation including signature parsing, `add_target_and_index()` calls, `DocFieldTransformer` processing, and emitting the `object-description-transform` event. The `XRefRole` used for the 'ref' role inherits from `ReferenceRole` (in `sphinx/util/docutils.py`), which inherits from `SphinxRole`. `ReferenceRole` parses the 'title <target>' syntax using the regex `r'^(.+?)\\s*(?<!\\x00)<(.*?)>$'` and supports disabled references (prefix with '!'). The `Index` base class (in `sphinx/domains/_index.py`) is abstract (ABC) and raises `SphinxError` if the subclass has no valid `name` or `localname`. The `Domain.directive()` method creates a `DirectiveAdapter` inner class that sets `self.name` to the full domain-qualified name (e.g., 'recipe:recipe') before calling the parent's `run()`. The `make_refnode` function (in `sphinx/util/nodes.py`) creates a `nodes.reference` node and uses `builder.get_relative_uri()` to compute the relative URI between documents, setting `refid` instead of `refuri` when source and target documents are the same. Domain registration goes through `sphinx.registry.SphinxRegistry.add_domain()`, which stores the domain class in `self.domains[domain.name]`, and `create_domains()` instantiates each domain and transplants extensions' extra directives, roles, and indices onto the domain instance.",
    "rubric": [
      "RecipeDomain inherits from Domain (in sphinx/domains/__init__.py), which on __init__ deep-copies initial_data into env.domaindata[domain_name] and adds a 'version' key set to data_version",
      "RecipeDirective inherits from ObjectDescription (in sphinx/directives/__init__.py), which is generic (ObjectDescription[ObjDescT]) and extends SphinxDirective",
      "ObjectDescription provides the run() method that handles signature parsing, calls add_target_and_index(), runs DocFieldTransformer, and emits the object-description-transform event",
      "XRefRole (used for the 'ref' role) inherits from ReferenceRole (in sphinx/util/docutils.py), which inherits from SphinxRole",
      "ReferenceRole parses 'title <target>' syntax using the regex r'^(.+?)\\s*(?<!\\x00)<(.*?)>$' and supports disabling references with a '!' prefix",
      "Index base class (in sphinx/domains/_index.py) is abstract (ABC) and raises SphinxError if the subclass lacks a valid name or localname",
      "Domain.directive() creates a DirectiveAdapter inner class that sets self.name to the full domain-qualified name (e.g., 'recipe:recipe') before calling super().run()",
      "make_refnode (in sphinx/util/nodes.py) creates a nodes.reference node, uses builder.get_relative_uri() for cross-document links, and sets refid instead of refuri when source and target documents are the same",
      "Domain registration goes through SphinxRegistry.add_domain() which stores the domain class in self.domains[domain.name]",
      "create_domains() instantiates each domain and transplants extension-added directives, roles, and indices onto the domain instance"
    ],
    "key_files": [
      "doc/development/tutorials/examples/recipe.py",
      "sphinx/domains/__init__.py",
      "sphinx/domains/_index.py",
      "sphinx/directives/__init__.py",
      "sphinx/roles.py",
      "sphinx/util/docutils.py",
      "sphinx/util/nodes.py",
      "sphinx/registry.py"
    ],
    "source_doc": "[doc_file: doc/development/tutorials/adding_domain.rst] Further reading\nFor more information, refer to the `docutils`_ documentation and\n:doc:`/extdev/index`.\n\nIf you wish to share your extension across multiple projects or with others,\ncheck out the :ref:`third-party-extensions` section.\n\n.. _docutils: https://docutils.sourceforge.io/docs/",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_46",
    "repo": "sphinx",
    "question": "Why does the IntEnumDocumenter in the tutorial example set its priority higher than ClassDocumenter's?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `IntEnumDocumenter` in the tutorial example sets `priority = 10 + ClassDocumenter.priority`, which evaluates to `priority = 25` (since `ClassDocumenter.priority = 15`). This higher priority ensures correct documenter selection: when autodoc needs to document a member, it iterates over ALL registered documenters calling `can_document_member()` on each, collects those that return True, then sorts them by priority and picks the one with the **highest** priority (using `classes.sort(key=lambda cls: cls.priority)` followed by `classes[-1]`). \n\nThe priority must be higher than `ClassDocumenter`'s because an `IntEnum` subclass is also a regular class \u2014 both `ClassDocumenter.can_document_member()` and `IntEnumDocumenter.can_document_member()` would return True for it. `ClassDocumenter.can_document_member()` simply checks `isinstance(member, type)`, which any IntEnum subclass satisfies. Without the higher priority, the generic `ClassDocumenter` could be selected instead of the specialized `IntEnumDocumenter`, meaning the enum member listing would never be generated.\n\nThis pattern is used throughout autodoc: `ExceptionDocumenter` sets `priority = ClassDocumenter.priority + 5` (= 20); `AttributeDocumenter` sets `priority = 10`; and `MethodDocumenter` sets `priority = 1`, while `FunctionDocumenter` inherits `priority = 0` from the base `Documenter` class. The sorting-based selection means each documenter declares its priority, and the highest-priority one wins when multiple documenters can handle the same member.",
    "rubric": [
      "Explains that autodoc collects all documenters whose can_document_member() returns True for a given member, sorts them by priority, and selects the highest-priority one (classes[-1] after sorting)",
      "Notes that ClassDocumenter.can_document_member() checks isinstance(member, type), which any IntEnum subclass satisfies, so both ClassDocumenter and IntEnumDocumenter would match",
      "States that without the higher priority, the generic ClassDocumenter could be selected instead of the specialized IntEnumDocumenter, bypassing the enum member listing logic",
      "Mentions that the priority value is 10 + ClassDocumenter.priority, evaluating to 25 (since ClassDocumenter.priority = 15)",
      "Identifies that this is a general pattern in autodoc (e.g., ExceptionDocumenter also sets priority = ClassDocumenter.priority + 5 = 20 for similar reasons)"
    ],
    "key_files": [
      "doc/development/tutorials/examples/autodoc_intenum.py",
      "sphinx/ext/autodoc/_legacy_class_based/_documenters.py",
      "sphinx/application.py",
      "sphinx/ext/autodoc/directive.py"
    ],
    "source_doc": "[doc_file: doc/development/tutorials/autodoc_ext.rst] Writing the extension\nStart with ``setup`` function for the extension.\n\n.. literalinclude:: examples/autodoc_intenum.py\n   :language: python\n   :linenos:\n   :pyobject: setup\n\n\nThe :meth:`~sphinx.application.Sphinx.setup_extension` method will pull the\nautodoc extension because our new extension depends on autodoc.\n:meth:`~sphinx.application.Sphinx.add_autodocumenter` is the method that\nregisters our new auto documenter class.\n\nWe want to import certain objects from the autodoc extension:\n\n.. literalinclude:: examples/autodoc_intenum.py\n   :language: python\n   :linenos:\n   :lines: 1-7\n\n\nThere are several different documenter classes such as ``MethodDocumenter``\nor ``AttributeDocumenter`` available in the autodoc extension but\nour new class is the subclass of ``ClassDocumenter`` which a\ndocumenter class used by autodoc to document classes.\n\nThis is the definition of our new the auto-documenter class:\n\n.. literalinclude:: examples/autodoc_intenum.py\n   :language: python\n   :linenos:\n   :pyobject: IntEnumDocumenter\n\n\nImportant attributes of the new class:\n\n**objtype**\n    This attribute determines the ``auto`` directive name. In\n    this case the auto directive will be ``autointenum``.\n\n**directivetype**\n    This attribute sets the generated directive name. In\n    this example the generated directive will be ``.. :py:class::``.\n\n**priority**\n    the larger the number the higher is the priority. We want our\n    documenter be higher priority than the parent.\n\n**option_spec**\n    option specifications. We copy the parent class options and\n    add a new option *hex*.\n\n\nOverridden members:\n\n**can_document_member**\n    This member is important to override. It should\n    return *True* when the passed object can be documented by this class.\n\n**add_directive_header**\n    This method generates the directive header. We add\n    **:final:** directive option. Remember to call **super** or no directive\n    will be generated.\n\n**add_content**\n    This method generates the body of the class documentation.\n    After calling the super method we generate lines for enum description.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states 'the larger the number the higher is the priority. We want our documenter be higher priority than the parent.' The answer's claim that priority = 10 + ClassDocumenter.priority = 25 (with ClassDocumenter.priority = 15) is plausible but not explicitly stated in the documentation - it's an extrapolation from the source code.",
      "The detailed explanation of the documenter selection algorithm (line numbers in _documenters.py, sorting mechanism, classes[-1] selection) is not present in the documentation and comes from source code inspection - plausible but not verifiable from the doc alone.",
      "The claims about ExceptionDocumenter, AttributeDocumenter, MethodDocumenter, and FunctionDocumenter priorities are not mentioned in the documentation - they are extrapolations from source code.",
      "The documentation does mention that 'the larger the number the higher is the priority' and that the new documenter should have higher priority than the parent, which is consistent with the answer's general explanation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Minor doc leakage from comment text references and design rationale characterizations; core algorithmic claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_47",
    "repo": "sphinx",
    "question": "Where are the todo extensions implemented, and how do their architectures differ between the tutorial example and the production version?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The todo extension (at `doc/development/tutorials/examples/todo.py`) and the production todo extension (at `sphinx/ext/todo.py`) are located in different places and have significantly different architectures:\n\n1. **Tutorial example location**: `doc/development/tutorials/examples/todo.py` - This implementation directly uses `env-purge-doc` and `env-merge-info` event handlers and stores todos as a list on `env.todo_all_todos`.\n\n2. **Production extension location**: `sphinx/ext/todo.py` - This extension uses a `TodoDomain` (subclass of `Domain` from `sphinx/domains/__init__.py`) to manage todos, storing them per-docname in `self.data['todos']` (a dict keyed by docname). It handles purging via `TodoDomain.clear_doc()` and merging via `TodoDomain.merge_domaindata()` instead of connecting to `env-purge-doc`/`env-merge-info` events.\n\n3. **Production extension uses `TodoListProcessor` class** connected to the `doctree-resolved` event (`app.connect('doctree-resolved', TodoListProcessor)`), which processes todo nodes by resolving cross-references via `addnodes.pending_xref` nodes and wrapping them in a `new_document('')` to call `env.resolve_references()`. The tutorial simply removes nodes or builds backlinks without resolving internal references.\n\n4. **The production extension registers a custom `todo-defined` event** (`app.add_event('todo-defined')`) that is emitted in `TodoDomain.process_doc()` every time a todo node is encountered. The domain's `_process_doc` is called from `sphinx/domains/_domains_container.py` which iterates over all domain instances.\n\n5. **The production `visit_todo_node` raises `nodes.SkipNode`** when `todo_include_todos` is False, while the tutorial version always renders the node. The production LaTeX visitor uses `\\begin{sphinxtodo}` / `\\end{sphinxtodo}` environment.\n\n6. **The production extension's `env_version` is 2** and does not include `'parallel_write_safe': True` in its return metadata, whereas the tutorial example has `env_version: 1` and includes both `parallel_read_safe` and `parallel_write_safe` as True.",
    "rubric": [
      "Identifies the tutorial example at doc/development/tutorials/examples/todo.py using env-purge-doc/env-merge-info event handlers and storing todos as a list on env.todo_all_todos",
      "Identifies the production extension at sphinx/ext/todo.py using a TodoDomain (Domain subclass) that stores todos per-docname in self.data['todos'] dict and handles purging/merging via clear_doc() and merge_domaindata()",
      "Notes that the production extension uses a TodoListProcessor class connected to doctree-resolved event, which resolves cross-references via pending_xref nodes and env.resolve_references()",
      "Notes that the production extension registers and emits a custom 'todo-defined' event in TodoDomain.process_doc()",
      "Notes differences in node visitors: production visit_todo_node raises SkipNode when todo_include_todos is False; tutorial always renders",
      "Notes metadata differences: production env_version is 2 without parallel_write_safe; tutorial env_version is 1 with both parallel_read_safe and parallel_write_safe True"
    ],
    "key_files": [
      "doc/development/tutorials/examples/todo.py",
      "sphinx/ext/todo.py",
      "sphinx/domains/__init__.py",
      "sphinx/domains/_domains_container.py",
      "sphinx/directives/admonitions.py"
    ],
    "source_doc": "[doc_file: doc/development/tutorials/extending_build.rst] Overview\n.. note::\n   To understand the design of this extension, refer to\n   :ref:`important-objects` and :ref:`build-phases`.\n\nWe want the extension to add the following to Sphinx:\n\n* A ``todo`` directive, containing some content that is marked with \"TODO\" and\n  only shown in the output if a new config value is set. Todo entries should not\n  be in the output by default.\n\n* A ``todolist`` directive that creates a list of all todo entries throughout\n  the documentation.\n\nFor that, we will need to add the following elements to Sphinx:\n\n* New directives, called ``todo`` and ``todolist``.\n\n* New document tree nodes to represent these directives, conventionally also\n  called ``todo`` and ``todolist``.  We wouldn't need new nodes if the new\n  directives only produced some content representable by existing nodes.\n\n* A new config value ``todo_include_todos`` (config value names should start\n  with the extension name, in order to stay unique) that controls whether todo\n  entries make it into the output.\n\n* New event handlers: one for the :event:`doctree-resolved` event, to\n  replace the todo and todolist nodes, one for :event:`env-merge-info`\n  to merge intermediate results from parallel builds, and one for\n  :event:`env-purge-doc` (the reason for that will be covered later).",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive claims about production extension internals (TodoDomain, TodoListProcessor, visit_todo_node, env_version, line numbers, etc.) that go far beyond what the provided documentation covers. These cannot be verified from the given doc chunk alone, though they are plausible code-level details.",
      "The documentation states that todo entries 'should not be in the output by default', and the answer's claim (#4 in point 6) about the tutorial example having 'env_version: 1' and parallel safety flags cannot be verified from this doc chunk.",
      "The answer's claim that the tutorial 'always renders the node' (point 5) seems to contradict the documentation which says todo entries should only be shown 'if a new config value is set' and 'should not be in the output by default' - suggesting the tutorial version also conditionally renders.",
      "The documentation explicitly mentions three event handlers: doctree-resolved, env-merge-info, and env-purge-doc. The answer correctly identifies these for the tutorial version.",
      "The documentation mentions the config value 'todo_include_todos', which aligns with the answer's references to this config value."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc-leakage in characterizations like 'simplified tutorial' and 'full-featured built-in' plus specific line number references; all architectural claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "sphinx_gen_48",
    "repo": "sphinx",
    "question": "How does the tutorial's todo extension differ from the official sphinx.ext.todo in terms of architecture, node registration, and event handling?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The tutorial's `todo` extension works through Sphinx's extension API by registering custom docutils nodes, directives, and event handlers. Here's how the implementation connects to the broader Sphinx framework:\n\n**Setup and Registration (the `setup()` function):**\nThe `setup()` function in `doc/development/tutorials/examples/todo.py` returns an `ExtensionMetadata` dict with `version='0.1'`, `env_version=1`, `parallel_read_safe=True`, and `parallel_write_safe=True`. It registers:\n\n1. **Config value**: `app.add_config_value('todo_include_todos', False, 'html')` \u2014 the third argument is `'html'`.\n\n2. **Custom nodes**: Two node classes are registered \u2014 `todo` (inheriting from `nodes.Admonition` and `nodes.Element`) and `todolist` (inheriting from `nodes.General` and `nodes.Element`). The `todo` node gets visitor functions for `html`, `latex`, and `text` builders.\n\n3. **Directives**: `TodoDirective` extends `SphinxDirective` (defined in `sphinx/util/docutils.py`), which itself extends docutils' `Directive`. `SphinxDirective` provides the `env` property (accessing `self.state.document.settings.env`) and the `config` property, plus helper methods like `parse_content_to_nodes()`, `get_source_info()`, `set_source_info()`, and `get_location()`.\n\n4. **Event handlers**: Three events are connected:\n   - `'doctree-resolved'` \u2192 `process_todo_nodes`\n   - `'env-purge-doc'` \u2192 `purge_todos` (called when documents are removed, as seen in the `read()` method of `sphinx/builders/__init__.py`)\n   - `'env-merge-info'` \u2192 `merge_todos` (called in `BuildEnvironment.merge_info_from()`)\n\n**TodoDirective.run() implementation details:**\n- Generates a unique target ID via `self.env.new_serialno('todo')`.\n- Creates a `nodes.target` with the generated ID and a `todo` admonition node.\n- Parses directive content using `self.parse_content_to_nodes()`, which internally calls `nested_parse_to_nodes()` from `sphinx/util/parsing.py`.\n- Stores todo metadata (docname, lineno, deepcopy of node, target) on `self.env.todo_all_todos` (a list attribute dynamically added to the environment).\n\n**process_todo_nodes() details:**\n- When `todo_include_todos` is False, it removes all `todo` nodes from the doctree AND replaces all `todolist` nodes with empty lists.\n- When True, it iterates through all collected todos and creates backlink references using `app.builder.get_relative_uri(fromdocname, todo_info['docname'])` plus a `#` anchor to the target's `refid`.\n\n**Comparison with the official `sphinx.ext.todo`:**\nThe official version (in `sphinx/ext/todo.py`) is significantly more sophisticated:\n- Uses a `TodoDomain` (a proper Sphinx domain) instead of storing data on `env` attributes directly.\n- The `Todo` directive class extends `SphinxAdmonition` (from `sphinx/directives/admonitions.py`), not `SphinxDirective` directly.\n- Adds a custom `'todo-defined'` event via `app.add_event()`.\n- Has additional config values: `todo_link_only` and `todo_emit_warnings`.\n- Has separate LaTeX visitor functions (`latex_visit_todo_node` wraps content in `\\begin{sphinxtodo}` / `\\end{sphinxtodo}`).\n- The HTML visitor raises `nodes.SkipNode` when `todo_include_todos` is False (rather than removing nodes in a post-processing step).\n- Uses `TodoListProcessor` class (its `__init__` is used as the event callback) which handles `NoUri` exceptions for builders like LaTeX.\n- Has `env_version=2` and uses `sphinx.__display_version__` instead of `'0.1'`.\n- The official version registers visitors for `man` and `texinfo` builders too, whereas the tutorial only registers for `html`, `latex`, and `text`.\n\nThe default priority for all event connections is 500, as seen in the `connect()` method overloads in `sphinx/application.py`.",
    "rubric": [
      "Explains that the tutorial stores todo data directly on env attributes (env.todo_all_todos) while the official version uses a TodoDomain (a proper Sphinx domain) with self.data",
      "Notes that the tutorial's TodoDirective extends SphinxDirective while the official Todo directive extends SphinxAdmonition",
      "Mentions that the official version adds a custom 'todo-defined' event via app.add_event() which the tutorial does not",
      "Notes the official version has additional config values (todo_link_only, todo_emit_warnings) beyond just todo_include_todos",
      "Explains that the official version's HTML visitor raises nodes.SkipNode when todo_include_todos is False, while the tutorial removes nodes in process_todo_nodes",
      "Notes the official version registers visitors for man and texinfo builders in addition to html, latex, and text",
      "Mentions the official version uses TodoListProcessor class as the doctree-resolved handler and handles NoUri exceptions",
      "Notes the tutorial connects three events (doctree-resolved, env-purge-doc, env-merge-info) while the official version relies on domain methods (clear_doc, merge_domaindata) instead of env-purge-doc/env-merge-info"
    ],
    "key_files": [
      "doc/development/tutorials/examples/todo.py",
      "sphinx/ext/todo.py",
      "sphinx/util/docutils.py",
      "sphinx/application.py",
      "sphinx/events.py",
      "sphinx/builders/__init__.py",
      "sphinx/environment/__init__.py"
    ],
    "source_doc": "[doc_file: doc/development/tutorials/extending_build.rst] Using the extension\nAs before, we need to enable the extension by declaring it in our\n:file:`conf.py` file. There are two steps necessary here:\n\n#. Add the :file:`_ext` directory to the `Python path`_ using\n   ``sys.path.append``. This should be placed at the top of the file.\n\n#. Update or create the :confval:`extensions` list and add the extension file\n   name to the list\n\nIn addition, we may wish to set the ``todo_include_todos`` config value. As\nnoted above, this defaults to ``False`` but we can set it explicitly.\n\nFor example:\n\n.. code-block:: python\n\n   import sys\n   from pathlib import Path\n\n   sys.path.append(str(Path('_ext').resolve()))\n\n   extensions = ['todo']\n\n   todo_include_todos = False\n\nYou can now use the extension throughout your project. For example:\n\n.. code-block:: rst\n   :caption: index.rst\n\n   Hello, world\n   ============\n\n   .. toctree::\n      somefile.rst\n      someotherfile.rst\n\n   Hello world. Below is the list of TODOs.\n\n   .. todolist::\n\n.. code-block:: rst\n   :caption: somefile.rst\n\n   foo\n   ===\n\n   Some intro text here...\n\n   .. todo:: Fix this\n\n.. code-block:: rst\n   :caption: someotherfile.rst\n\n   bar\n   ===\n\n   Some more text here...\n\n   .. todo:: Fix that\n\nBecause we have configured ``todo_include_todos`` to ``False``, we won't\nactually see anything rendered for the ``todo`` and ``todolist`` directives.\nHowever, if we toggle this to true, we will see the output described\npreviously.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed line-number references and documentation-only semantic descriptions of parameters.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_01",
    "repo": "xarray",
    "question": "What is the role and architecture of `_normalize_path` in xarray's backend system, and where is it used across the codebase?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "`_normalize_path` is a utility function in `xarray/backends/common.py` that normalizes file paths throughout xarray's backend I/O system. Here is its architecture and role:\n\n**Implementation Details:**\n- The function is defined in `xarray/backends/common.py` with three `@overload` signatures using `typing.overload`: one for `os.PathLike` \u2192 `str`, one for `str` \u2192 `str`, and a generic `T` \u2192 `T` pass-through.\n- The actual implementation accepts `os.PathLike | str | T` and returns `str | T`.\n- It performs two transformations: (1) converts `os.PathLike` objects to strings via `os.fspath()`, and (2) for string paths that are NOT remote URIs, it applies `os.path.abspath(os.path.expanduser(path))` to resolve user home directory references (`~`) and make the path absolute.\n- Remote URI detection is delegated to `is_remote_uri()` from `xarray/core/utils.py`, which uses a regex `r\"^[a-zA-Z][a-zA-Z0-9]*(\\://|\\:\\:)\"`.\n- If the input is neither a string nor a PathLike (e.g., a `ReadBuffer` or an `AbstractDataStore`), the function returns it unchanged via the generic `T` pass-through.\n\n**System-wide Usage:**\n`_normalize_path` is called across virtually every backend engine entrypoint and writer:\n- **NetCDF4 backend** (`netCDF4_.py`): imported and called in `NetCDF4BackendEntrypoint.open_dataset()` before opening a `NetCDF4DataStore`.\n- **h5netcdf backend** (`h5netcdf_.py`): used inside a `_normalize_filename_or_obj` wrapper that also handles `bytes`/`memoryview` inputs by wrapping them in `io.BytesIO`.\n- **Zarr backend** (`zarr.py`): used in `_get_mappers()` to normalize both `store` and `chunk_store` arguments.\n- **API layer** (`api.py`): used to set `ds.encoding[\"source\"]` and `tree.encoding[\"source\"]` in `_dataset_from_backend_dataset` and `_datatree_from_backend_datatree`.\n- **Writers** (`writers.py`): used in `to_netcdf` implementation via `normalized_path = _normalize_path(path_or_file)`.\n- **`_find_absolute_paths`** in `common.py`: uses `_normalize_path` before glob expansion, and also uses it in the nested `_normalize_path_list` helper for recursive normalization of nested path sequences.\n\n**Type System Integration:**\nThe function operates on the `T_PathFileOrDataStore` type alias, which is defined as `str | os.PathLike[Any] | ReadBuffer | bytes | memoryview | AbstractDataStore`. The generic `T` overload ensures that non-path types like `ReadBuffer` or `AbstractDataStore` pass through unchanged.\n\n**Key Behavior:** Remote URIs are explicitly left unmodified \u2014 `os.path.abspath` and `os.path.expanduser` are only applied to local file paths, as determined by the `not is_remote_uri(path)` guard.",
    "rubric": [
      "Explains that _normalize_path is defined in xarray/backends/common.py with three @overload signatures (PathLike\u2192str, str\u2192str, T\u2192T) and a single implementation accepting os.PathLike | str | T",
      "Describes the two transformations: os.fspath() for PathLike objects, and os.path.abspath(os.path.expanduser(path)) for local string paths",
      "Explains that remote URIs are detected via is_remote_uri() (regex-based in xarray/core/utils.py) and left unmodified",
      "Mentions the generic T pass-through behavior for non-path/non-string inputs like ReadBuffer or AbstractDataStore",
      "Identifies usage in NetCDF4 backend (netCDF4_.py) open_dataset entrypoint",
      "Identifies usage in h5netcdf backend, possibly mentioning its wrapper that also handles bytes/memoryview",
      "Identifies usage in Zarr backend (_get_mappers or similar) for normalizing store and chunk_store",
      "Identifies usage in the API layer (api.py) for setting encoding['source']",
      "Identifies usage in writers.py for normalizing path_or_file",
      "Identifies usage in _find_absolute_paths and its nested _normalize_path_list helper for recursive normalization"
    ],
    "key_files": [
      "xarray/backends/common.py",
      "xarray/core/utils.py",
      "xarray/backends/api.py",
      "xarray/backends/zarr.py",
      "xarray/backends/netCDF4_.py",
      "xarray/backends/h5netcdf_.py",
      "xarray/backends/writers.py"
    ],
    "source_doc": "[docstring: xarray/backends/common.py] xarray.backends.common._normalize_path\nNormalize pathlikes to string.\n\n    Parameters\n    ----------\n    path :\n        Path to file.\n\n    Examples\n    --------\n    >>> from pathlib import Path\n\n    >>> directory = Path(xr.backends.common.__file__).parent\n    >>> paths_path = Path(directory).joinpath(\"comm*n.py\")\n    >>> paths_str = xr.backends.common._normalize_path(paths_path)\n    >>> print([type(p) for p in (paths_str,)])\n    [<class 'str'>]",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only shows the function's docstring and a basic usage example demonstrating PathLike-to-string conversion. The generated answer contains extensive implementation details (line numbers, overload signatures, remote URI handling, system-wide usage across multiple files) that cannot be verified from the provided documentation alone.",
      "The claim about three @overload signatures, os.fspath(), os.path.abspath, os.path.expanduser, remote URI detection via regex, and all the specific file/line references are plausible code-level details but entirely unsupported by the provided documentation chunk.",
      "The basic claim that _normalize_path converts PathLike objects to strings is consistent with the documentation's docstring ('Normalize pathlikes to string') and the example showing the output type is <class 'str'>."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor removals for unverifiable backend files (scipy, pydap) and slight rewrites for design rationale language.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_02",
    "repo": "xarray",
    "question": "Why does `_find_absolute_paths` only allow wildcard matching on remote URIs when engine is 'zarr'?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `_find_absolute_paths` function has multiple branching strategies to handle different path types: wildcard/glob expansion for local paths, remote URI handling via fsspec for zarr engines, explicit sequences of paths (including nested lists), and in-memory file-like objects (ReadBuffer). The key design choices visible in the code are:\n\n1. **Remote URIs are only supported with wildcards when engine='zarr'**: For remote paths (detected by `is_remote_uri` which matches `protocol://` or `protocol::` patterns via regex `r'^[a-zA-Z][a-zA-Z0-9]*(\\://|\\:\\:)'`), glob-style wildcard matching is only permitted when `engine='zarr'` because the function returns `fs.get_mapper(path)` objects (fsspec mappers) rather than plain strings. For non-zarr remote URIs with wildcards, it raises a ValueError telling users to supply explicit path lists instead.\n\n2. **Local paths are sorted after glob expansion**: The `sorted(glob(_normalize_path(paths)))` call ensures deterministic file ordering regardless of filesystem enumeration order.\n\n3. **`_normalize_path` applies `os.path.abspath(os.path.expanduser(path))` only for non-remote strings**: The condition `if isinstance(path, str) and not is_remote_uri(path)` guards the normalization, so remote URIs are left untouched. `PathLike` objects are first converted to strings via `os.fspath()` before the absolute path normalization.\n\n4. **Nested list support via `_normalize_path_list`**: When paths is a sequence (not a string, PathLike, or ReadBuffer), a recursive helper `_normalize_path_list` is used to handle arbitrarily nested list structures. This supports multi-dimensional path specifications (note that `_infer_concat_order_from_positions` is imported in the api module).\n\n5. **fsspec is lazily imported via `attempt_import`**: fsspec is only imported when remote URIs with zarr are actually encountered, using `attempt_import('fsspec')` inside the conditional branch.\n\n6. **Storage options are extracted from `backend_kwargs`**: For remote zarr paths, storage options are read from `kwargs.get('backend_kwargs', {}).get('storage_options', {})`, which are passed to `fsspec.core.get_fs_token_paths`.",
    "rubric": [
      "Explains that remote URI wildcard matching uses fsspec's glob and returns fs.get_mapper() objects (fsspec mappers) rather than plain string paths",
      "Notes that for non-zarr engines with remote URIs, a ValueError is raised directing users to supply explicit path lists instead",
      "Mentions that fsspec is lazily imported via attempt_import only when this zarr+remote branch is triggered",
      "Mentions that storage_options for fsspec are extracted from backend_kwargs in kwargs",
      "Explains that local paths use sorted(glob(...)) after normalization for deterministic ordering",
      "Describes how _normalize_path handles the distinction between remote URIs (left untouched) and local paths (expanded with abspath/expanduser)"
    ],
    "key_files": [
      "xarray/backends/common.py",
      "xarray/backends/api.py",
      "xarray/core/utils.py",
      "xarray/core/types.py"
    ],
    "source_doc": "[docstring: xarray/backends/common.py] xarray.backends.common._find_absolute_paths\nFind absolute paths from the pattern.\n\n    Parameters\n    ----------\n    paths :\n        Path(s) to file(s). Can include wildcards like * .\n    **kwargs :\n        Extra kwargs. Mainly for fsspec.\n\n    Examples\n    --------\n    >>> from pathlib import Path\n\n    >>> directory = Path(xr.backends.common.__file__).parent\n    >>> paths = str(Path(directory).joinpath(\"comm*n.py\"))  # Find common with wildcard\n    >>> paths = xr.backends.common._find_absolute_paths(paths)\n    >>> [Path(p).name for p in paths]\n    ['common.py']",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a brief docstring describing the function's purpose (finding absolute paths from patterns with wildcard support) and a simple example. The generated answer contains extensive implementation details (remote URI handling, fsspec mappers, zarr engine restrictions, nested list support, lazy imports, storage options extraction) that are not present in or supported by the provided documentation chunk.",
      "While these implementation details may be plausible based on actual source code, they cannot be verified from the documentation provided and constitute extrapolation beyond the doc scope.",
      "The core claim that the function finds absolute paths from patterns with wildcard support is supported by the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code structure; a few design rationale explanations and documentation references needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_04",
    "repo": "xarray",
    "question": "How does `xr.cov()` compute weighted covariance, including alignment, NaN handling, demeaning, and the degrees-of-freedom adjustment?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `xr.cov()` function computes covariance between two DataArray objects by delegating to a shared internal function `_cov_corr()` (which is also used by `xr.corr()`). The algorithm works in 4 steps:\n\n1. **Alignment**: The two arrays are aligned using `align(da_a, da_b, join='inner', copy=False)`.\n\n2. **NaN masking**: A boolean mask `valid_values = da_a.notnull() & da_b.notnull()` is computed, and both arrays are masked via `.where(valid_values)` so that if either has NaN at a position, both are set to NaN there.\n\n3. **Demeaning**: Each array is demeaned along the specified dimension. If weights are provided, `da.weighted(weights).mean(dim=dim)` is used (which internally uses `xr.dot()` via `_reduce()`). Without weights, plain `.mean(dim=dim)` is used.\n\n4. **Covariance computation**: The product `demeaned_da_a.conj() * demeaned_da_b` is computed (using complex conjugate of the first array). Then `.mean(dim=dim, skipna=True)` is applied (weighted or unweighted).\n\nFor the 'cov' method specifically, a degrees-of-freedom adjustment is applied: `adjust = valid_count / (valid_count - ddof)` where `valid_count = valid_values.sum(dim)`. Note that `valid_count` uses the unweighted count of valid pairs even when weights are provided. The final result is `cov * adjust`.\n\nKey design decisions:\n- The default `ddof=0` in `_cov_corr()` differs from `cov()`'s default `ddof=1` \u2014 `cov()` explicitly passes its user-specified ddof, while `corr()` doesn't pass ddof (so uses 0), but the ddof adjustment code path is only executed when `method='cov'`.\n- The `Weighted._reduce()` method uses `xr.dot()` (via `dot(da, weights, dim=dim)`) rather than `(da * weights).sum()`.\n- The `Weighted` class validates that weights contain no missing values at initialization, raising ValueError if NaN is found.\n- The `Weighted._weighted_mean` divides `weighted_sum` by `sum_of_weights`, where `sum_of_weights` accounts for NaN positions by multiplying a `notnull()` mask with the weights before summing.\n- `DataArrayWeighted._implementation()` converts the DataArray to a temporary Dataset, maps the function, then converts back.",
    "rubric": [
      "Explains that `cov()` delegates to `_cov_corr()`, which is shared with `corr()`",
      "Describes alignment step: `align(da_a, da_b, join='inner', copy=False)`",
      "Describes NaN masking: a `valid_values = da_a.notnull() & da_b.notnull()` mask is computed and both arrays are masked with `.where(valid_values)`",
      "Describes demeaning step: each array is demeaned using weighted mean (`da.weighted(weights).mean(dim)`) when weights are provided, or plain `.mean(dim)` otherwise",
      "Describes covariance computation: `(demeaned_da_a.conj() * demeaned_da_b).weighted(weights).mean(dim, skipna=True)` (or unweighted `.mean` without weights)",
      "Explains ddof adjustment: `valid_count / (valid_count - ddof)` is multiplied into the result, where `valid_count = valid_values.sum(dim)` uses unweighted count even with weights",
      "Notes that `cov()` defaults to `ddof=1` while `_cov_corr()` defaults to `ddof=0`, and the ddof adjustment only applies when `method='cov'`",
      "Explains that `Weighted._reduce()` uses `dot(da, weights, dim=dim)` rather than `(da * weights).sum()`",
      "Mentions that the `Weighted` constructor validates weights contain no missing values (raises ValueError if NaN found)",
      "Explains that `_sum_of_weights` accounts for NaN positions by using a `notnull()` mask dotted with weights"
    ],
    "key_files": [
      "xarray/computation/computation.py",
      "xarray/computation/weighted.py",
      "xarray/__init__.py"
    ],
    "source_doc": "[docstring: xarray/computation/computation.py] xarray.computation.computation.cov\nCompute covariance between two DataArray objects along a shared dimension.\n\n    Parameters\n    ----------\n    da_a : DataArray\n        Array to compute.\n    da_b : DataArray\n        Array to compute.\n    dim : str, iterable of hashable, \"...\" or None, optional\n        The dimension along which the covariance will be computed\n    ddof : int, default: 1\n        If ddof=1, covariance is normalized by N-1, giving an unbiased estimate,\n        else normalization is by N.\n    weights : DataArray, optional\n        Array of weights.\n\n    Returns\n    -------\n    covariance : DataArray\n\n    See Also\n    --------\n    pandas.Series.cov : corresponding pandas function\n    xarray.corr : respective function to calculate correlation\n\n    Examples\n    --------\n    >>> from xarray import DataArray\n    >>> da_a = DataArray(\n    ...     np.array([[1, 2, 3], [0.1, 0.2, 0.3], [3.2, 0.6, 1.8]]),\n    ...     dims=(\"space\", \"time\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...         (\"time\", pd.date_range(\"2000-01-01\", freq=\"1D\", periods=3)),\n    ...     ],\n    ... )\n    >>> da_a\n    <xarray.DataArray (space: 3, time: 3)> Size: 72B\n    array([[1. , 2. , 3. ],\n           [0.1, 0.2, 0.3],\n           [3.2, 0.6, 1.8]])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03\n    >>> da_b = DataArray(\n    ...     np.array([[0.2, 0.4, 0.6], [15, 10, 5], [3.2, 0.6, 1.8]]),\n    ...     dims=(\"space\", \"time\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...         (\"time\", pd.date_range(\"2000-01-01\", freq=\"1D\", periods=3)),\n    ...     ],\n    ... )\n    >>> da_b\n    <xarray.DataArray (space: 3, time: 3)> Size: 72B\n    array([[ 0.2,  0.4,  0.6],\n           [15. , 10. ,  5. ],\n           [ 3.2,  0.6,  1.8]])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03\n    >>> xr.cov(da_a, da_b)\n    <xarray.DataArray ()> Size: 8B\n    array(-3.53055556)\n    >>> xr.cov(da_a, da_b, dim=\"time\")\n    <xarray.DataArray (space: 3)> Size: 24B\n    array([ 0.2       , -0.5       ,  1.69333333])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n    >>> weights = DataArray(\n    ...     [4, 2, 1],\n    ...     dims=(\"space\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...     ],\n    ... )\n    >>> weights\n    <xarray.DataArray (space: 3)> Size: 24B\n    array([4, 2, 1])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n    >>> xr.cov(da_a, da_b, dim=\"space\", weights=weights)\n    <xarray.DataArray (time: 3)> Size: 24B\n    array([-4.69346939, -4.49632653, -3.37959184])\n    Coordinates:\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims the default ddof in _cov_corr() is 0, while cov()'s default is 1. The documentation confirms cov()'s default ddof=1, but the internal _cov_corr() default of 0 is not documented and cannot be verified from the provided documentation alone.",
      "The answer describes detailed internal implementation (align with inner join, NaN masking, complex conjugate, xr.dot in Weighted, Weighted validation of no missing values, DataArrayWeighted._implementation converting to Dataset) \u2014 none of these are mentioned in the documentation. These are plausible code-level details but cannot be verified from the doc.",
      "The answer states that valid_count is 'the unweighted count of valid pairs, even when weights are provided' \u2014 this is not documented and is a specific implementation claim that cannot be verified.",
      "The answer says skipna=True is 'explicitly noted as required to handle auto-covariance correctly when NaN values are present' \u2014 this is not mentioned in the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in design rationale phrases like 'to avoid broadcasting' and 'to support complex-valued data'.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_03",
    "repo": "xarray",
    "question": "Where does data flow through when `save_mfdataset` writes multiple datasets, and how does the `multifile=True` flag alter the normal `to_netcdf` path?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `save_mfdataset` function in `xarray/backends/writers.py` flows data through the following call chain:\n\n1. **Validation phase**: It first checks that mode='w' doesn't have duplicate paths (using `len(set(paths)) < len(paths)`), validates all objects are `Dataset` instances, and checks that datasets/paths/groups lists have equal lengths.\n\n2. **Store creation phase**: It calls `to_netcdf()` (also in `xarray/backends/writers.py`) for each dataset with `multifile=True`. This flag causes `to_netcdf` to return as a `(writer, store)` tuple *before* calling `writer.sync()`. Inside `to_netcdf`, the call chain is:\n   - `_normalize_path` (imported from `xarray/backends/api.py`) converts path-like objects to absolute string paths\n   - `get_default_netcdf_write_engine` determines the engine, consulting `OPTIONS[\"netcdf_engine_order\"]`\n   - `_validate_dataset_names` and `_validate_attrs` perform validation\n   - `_get_netcdf_autoclose` is called to determine autoclose behavior\n   - `get_writable_netcdf_store` uses the `WRITEABLE_STORES` dict mapping engine names to store openers: `{\"netcdf4\": backends.NetCDF4DataStore.open, \"scipy\": backends.ScipyDataStore, \"h5netcdf\": backends.H5NetCDFStore.open}`\n   - `dump_to_store` encodes coordinates via `conventions.encode_dataset_coordinates` and calls `store.store()`\n\n3. **Sync phase**: After collecting all `(writer, store)` pairs, `save_mfdataset` calls `writer.sync(compute=compute)` for each writer. The `ArrayWriter.sync()` method (in `xarray/backends/common.py`) uses `get_chunked_array_type` to detect the chunk manager and calls `chunkmanager.store()` with `lock`, `compute`, `flush=True`, and `regions` parameters. After sync, it clears `self.sources`, `self.targets`, and `self.regions`.\n\n4. **Cleanup phase**: In a `finally` block, if `compute=True`, stores are closed; otherwise `store.sync()` is called. If `compute=False`, it imports dask and returns `dask.delayed(list(starmap(delayed_close_after_writes, zip(writes, stores))))`. The `delayed_close_after_writes` function (in `xarray/backends/api.py`) wraps `_finalize_store` which deletes the writes reference then calls `store.close()`.\n\n5. **Export**: `save_mfdataset` is accessible as `xr.save_mfdataset`.",
    "rubric": [
      "Mentions that save_mfdataset validates mode='w' for duplicate paths, checks all objects are Dataset instances, and checks equal lengths of datasets/paths/groups",
      "Explains that save_mfdataset calls to_netcdf for each dataset with multifile=True, which causes to_netcdf to return a (writer, store) tuple before calling writer.sync()",
      "Describes the to_netcdf internal call chain: _normalize_path, get_default_netcdf_write_engine, _validate_dataset_names/_validate_attrs, get_writable_netcdf_store (using WRITEABLE_STORES dict), and dump_to_store",
      "Explains that dump_to_store encodes coordinates via conventions.encode_dataset_coordinates and calls store.store()",
      "Describes that after collecting all (writer, store) pairs, save_mfdataset calls writer.sync(compute=compute) for each writer, where ArrayWriter.sync uses chunkmanager.store()",
      "Describes the cleanup phase: in a finally block, stores are closed if compute=True, otherwise store.sync() is called",
      "Explains that when compute=False, save_mfdataset returns a dask.delayed wrapping delayed_close_after_writes, which calls _finalize_store to delete writes and close the store"
    ],
    "key_files": [
      "xarray/backends/writers.py",
      "xarray/backends/common.py",
      "xarray/backends/api.py",
      "xarray/backends/locks.py"
    ],
    "source_doc": "[docstring: xarray/backends/writers.py] xarray.backends.writers.save_mfdataset\nWrite multiple datasets to disk as netCDF files simultaneously.\n\n    This function is intended for use with datasets consisting of dask.array\n    objects, in which case it can write the multiple datasets to disk\n    simultaneously using a shared thread pool.\n\n    When not using dask, it is no different than calling ``to_netcdf``\n    repeatedly.\n\n    Parameters\n    ----------\n    datasets : list of Dataset\n        List of datasets to save.\n    paths : list of str or list of path-like objects\n        List of paths to which to save each corresponding dataset.\n    mode : {\"w\", \"a\"}, optional\n        Write (\"w\") or append (\"a\") mode. If mode=\"w\", any existing file at\n        these locations will be overwritten.\n    format : {\"NETCDF4\", \"NETCDF4_CLASSIC\", \"NETCDF3_64BIT\",               \"NETCDF3_CLASSIC\"}, optional\n        File format for the resulting netCDF file:\n\n        * NETCDF4: Data is stored in an HDF5 file, using netCDF4 API\n          features.\n        * NETCDF4_CLASSIC: Data is stored in an HDF5 file, using only\n          netCDF 3 compatible API features.\n        * NETCDF3_64BIT: 64-bit offset version of the netCDF 3 file format,\n          which fully supports 2+ GB files, but is only compatible with\n          clients linked against netCDF version 3.6.0 or later.\n        * NETCDF3_CLASSIC: The classic netCDF 3 file format. It does not\n          handle 2+ GB files very well.\n\n        All formats are supported by the netCDF4-python library.\n        scipy.io.netcdf only supports the last two formats.\n\n        The default format is NETCDF4 if you are saving a file to disk and\n        have the netCDF4-python library available. Otherwise, xarray falls\n        back to using scipy to write netCDF files and defaults to the\n        NETCDF3_64BIT format (scipy does not support netCDF4).\n    groups : list of str, optional\n        Paths to the netCDF4 group in each corresponding file to which to save\n        datasets (only works for format=\"NETCDF4\"). The groups will be created\n        if necessary.\n    engine : {\"netcdf4\", \"h5netcdf\", \"scipy\"}, optional\n        Engine to use when writing netCDF files. If not provided, the\n        default engine is chosen based on available dependencies, by default\n        preferring \"netcdf4\" over \"h5netcdf\" over \"scipy\" (customizable via\n        ``netcdf_engine_order`` in ``xarray.set_options()``).\n    compute : bool\n        If true compute immediately, otherwise return a\n        ``dask.delayed.Delayed`` object that can be computed later.\n    **kwargs : dict, optional\n        Additional arguments are passed along to ``to_netcdf``.\n\n    Examples\n    --------\n    Save a dataset into one netCDF per year of data:\n\n    >>> ds = xr.Dataset(\n    ...     {\"a\": (\"time\", np.linspace(0, 1, 48))},\n    ...     coords={\"time\": pd.date_range(\"2010-01-01\", freq=\"ME\", periods=48)},\n    ... )\n    >>> ds\n    <xarray.Dataset> Size: 768B\n    Dimensions:  (time: 48)\n    Coordinates:\n      * time     (time) datetime64[us] 384B 2010-01-31 2010-02-28 ... 2013-12-31\n    Data variables:\n        a        (time) float64 384B 0.0 0.02128 0.04255 ... 0.9574 0.9787 1.0\n    >>> years, datasets = zip(*ds.groupby(\"time.year\"))\n    >>> paths = [f\"{y}.nc\" for y in years]\n    >>> xr.save_mfdataset(datasets, paths)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, internal function names, specific class names, dict mappings) that are not present in the provided documentation. These cannot be verified from the doc alone, but they are plausible code-level details.",
      "The answer claims the engine preference order defaults to ('netcdf4', 'h5netcdf', 'scipy'), which aligns with the documentation stating 'preferring \"netcdf4\" over \"h5netcdf\" over \"scipy\"'.",
      "The answer states mode='w' doesn't allow duplicate paths - the documentation only says mode='w' will overwrite existing files, but doesn't mention duplicate path validation. This is a plausible implementation detail but unverifiable from the doc.",
      "The answer's description of the compute parameter behavior (returning dask.delayed when compute=False) is consistent with the documentation's description of the compute parameter.",
      "Many specific implementation details (line numbers like 'line 375', 'line 448', 'line 409', specific internal functions like '_normalize_path', '_get_netcdf_autoclose', 'delayed_close_after_writes') cannot be verified from the documentation and could potentially be inaccurate, but they don't contradict anything stated in the docs."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in line numbers, a default value from options module, and one rationale comment; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_06",
    "repo": "xarray",
    "question": "Why does the rolling module's `_array_reduce` method use a prioritized dispatch pattern with numbagg, bottleneck, and construct-based fallbacks, and what design choices optimize the `_counts` and `_mean` implementations?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `DataArrayRolling._counts` method uses `False` as the `fill_value` and `skipna=False` when computing counts. Specifically, it calls `.construct(rolling_dim, fill_value=False, keep_attrs=keep_attrs).sum(dim=dim, skipna=False, keep_attrs=keep_attrs)`.\n\nThe rolling module's reduction strategy (`_array_reduce`) follows a prioritized dispatch pattern:\n\n1. **numbagg** is tried first (if `OPTIONS['use_numbagg']` is True, numbagg is available, the array is not dask, dtype is not in 'ObMm', ndim==1, and the namespace is numpy). For `move_std`/`move_var`, numbagg is only used if `ddof==1`.\n\n2. **bottleneck** is tried second (if `OPTIONS['use_bottleneck']` is True, bottleneck has the function, either it's not a dask array OR dask >= 2024.11.0, ndim==1, namespace is numpy).\n\n3. **rolling_agg_func** (like `_mean`) is used as a third fallback for operations like `mean` which is implemented as `sum/count`.\n\n4. The final fallback uses `self.reduce(array_agg_func, ...)` which goes through `_construct` + reduce.\n\nThe `automatic_rechunk` parameter is set to `True` for argmax, argmin, std, var, and median, while max, min, prod, and sum use the default `False`.\n\nThe `Variable.rolling_window` method handles centering by computing asymmetric padding: for centered windows, `start = win // 2` and `end = win - 1 - start` (so a window of 10 gives padding of (5, 4), and a window of 9 gives (4, 4)). For non-centered windows, padding is `(win - 1, 0)`. After padding, it calls `duck_array_ops.sliding_window_view` which dispatches to either numpy's `sliding_window_view` or dask's implementation (via `dask_array_compat.sliding_window_view`).\n\nThe `_mean` method divides `sum` by `count` and explicitly casts count to the result's dtype using `duck_array_ops.astype(self.count(keep_attrs=False), dtype=result.dtype, copy=False)`.\n\nWhen `min_periods` is `None`, the parent `Rolling.__init__` sets it to `math.prod(self.window)` \u2014 the product of all window sizes (supporting n-dimensional rolling). The `count` method then applies a threshold: `rolling_count.where(enough_periods)` where `enough_periods = rolling_count >= self.min_periods`.",
    "rubric": [
      "Explains that numbagg is tried first when available, the array is not dask, dtype is not in 'ObMm', ndim==1, namespace is numpy, and for move_std/move_var only when ddof==1",
      "Explains that bottleneck is tried second when available and the function exists, with dask support requiring dask >= 2024.11.0, ndim==1, and numpy namespace",
      "Explains that rolling_agg_func (like _mean) is the third fallback for operations that have a custom implementation",
      "Explains the final fallback uses self.reduce which goes through construct + reduce",
      "Explains that _counts uses fill_value=False (instead of NaN) for speed with boolean arrays and skipna=False to avoid copying the strided array",
      "Explains that _mean divides sum by count, casting count to the result's dtype (using duck_array_ops.astype) to handle type consistency",
      "Explains that min_periods defaults to math.prod(self.window) when None, and count applies a threshold via rolling_count.where(enough_periods)",
      "Explains that Variable.rolling_window handles centering with asymmetric padding: start = win // 2, end = win - 1 - start",
      "Mentions that automatic_rechunk=True is set for argmax, argmin, std, var, and median because those reductions make memory copies"
    ],
    "key_files": [
      "xarray/computation/rolling.py",
      "xarray/core/variable.py",
      "xarray/core/duck_array_ops.py",
      "xarray/compat/dask_array_compat.py",
      "xarray/compat/dask_array_ops.py"
    ],
    "source_doc": "[docstring: xarray/computation/rolling.py] xarray.computation.rolling.DataArrayRolling\nxarray.computation.rolling.DataArrayRolling.__init__:\n  Moving window object for DataArray.\n        You should use DataArray.rolling() method to construct this object\n        instead of the class constructor.\n\n        Parameters\n        ----------\n        obj : DataArray\n            Object to window.\n        windows : mapping of hashable to int\n            A mapping from the name of the dimension to create the rolling\n            window along (e.g. `time`) to the size of the moving window.\n        min_periods : int, default: None\n            Minimum number of observations in window required to have a value\n            (otherwise result is NA). The default, None, is equivalent to\n            setting min_periods equal to the size of the window.\n        center : bool, default: False\n            Set the labels at the center of the window. The default, False,\n            sets the labels at the right edge of the window.\n\n        Returns\n        -------\n        rolling : type of input argument\n\n        See Also\n        --------\n        xarray.DataArray.rolling\n        xarray.DataArray.groupby\n        xarray.Dataset.rolling\n        xarray.Dataset.groupby\n\nxarray.computation.rolling.DataArrayRolling.construct:\n  Convert this rolling object to xr.DataArray,\n        where the window dimension is stacked as a new dimension\n\n        Parameters\n        ----------\n        window_dim : Hashable or dict-like to Hashable, optional\n            A mapping from dimension name to the new window dimension names.\n        stride : int or mapping of int, default: 1\n            Size of stride for the rolling window.\n        fill_value : default: dtypes.NA\n            Filling value to match the dimension size.\n        keep_attrs : bool, default: None\n            If True, the attributes (``attrs``) will be copied from the original\n            object to the new one. If False, the new object will be returned\n            without attributes. If None uses the global default.\n        sliding_window_view_kwargs : Mapping\n            Keyword arguments that should be passed to the underlying array type's\n            ``sliding_window_view`` function.\n        **window_dim_kwargs : Hashable, optional\n            The keyword arguments form of ``window_dim`` {dim: new_name, ...}.\n\n        Returns\n        -------\n        DataArray\n            a view of the original array. By default, the returned array is not writeable.\n            For numpy arrays, one can pass ``writeable=True`` in ``sliding_window_view_kwargs``.\n\n        See Also\n        --------\n        numpy.lib.stride_tricks.sliding_window_view\n        dask.array.lib.stride_tricks.sliding_window_view\n\n        Notes\n        -----\n        With dask arrays, it's possible to pass the ``automatic_rechunk`` kwarg as\n        ``sliding_window_view_kwargs={\"automatic_rechunk\": True}``. This controls\n        whether dask should automatically rechunk the output to avoid\n        exploding chunk sizes. Automatically rechunking is the default behaviour.\n        Importantly, each chunk will be a view of the data so large chunk sizes are\n        only safe if *no* copies are made later.\n\n        Examples\n        --------\n        >>> da = xr.DataArray(np.arange(8).reshape(2, 4), dims=(\"a\", \"b\"))\n\n        >>> rolling = da.rolling(b=3)\n        >>> rolling.construct(\"window_dim\")\n        <xarray.DataArray (a: 2, b: 4, window_dim: 3)> Size: 192B\n        array([[[nan, nan,  0.],\n                [nan,  0.,  1.],\n                [ 0.,  1.,  2.],\n                [ 1.,  2.,  3.]],\n        <BLANKLINE>\n               [[nan, nan,  4.],\n                [nan,  4.,  5.],\n                [ 4.,  5.,  6.],\n                [ 5.,  6.,  7.]]])\n        Dimensions without coordinates: a, b, window_dim\n\n        >>> rolling = da.rolling(b=3, center=True)\n        >>> rolling.construct(\"window_dim\")\n        <xarray.DataArray (a: 2, b: 4, window_dim: 3)> Size: 192B\n        array([[[nan,  0.,  1.],\n                [ 0.,  1.,  2.],\n                [ 1.,  2.,  3.],\n                [ 2.,  3., nan]],\n        <BLANKLINE>\n               [[nan,  4.,  5.],\n                [ 4.,  5.,  6.],\n                [ 5.,  6.,  7.],\n                [ 6.,  7., nan]]])\n        Dimensions without coordinates: a, b, window_dim\n\nxarray.computation.rolling.DataArrayRolling.reduce:\n  Reduce each window by applying `func`.\n\n        Equivalent to ``.construct(...).reduce(func, ...)``.\n\n        Parameters\n        ----------\n        func : callable\n            Function which can be called in the form\n            `func(x, **kwargs)` to return the result of collapsing an\n            np.ndarray over the rolling dimension.\n        keep_attrs : bool, default: None\n            If True, the attributes (``attrs``) will be copied from the original\n            object to the new one. If False, the new object will be returned\n            without attributes. If None uses the global default.\n        sliding_window_view_kwargs\n            Keyword arguments that should be passed to the underlying array type's\n            ``sliding_window_view`` function.\n        **kwargs : dict\n            Additional keyword arguments passed on to `func`.\n\n        Returns\n        -------\n        reduced : DataArray\n            Array with summarized data.\n\n        See Also\n        --------\n        numpy.lib.stride_tricks.sliding_window_view\n        dask.array.lib.stride_tricks.sliding_window_view\n\n        Notes\n        -----\n        With dask arrays, it's possible to pass the ``automatic_rechunk`` kwarg as\n        ``sliding_window_view_kwargs={\"automatic_rechunk\": True}``. This controls\n        whether dask should automatically rechunk the output to avoid\n        exploding chunk sizes. Automatically rechunking is the default behaviour.\n        Importantly, each chunk will be a view of the data so large chunk sizes are\n        only safe if *no* copies are made later.\n\n        Examples\n        --------\n        >>> da = xr.DataArray(np.arange(8).reshape(2, 4), dims=(\"a\", \"b\"))\n        >>> rolling = da.rolling(b=3)\n        >>> rolling.construct(\"window_dim\")\n        <xarray.DataArray (a: 2, b: 4, window_dim: 3)> Size: 192B\n        array([[[nan, nan,  0.],\n                [nan,  0.,  1.],\n                [ 0.,  1.,  2.],\n                [ 1.,  2.,  3.]],\n        <BLANKLINE>\n               [[nan, nan,  4.],\n                [nan,  4.,  5.],\n                [ 4.,  5.,  6.],\n                [ 5.,  6.,  7.]]])\n        Dimensions without coordinates: a, b, window_dim\n\n        >>> rolling.reduce(np.sum)\n        <xarray.DataArray (a: 2, b: 4)> Size: 64B\n        array([[nan, nan,  3.,  6.],\n               [nan, nan, 15., 18.]])\n        Dimensions without coordinates: a, b\n\n        >>> rolling = da.rolling(b=3, min_periods=1)\n        >>> rolling.reduce(np.nansum)\n        <xarray.DataArray (a: 2, b: 4)> Size: 64B\n        array([[ 0.,  1.,  3.,  6.],\n               [ 4.,  9., 15., 18.]])\n        Dimensions without coordinates: a, b\n\nxarray.computation.rolling.DataArrayRolling._counts:\n  Number of non-nan entries in each rolling window.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific optimization strategies, dispatch patterns, numbagg/bottleneck fallback logic, automatic_rechunk behavior, Variable.rolling_window padding calculations, _mean implementation details, GH references) that are not present in the provided documentation and appear to be sourced from actual source code rather than the given doc chunk.",
      "The claim about `_counts` using 'False as fill_value instead of np.nan' and 'skipna=False' is not supported by the documentation, which only states '_counts' provides 'Number of non-nan entries in each rolling window'.",
      "The claim about min_periods defaulting to 'math.prod(self.window)' is a plausible extrapolation from the doc statement that 'None is equivalent to setting min_periods equal to the size of the window', but the specific implementation detail (math.prod) is not in the doc.",
      "The documentation states min_periods default None 'is equivalent to setting min_periods equal to the size of the window' - the answer's claim about 'math.prod(self.window)' for n-dimensional rolling is not contradicted but is unsupported detail."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about design rationale (performance optimization reasons, GH issue references, memory copy explanations) rely on stripped comments/docs and need removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_05",
    "repo": "xarray",
    "question": "What does `xarray.corr` do, and how does its implementation differ from `xarray.cov` in terms of parameters and internal computation?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `xarray.corr` function computes the correlation coefficient between two DataArray objects. It is defined in `xarray/computation/computation.py` and is exposed at the top level via `xarray/__init__.py`.\n\n**Key implementation details:**\n\n1. **Internal delegation**: Both `corr()` and `cov()` delegate to a shared private function `_cov_corr()`. The `corr()` function calls `_cov_corr(da_a, da_b, weights=weights, dim=dim, method=\"corr\")` without passing a `ddof` parameter. The `_cov_corr` function's default `ddof` is 0 (not 1), though `ddof` is only relevant when `method=\"cov\"` \u2014 when `method=\"corr\"`, `ddof` is never used.\n\n2. **Unlike `cov()`, `corr()` does not accept a `ddof` parameter** in its signature. The `cov()` function accepts `ddof` with a default of 1, while `corr()` has no such parameter.\n\n3. **Alignment**: The `_cov_corr` function first aligns the two input arrays using `align(da_a, da_b, join=\"inner\", copy=False)` from `xarray.structure.alignment`.\n\n4. **NaN handling**: After alignment, a `valid_values` mask is computed as `da_a.notnull() & da_b.notnull()`, and both arrays are filtered using `.where(valid_values)` to ensure NaN values in either array propagate to both. The covariance computation explicitly uses `skipna=True`.\n\n5. **Complex number support**: The covariance is computed as `(demeaned_da_a.conj() * demeaned_da_b)`, where `.conj()` takes the complex conjugate of `da_a`, allowing `corr` to work correctly with complex-valued data.\n\n6. **Correlation formula**: For the `corr` method branch, after computing `cov`, it computes `corr = cov / (da_a_std * da_b_std)` where std is computed either as unweighted `da_a.std(dim=dim)` or weighted `da_a.weighted(weights).std(dim=dim)`.\n\n7. **Weighted operations chain**: When weights are provided, detrending uses `da_a.weighted(weights).mean(dim=dim)`, covariance uses `.weighted(weights).mean(dim=dim, skipna=True)`, and normalization uses `.weighted(weights).std(dim=dim)`. The `Weighted._weighted_std` method is implemented as `np.sqrt(self._weighted_var(da, dim, skipna))`, where `_weighted_var` computes `sum_of_squares / sum_of_weights`. The `_reduce` static method of `Weighted` uses `dot()` (from the same computation module) internally.\n\n8. **Type checking**: Both `corr()` and `cov()` raise `TypeError` with message \"Only xr.DataArray is supported.Given {types}.\" if either `da_a` or `da_b` is not a DataArray. The same error is raised for non-DataArray weights.\n\n9. **Return type**: The result is wrapped in `cast(T_DataArray, corr)`.",
    "rubric": [
      "States that corr computes the Pearson correlation coefficient between two DataArray objects",
      "Explains that both corr() and cov() delegate to a shared internal function _cov_corr(), with corr passing method='corr' and cov passing method='cov'",
      "Notes that corr() does not accept a ddof parameter, while cov() does (with a default of 1)",
      "Describes that _cov_corr aligns the two arrays using align() with join='inner'",
      "Explains NaN handling: a valid_values mask is computed from notnull() on both arrays, and both are filtered with .where(valid_values)",
      "Mentions the use of .conj() on the first demeaned array for complex number support",
      "Explains the correlation formula: cov / (da_a_std * da_b_std), where std is optionally weighted",
      "Notes that corr() and cov() raise TypeError if inputs are not DataArray objects"
    ],
    "key_files": [
      "xarray/computation/computation.py",
      "xarray/computation/weighted.py",
      "xarray/__init__.py",
      "xarray/structure/alignment.py"
    ],
    "source_doc": "[docstring: xarray/computation/computation.py] xarray.computation.computation.corr\nCompute the Pearson correlation coefficient between\n    two DataArray objects along a shared dimension.\n\n    Parameters\n    ----------\n    da_a : DataArray\n        Array to compute.\n    da_b : DataArray\n        Array to compute.\n    dim : str, iterable of hashable, \"...\" or None, optional\n        The dimension along which the correlation will be computed\n    weights : DataArray, optional\n        Array of weights.\n\n    Returns\n    -------\n    correlation: DataArray\n\n    See Also\n    --------\n    pandas.Series.corr : corresponding pandas function\n    xarray.cov : underlying covariance function\n\n    Examples\n    --------\n    >>> from xarray import DataArray\n    >>> da_a = DataArray(\n    ...     np.array([[1, 2, 3], [0.1, 0.2, 0.3], [3.2, 0.6, 1.8]]),\n    ...     dims=(\"space\", \"time\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...         (\"time\", pd.date_range(\"2000-01-01\", freq=\"1D\", periods=3)),\n    ...     ],\n    ... )\n    >>> da_a\n    <xarray.DataArray (space: 3, time: 3)> Size: 72B\n    array([[1. , 2. , 3. ],\n           [0.1, 0.2, 0.3],\n           [3.2, 0.6, 1.8]])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03\n    >>> da_b = DataArray(\n    ...     np.array([[0.2, 0.4, 0.6], [15, 10, 5], [3.2, 0.6, 1.8]]),\n    ...     dims=(\"space\", \"time\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...         (\"time\", pd.date_range(\"2000-01-01\", freq=\"1D\", periods=3)),\n    ...     ],\n    ... )\n    >>> da_b\n    <xarray.DataArray (space: 3, time: 3)> Size: 72B\n    array([[ 0.2,  0.4,  0.6],\n           [15. , 10. ,  5. ],\n           [ 3.2,  0.6,  1.8]])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03\n    >>> xr.corr(da_a, da_b)\n    <xarray.DataArray ()> Size: 8B\n    array(-0.57087777)\n    >>> xr.corr(da_a, da_b, dim=\"time\")\n    <xarray.DataArray (space: 3)> Size: 24B\n    array([ 1., -1.,  1.])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n    >>> weights = DataArray(\n    ...     [4, 2, 1],\n    ...     dims=(\"space\"),\n    ...     coords=[\n    ...         (\"space\", [\"IA\", \"IL\", \"IN\"]),\n    ...     ],\n    ... )\n    >>> weights\n    <xarray.DataArray (space: 3)> Size: 24B\n    array([4, 2, 1])\n    Coordinates:\n      * space    (space) <U2 24B 'IA' 'IL' 'IN'\n    >>> xr.corr(da_a, da_b, dim=\"space\", weights=weights)\n    <xarray.DataArray (time: 3)> Size: 24B\n    array([-0.50240504, -0.83215028, -0.99057446])\n    Coordinates:\n      * time     (time) datetime64[us] 24B 2000-01-01 2000-01-02 2000-01-03",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in rationale descriptions (Pearson label, unbiased estimation purpose, auto-covariance comment, type system limitations), but most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_07",
    "repo": "xarray",
    "question": "Where is the Coarsen class defined, what does it inherit from, and how do its reduction methods get injected?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `Coarsen` class is located in `xarray/computation/rolling.py`. It inherits from `CoarsenArithmetic` (defined in `xarray/computation/arithmetic.py`) and `Generic[T_Xarray]`. The `CoarsenArithmetic` class itself inherits from `IncludeReduceMethods` (defined in `xarray/computation/ops.py`), which uses `__init_subclass__` to automatically inject reduction methods into concrete subclasses.\n\nKey implementation details:\n\n1. **Subclasses**: `DataArrayCoarsen` and `DatasetCoarsen` are the concrete subclasses that provide `_reduce_method` classmethod implementations, which are required by the injection mechanism in `inject_reduce_methods`.\n\n2. **Injected reduction methods**: Through `inject_reduce_methods`, the Coarsen subclasses get: `all`, `any` (from `REDUCE_METHODS`), `max`, `min`, `mean`, `prod`, `sum`, `std`, `var`, `median` (from `NAN_REDUCE_METHODS`), and `count`.\n\n3. **`trim_excess` ghost attribute**: The `__slots__` tuple and `_attributes` tuple both include `\"trim_excess\"`, but it is never assigned in `__init__`. The `__repr__` method guards against this using `getattr(self, k, None) is not None`, so `trim_excess` is silently excluded from the string representation.\n\n4. **Missing dimension errors**: `Rolling.__init__` raises `KeyError` for missing dimensions, while `Coarsen.__init__` raises `ValueError`.\n\n5. **`coord_func` default behavior**: When `coord_func` is not dict-like, it's broadcast to all dimensions via `dict.fromkeys(self.obj.dims, coord_func)`. Then, for any coordinate not already in the map, `duck_array_ops.mean` is used as the default function.\n\n6. **`construct` method validation**: The `construct` method performs three validation checks: (a) each window_dim entry must be a non-string sequence of exactly 2 elements, (b) `window_dim` keys must be a subset of `self.windows`, and (c) `self.windows` keys must be a subset of `window_dim` \u2014 both directions are checked separately with different error messages.\n\n7. **DataArray handling in `construct`**: For DataArray objects, `construct` converts the DataArray to a temporary Dataset using `_to_temp_dataset()` (which stores the array under the name `_THIS_ARRAY = ReprObject(\"<this-array>\")`), performs the reshape, then converts back using `_from_temp_dataset()`.\n\n8. **Variable.coarsen_reshape**: The actual reshaping (in `xarray/core/variable.py`) handles boundary modes: 'exact' raises ValueError if size isn't divisible, 'trim' slices from left or right depending on `side`, and 'pad' uses `variable.pad()` with mode=\"constant\". The method returns a tuple of `(reshaped_data, axes)` where axes are the positions of the window dimensions in the reshaped array.\n\n9. **DataArray.coarsen()** and **Dataset.coarsen()** create `DataArrayCoarsen` and `DatasetCoarsen` respectively, both using `either_dict_or_kwargs` to handle dimension arguments.\n\n10. **`DataArrayCoarsen._reduce_method`**: Calls `self.obj.variable.coarsen()` which in turn calls `coarsen_reshape` and then applies `func(reshaped, axis=axes)`. Coordinates are coarsened separately using functions from `self.coord_func`.",
    "rubric": [
      "Identifies that Coarsen is defined in xarray/computation/rolling.py",
      "States that Coarsen inherits from CoarsenArithmetic and Generic[T_Xarray]",
      "Identifies that CoarsenArithmetic inherits from IncludeReduceMethods (in xarray/computation/ops.py)",
      "Explains that IncludeReduceMethods uses __init_subclass__ to call inject_reduce_methods on subclasses that define _reduce_method",
      "Identifies DataArrayCoarsen and DatasetCoarsen as the concrete subclasses that provide _reduce_method",
      "Lists the injected methods: all, any (REDUCE_METHODS), max, min, mean, prod, sum, std, var, median (NAN_REDUCE_METHODS), and count"
    ],
    "key_files": [
      "xarray/computation/rolling.py",
      "xarray/computation/arithmetic.py",
      "xarray/computation/ops.py",
      "xarray/core/variable.py",
      "xarray/core/dataarray.py",
      "xarray/core/dataset.py"
    ],
    "source_doc": "[docstring: xarray/computation/rolling.py] xarray.computation.rolling.Coarsen\nxarray.computation.rolling.Coarsen:\n  A object that implements the coarsen.\n\n    See Also\n    --------\n    Dataset.coarsen\n    DataArray.coarsen\n\nxarray.computation.rolling.Coarsen.__init__:\n  Moving window object.\n\n        Parameters\n        ----------\n        obj : Dataset or DataArray\n            Object to window.\n        windows : mapping of hashable to int\n            A mapping from the name of the dimension to create the rolling\n            window along (e.g. `time`) to the size of the moving window.\n        boundary : {\"exact\", \"trim\", \"pad\"}\n            If 'exact', a ValueError will be raised if dimension size is not a\n            multiple of window size. If 'trim', the excess indexes are trimmed.\n            If 'pad', NA will be padded.\n        side : 'left' or 'right' or mapping from dimension to 'left' or 'right'\n        coord_func : function (name) or mapping from coordinate name to function (name).\n\n        Returns\n        -------\n        coarsen\n\nxarray.computation.rolling.Coarsen.__repr__:\n  provide a nice str repr of our coarsen object\n\nxarray.computation.rolling.Coarsen.construct:\n  Convert this Coarsen object to a DataArray or Dataset,\n        where the coarsening dimension is split or reshaped to two\n        new dimensions.\n\n        Parameters\n        ----------\n        window_dim: mapping\n            A mapping from existing dimension name to new dimension names.\n            The size of the second dimension will be the length of the\n            coarsening window.\n        keep_attrs: bool, optional\n            Preserve attributes if True\n        **window_dim_kwargs : {dim: new_name, ...}\n            The keyword arguments form of ``window_dim``.\n\n        Returns\n        -------\n        Dataset or DataArray with reshaped dimensions\n\n        Examples\n        --------\n        >>> da = xr.DataArray(np.arange(24), dims=\"time\")\n        >>> da.coarsen(time=12).construct(time=(\"year\", \"month\"))\n        <xarray.DataArray (year: 2, month: 12)> Size: 192B\n        array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11],\n               [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])\n        Dimensions without coordinates: year, month\n\n        See Also\n        --------\n        DataArrayRolling.construct\n        DatasetRolling.construct",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from code structure; removed comment-dependent claim and line number references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_08",
    "repo": "xarray",
    "question": "How does the `RollingExp` class implement exponentially-weighted moving window computations, including alpha conversion and delegation to numbagg?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `RollingExp` class implements exponentially-weighted moving window operations by delegating all computation to the `numbagg` library via `apply_ufunc`. Here is how the algorithm is implemented:\n\n**Initialization (`__init__`):**\n1. The constructor checks that `numbagg` is available using `module_available(\"numbagg\")`.\n2. Only a single dimension is extracted from the `windows` mapping using `next(iter(windows.items()))` \u2014 so even if multiple dimensions are provided, only the first one is used.\n3. The `window_type` and window value are converted to an alpha parameter via `_get_alpha()`, which implements these formulas:\n   - `com`: `alpha = 1 / (com + 1)`, requires `com >= 0`\n   - `span`: `alpha = 2 / (span + 1)`, requires `span >= 1`\n   - `halflife`: `alpha = 1 - exp(log(0.5) / halflife)`, requires `halflife > 0`\n   - `alpha`: passed through directly, requires `0 < alpha <= 1`\n4. A `kwargs` dict is built with `alpha` and `axis=-1`. The `min_weight` keyword is only included in `kwargs` if `min_weight > 0`.\n\n**Method implementations:**\n- All six methods (`mean`, `sum`, `std`, `var`, `cov`, `corr`) follow the same pattern: they call `apply_ufunc` with the corresponding `numbagg` function, passing `input_core_dims=[[self.dim]]` (or `[[self.dim], [self.dim]]` for two-argument methods `cov` and `corr`), `output_core_dims=[[self.dim]]`, `on_missing_core_dim=\"copy\"`, and `dask=\"parallelized\"`. Each method then calls `.transpose(*dim_order)` on the result to restore the original dimension order.\n- **`keep_attrs` handling diverges across methods:** `mean()` and `sum()` accept a `keep_attrs` parameter and use `_get_keep_attrs(default=True)` when it's `None`, which checks the global `xarray.set_options(keep_attrs=...)` setting (defaulting to `True`). In contrast, `std()`, `var()`, `cov()`, and `corr()` do **not** accept a `keep_attrs` parameter \u2014 they always hardcode `keep_attrs=True` in their `apply_ufunc` call.\n- The `_get_keep_attrs` function calls `_get_boolean_with_default(\"keep_attrs\", default)` which reads the global `OPTIONS[\"keep_attrs\"]` value. If it is `\"default\"`, it returns the `default` parameter (which is `True` for these methods).\n\n**numbagg functions used:**\n- `mean` \u2192 `numbagg.move_exp_nanmean`\n- `sum` \u2192 `numbagg.move_exp_nansum`\n- `std` \u2192 `numbagg.move_exp_nanstd`\n- `var` \u2192 `numbagg.move_exp_nanvar`\n- `cov` \u2192 `numbagg.move_exp_nancov`\n- `corr` \u2192 `numbagg.move_exp_nancorr`\n\n**`on_missing_core_dim=\"copy\"` behavior:** When applied to a Dataset, if a variable is missing the core dimension, `apply_dict_of_variables_vfunc` copies that variable unchanged into the result (rather than raising an error or dropping it).\n\n**Entry point:** The `DataWithCoords.rolling_exp()` method (in `xarray/core/common.py`) creates a `RollingExp` object. It warns if `keep_attrs` is passed as a keyword argument, and does not pass `min_weight` to the constructor (so it defaults to `0.0`).\n\n**`_get_alpha` validation:** The `count_not_none` function ensures that at most one of `com`, `span`, `halflife`, or `alpha` is specified \u2014 if more than one is given, a `ValueError` is raised.",
    "rubric": [
      "Explains that `_get_alpha` converts com/span/halflife/alpha to a single alpha value with formulas: com\u21921/(com+1), span\u21922/(span+1), halflife\u21921-exp(log(0.5)/halflife), alpha passed through",
      "Mentions that only one of com/span/halflife/alpha can be specified (mutually exclusive, enforced via count_not_none)",
      "Explains that __init__ extracts a single dimension from windows via next(iter(windows.items())) and checks numbagg availability",
      "Describes that kwargs contains alpha and axis=-1, with min_weight only included if greater than 0",
      "Identifies the numbagg functions used: move_exp_nanmean, move_exp_nansum, move_exp_nanstd, move_exp_nanvar, move_exp_nancov, move_exp_nancorr",
      "Explains that all methods use apply_ufunc with input_core_dims=[[self.dim]], output_core_dims=[[self.dim]], on_missing_core_dim='copy', and dask='parallelized'",
      "Notes that cov and corr take two inputs with input_core_dims=[[self.dim], [self.dim]]",
      "Explains the keep_attrs divergence: mean/sum accept keep_attrs parameter (defaulting via _get_keep_attrs(default=True)), while std/var/cov/corr hardcode keep_attrs=True",
      "Mentions that results are transposed back to original dimension order via .transpose(*dim_order)"
    ],
    "key_files": [
      "xarray/computation/rolling_exp.py",
      "xarray/core/common.py",
      "xarray/core/options.py",
      "xarray/compat/pdcompat.py",
      "xarray/computation/apply_ufunc.py"
    ],
    "source_doc": "[docstring: xarray/computation/rolling_exp.py] xarray.computation.rolling_exp.RollingExp\nxarray.computation.rolling_exp.RollingExp:\n  Exponentially-weighted moving window object.\n    Similar to EWM in pandas\n\n    Parameters\n    ----------\n    obj : Dataset or DataArray\n        Object to window.\n    windows : mapping of hashable to int (or float for alpha type)\n        A mapping from the name of the dimension to create the rolling\n        exponential window along (e.g. `time`) to the size of the moving window.\n    window_type : {\"span\", \"com\", \"halflife\", \"alpha\"}, default: \"span\"\n        The format of the previously supplied window. Each is a simple\n        numerical transformation of the others. Described in detail:\n        https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.ewm.html\n\n    Returns\n    -------\n    RollingExp : type of input argument\n\nxarray.computation.rolling_exp.RollingExp.mean:\n  Exponentially weighted moving average.\n\n        Parameters\n        ----------\n        keep_attrs : bool, default: None\n            If True, the attributes (``attrs``) will be copied from the original\n            object to the new one. If False, the new object will be returned\n            without attributes. If None uses the global default.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").mean()\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([1.        , 1.        , 1.69230769, 1.9       , 1.96694215])\n        Dimensions without coordinates: x\n\nxarray.computation.rolling_exp.RollingExp.sum:\n  Exponentially weighted moving sum.\n\n        Parameters\n        ----------\n        keep_attrs : bool, default: None\n            If True, the attributes (``attrs``) will be copied from the original\n            object to the new one. If False, the new object will be returned\n            without attributes. If None uses the global default.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").sum()\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([1.        , 1.33333333, 2.44444444, 2.81481481, 2.9382716 ])\n        Dimensions without coordinates: x\n\nxarray.computation.rolling_exp.RollingExp.std:\n  Exponentially weighted moving standard deviation.\n\n        `keep_attrs` is always True for this method. Drop attrs separately to remove attrs.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").std()\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([       nan, 0.        , 0.67936622, 0.42966892, 0.25389527])\n        Dimensions without coordinates: x\n\nxarray.computation.rolling_exp.RollingExp.var:\n  Exponentially weighted moving variance.\n\n        `keep_attrs` is always True for this method. Drop attrs separately to remove attrs.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").var()\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([       nan, 0.        , 0.46153846, 0.18461538, 0.06446281])\n        Dimensions without coordinates: x\n\nxarray.computation.rolling_exp.RollingExp.cov:\n  Exponentially weighted moving covariance.\n\n        `keep_attrs` is always True for this method. Drop attrs separately to remove attrs.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").cov(da**2)\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([       nan, 0.        , 1.38461538, 0.55384615, 0.19338843])\n        Dimensions without coordinates: x\n\nxarray.computation.rolling_exp.RollingExp.corr:\n  Exponentially weighted moving correlation.\n\n        `keep_attrs` is always True for this method. Drop attrs separately to remove attrs.\n\n        Examples\n        --------\n        >>> da = xr.DataArray([1, 1, 2, 2, 2], dims=\"x\")\n        >>> da.rolling_exp(x=2, window_type=\"span\").corr(da.shift(x=1))\n        <xarray.DataArray (x: 5)> Size: 40B\n        array([       nan,        nan,        nan, 0.4330127 , 0.48038446])\n        Dimensions without coordinates: x",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: rationale about backward compatibility from a comment, 'copied from pandas internals' provenance, 'since it's now per-method' explanation, and implementation detail about importlib.util.find_spec.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_10",
    "repo": "xarray",
    "question": "Why does `full_like` use `chunkmanager.array_api.full()` for chunked arrays instead of creating a numpy array and converting it with `from_array`?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `full_like` function in `xarray/core/common.py` uses a performance optimization for chunked arrays: it calls `chunkmanager.array_api.full()` to create chunked arrays directly rather than creating a filled numpy array and then converting it. For dask specifically, the `DaskManager.array_api` property returns `dask.array`, meaning `dask.array.full()` is called with the shape, fill_value, dtype, and chunks parameters all at once. This creates a lazy dask array that is already properly chunked.\n\nIn contrast, the non-chunked path uses `duck_array_ops.full_like(other.data, fill_value, dtype=dtype)` which calls `xp.full_like(a, fill_value, **kwargs)` via the array namespace (numpy or cupy), creating an immediately materialized array.\n\nKey design decisions:\n1. **Using `array_api.full()` instead of `from_array`**: The code uses `chunkmanager.array_api.full()` which creates the chunked representation directly, rather than using `chunkmanager.from_array()` which would require an intermediate array to exist first.\n2. **Chunk preservation**: When no explicit chunks are provided, the code reuses the existing chunk structure from the input array.\n3. **Sentinel NA value for missing dict keys**: When `fill_value` is passed as a dict for Datasets, variables whose keys are absent from the dict receive `dtypes.NA` (a `ReprObject` sentinel). The `_full_like_variable` function then resolves this to an appropriate NA value via `dtypes.get_fill_value()`, which internally calls `maybe_promote()` to determine the correct missing value for the dtype (e.g., NaN for floats, NaT for datetime64/timedelta64).",
    "rubric": [
      "Explains that `array_api.full()` creates the chunked/lazy array directly without needing an intermediate materialized array",
      "Notes that for dask, `array_api` returns `dask.array`, so `dask.array.full()` is called with shape, fill_value, dtype, and chunks all at once",
      "Mentions the performance benefit: avoids creating and then re-chunking a fully materialized numpy array",
      "Describes the non-chunked path using `duck_array_ops.full_like` which calls the array namespace's `full_like` to create an immediately materialized array",
      "Notes that when no explicit chunks are given, the existing chunk structure from the input array is reused (`other.data.chunks`)",
      "Mentions how `dtypes.NA` sentinel is used for missing dict keys in Dataset fill values and resolved via `get_fill_value`/`maybe_promote`"
    ],
    "key_files": [
      "xarray/core/common.py",
      "xarray/namedarray/daskmanager.py",
      "xarray/namedarray/parallelcompat.py",
      "xarray/core/duck_array_ops.py",
      "xarray/core/dtypes.py"
    ],
    "source_doc": "[docstring: xarray/core/common.py] xarray.core.common.full_like\nReturn a new object with the same shape and type as a given object.\n\n    Returned object will be chunked if if the given object is chunked, or if chunks or chunked_array_type are specified.\n\n    Parameters\n    ----------\n    other : DataArray, Dataset or Variable\n        The reference object in input\n    fill_value : scalar or dict-like\n        Value to fill the new object with before returning it. If\n        other is a Dataset, may also be a dict-like mapping data\n        variables to fill values.\n    dtype : dtype or dict-like of dtype, optional\n        dtype of the new array. If a dict-like, maps dtypes to\n        variables. If omitted, it defaults to other.dtype.\n    chunks : int, \"auto\", tuple of int or mapping of Hashable to int, optional\n        Chunk sizes along each dimension, e.g., ``5``, ``\"auto\"``, ``(5, 5)`` or\n        ``{\"x\": 5, \"y\": 5}``.\n    chunked_array_type: str, optional\n        Which chunked array type to coerce the underlying data array to.\n        Defaults to 'dask' if installed, else whatever is registered via the `ChunkManagerEnetryPoint` system.\n        Experimental API that should not be relied upon.\n    from_array_kwargs: dict, optional\n        Additional keyword arguments passed on to the `ChunkManagerEntrypoint.from_array` method used to create\n        chunked arrays, via whichever chunk manager is specified through the `chunked_array_type` kwarg.\n        For example, with dask as the default chunked array type, this method would pass additional kwargs\n        to :py:func:`dask.array.from_array`. Experimental API that should not be relied upon.\n\n    Returns\n    -------\n    out : same as object\n        New object with the same shape and type as other, with the data\n        filled with fill_value. Coords will be copied from other.\n        If other is based on dask, the new one will be as well, and will be\n        split in the same chunks.\n\n    Examples\n    --------\n    >>> x = xr.DataArray(\n    ...     np.arange(6).reshape(2, 3),\n    ...     dims=[\"lat\", \"lon\"],\n    ...     coords={\"lat\": [1, 2], \"lon\": [0, 1, 2]},\n    ... )\n    >>> x\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.full_like(x, 1)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[1, 1, 1],\n           [1, 1, 1]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.full_like(x, 0.5)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0, 0, 0],\n           [0, 0, 0]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.full_like(x, 0.5, dtype=np.double)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0.5, 0.5, 0.5],\n           [0.5, 0.5, 0.5]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.full_like(x, np.nan, dtype=np.double)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[nan, nan, nan],\n           [nan, nan, nan]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> ds = xr.Dataset(\n    ...     {\"a\": (\"x\", [3, 5, 2]), \"b\": (\"x\", [9, 1, 0])}, coords={\"x\": [2, 4, 6]}\n    ... )\n    >>> ds\n    <xarray.Dataset> Size: 72B\n    Dimensions:  (x: 3)\n    Coordinates:\n      * x        (x) int64 24B 2 4 6\n    Data variables:\n        a        (x) int64 24B 3 5 2\n        b        (x) int64 24B 9 1 0\n    >>> xr.full_like(ds, fill_value={\"a\": 1, \"b\": 2})\n    <xarray.Dataset> Size: 72B\n    Dimensions:  (x: 3)\n    Coordinates:\n      * x        (x) int64 24B 2 4 6\n    Data variables:\n        a        (x) int64 24B 1 1 1\n        b        (x) int64 24B 2 2 2\n    >>> xr.full_like(ds, fill_value={\"a\": 1, \"b\": 2}, dtype={\"a\": bool, \"b\": float})\n    <xarray.Dataset> Size: 51B\n    Dimensions:  (x: 3)\n    Coordinates:\n      * x        (x) int64 24B 2 4 6\n    Data variables:\n        a        (x) bool 3B True True True\n        b        (x) float64 24B 2.0 2.0 2.0\n\n    See Also\n    --------\n    zeros_like\n    ones_like",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (line 1773, line 93-96 of daskmanager.py) and internal implementation details (chunkmanager.array_api.full(), duck_array_ops.full_like, dtypes.NA, dtypes.get_fill_value(), maybe_promote()) that are not present in the provided documentation. These are plausible implementation details but cannot be verified from the docstring alone.",
      "The answer claims specific performance optimization strategies (avoiding from_array for constant fills, direct lazy dask array creation) that are not described in the documentation. These are reasonable inferences about implementation but are not supported by the provided text.",
      "The documentation does support that the returned object will be chunked if the input is chunked or if chunks/chunked_array_type are specified, and that it will be 'split in the same chunks' - which aligns with the answer's claim about chunk preservation.",
      "The documentation does support that fill_value can be a dict-like for Datasets, which aligns with the answer's discussion of dict-based fill values.",
      "The answer's claim about 'sentinel NA value for missing dict keys' is not verifiable from the documentation - the docs only show examples where all keys are provided in the dict."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code structure; some performance rationale language was softened as it derives from design intent rather than code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_09",
    "repo": "xarray",
    "question": "What is the full architecture of the `.dt` accessor on DataArray, including how it dispatches between datetime and timedelta types?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `DatetimeAccessor` class in `xarray/core/accessor_dt.py` is part of a class hierarchy where `DatetimeAccessor` extends `TimeAccessor`, which is a generic class (`Generic[T_DataArray]`). The `.dt` attribute on `DataArray` is actually registered via `CombinedDatetimelikeAccessor`, which inherits from both `DatetimeAccessor` and `TimedeltaAccessor` and uses a custom `__new__` method that never actually instantiates itself\u2014instead it returns either a `DatetimeAccessor` or `TimedeltaAccessor` instance based on the dtype check.\n\nKey architectural and dependency details:\n\n1. **Registration mechanism**: The `.dt` accessor is registered on `DataArray` as `dt = utils.UncachedAccessor(CombinedDatetimelikeAccessor['DataArray'])`. `UncachedAccessor` is defined in `xarray/core/utils.py` and is a descriptor that, when accessed on an instance, triggers `CombinedDatetimelikeAccessor.__new__`.\n\n2. **Dtype dispatch in CombinedDatetimelikeAccessor.__new__**: Uses `_contains_datetime_like_objects(obj.variable)` to validate the data contains datetime-like objects. If the dtype is timedelta64 (checked via `is_np_timedelta_like`), returns `TimedeltaAccessor(obj)`; otherwise returns `DatetimeAccessor(obj)`. If validation fails, it raises `AttributeError` (not `TypeError`).\n\n3. **`_index_or_data` optimization**: The helper function `_index_or_data` checks if the DataArray's variable is an `IndexVariable` and if so, calls `obj.to_index()` to return a `pd.Index` directly rather than converting to a numpy array.\n\n4. **Two access paths for datetime fields**: `_get_date_field` dispatches between `_access_through_series` (for numpy datetime64/timedelta64 dtypes) and `_access_through_cftimeindex` (for cftime objects), determined by `is_np_datetime_like(values.dtype)`.\n\n5. **Season computation**: The `_season_from_months` function maps month numbers to season strings using the formula `idx = (months // 3) % 4` to index into `['DJF', 'MAM', 'JJA', 'SON', 'nan']`, where NaN months map to index 4 (the 'nan' string).\n\n6. **cftime 'date' attribute error**: `_access_through_cftimeindex` explicitly raises an `AttributeError` when the field name is 'date', telling users to use `.floor('D')` instead, since CFTimeIndex has no `date` attribute.\n\n7. **`isocalendar` special handling**: In `_access_through_series`, `isocalendar` gets special NaT handling\u2014it checks for null values via `field_values.year.isnull()` and converts to float64 if present, otherwise uses int64. The result is reshaped from (N, 3) to (3, *values.shape). In the `DatetimeAccessor.isocalendar()` method, it explicitly raises `AttributeError` for cftime objects.\n\n8. **`calendar` property**: Calls `infer_calendar_name` from `xarray/coding/times.py`.\n\n9. **`decimal_year` implementation**: Delegates to `_decimal_year` in `xarray/coding/calendar_ops.py`, which uses `apply_ufunc` with `vectorize=True` and `dask='parallelized'`. It dispatches between `_decimal_year_numpy` and `_decimal_year_cftime` based on whether `times.dtype == 'O'`. The numpy version calculates `year + (time - year_start) / timedelta64(days_in_year, 'D')`.\n\n10. **`days_in_year` property**: Computes leap year information\u2014for '360_day' calendars returns a constant 360 using `full_like`, otherwise calculates `is_leap_year.astype(int) + 365`.\n\n11. **CFTimeIndex field accessors**: Defined via `_field_accessor` factory function in `cftimeindex.py`, which creates property objects that check the cftime version. The `dayofyear` field accessor maps to the cftime attribute name `'dayofyr'` (not `'dayofyear'`), and `dayofweek` maps to `'dayofwk'`.\n\n12. **Dask support**: `_get_date_field` and `_round_field` both check for dask arrays using `is_duck_dask_array` (from `xarray/namedarray/utils.py`) and use `dask.array.map_blocks` for lazy evaluation.",
    "rubric": [
      "Explains that `.dt` is registered via `UncachedAccessor(CombinedDatetimelikeAccessor)` on DataArray",
      "Describes that `UncachedAccessor` is a descriptor in utils.py that calls the accessor class when accessed on an instance",
      "Explains that `CombinedDatetimelikeAccessor.__new__` never instantiates itself but returns either `DatetimeAccessor` or `TimedeltaAccessor`",
      "Mentions dtype dispatch uses `is_np_timedelta_like` to decide between TimedeltaAccessor and DatetimeAccessor",
      "Notes that `_contains_datetime_like_objects` validation raises `AttributeError` if data isn't datetime-like",
      "Describes `DatetimeAccessor` extends `TimeAccessor` which is generic over T_DataArray",
      "Explains `_index_or_data` optimization that uses `obj.to_index()` for IndexVariable instead of numpy array conversion",
      "Describes dual access paths: `_access_through_series` for numpy dtypes and `_access_through_cftimeindex` for cftime",
      "Mentions season computation using `(months // 3) % 4` indexing into ['DJF', 'MAM', 'JJA', 'SON', 'nan']",
      "Notes that `_access_through_cftimeindex` raises AttributeError for 'date' field suggesting `.floor('D')` instead",
      "Describes `isocalendar` special NaT handling and reshape from (N,3) to (3, *values.shape)",
      "Mentions dask support via `is_duck_dask_array` check and `dask.array.map_blocks`"
    ],
    "key_files": [
      "xarray/core/accessor_dt.py",
      "xarray/core/dataarray.py",
      "xarray/core/utils.py",
      "xarray/core/common.py",
      "xarray/coding/calendar_ops.py",
      "xarray/coding/times.py",
      "xarray/coding/cftimeindex.py"
    ],
    "source_doc": "[docstring: xarray/core/accessor_dt.py] xarray.core.accessor_dt.DatetimeAccessor\nxarray.core.accessor_dt.DatetimeAccessor:\n  Access datetime fields for DataArrays with datetime-like dtypes.\n\n    Fields can be accessed through the `.dt` attribute\n    for applicable DataArrays.\n\n    Examples\n    ---------\n    >>> dates = pd.date_range(start=\"2000/01/01\", freq=\"D\", periods=10)\n    >>> ts = xr.DataArray(dates, dims=(\"time\"))\n    >>> ts\n    <xarray.DataArray (time: 10)> Size: 80B\n    array(['2000-01-01T00:00:00.000000', '2000-01-02T00:00:00.000000',\n           '2000-01-03T00:00:00.000000', '2000-01-04T00:00:00.000000',\n           '2000-01-05T00:00:00.000000', '2000-01-06T00:00:00.000000',\n           '2000-01-07T00:00:00.000000', '2000-01-08T00:00:00.000000',\n           '2000-01-09T00:00:00.000000', '2000-01-10T00:00:00.000000'],\n          dtype='datetime64[us]')\n    Coordinates:\n      * time     (time) datetime64[us] 80B 2000-01-01 2000-01-02 ... 2000-01-10\n    >>> ts.dt  # doctest: +ELLIPSIS\n    <xarray.core.accessor_dt.DatetimeAccessor object at 0x...>\n    >>> ts.dt.dayofyear\n    <xarray.DataArray 'dayofyear' (time: 10)> Size: 80B\n    array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])\n    Coordinates:\n      * time     (time) datetime64[us] 80B 2000-01-01 2000-01-02 ... 2000-01-10\n    >>> ts.dt.quarter\n    <xarray.DataArray 'quarter' (time: 10)> Size: 80B\n    array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1])\n    Coordinates:\n      * time     (time) datetime64[us] 80B 2000-01-01 2000-01-02 ... 2000-01-10\n\nxarray.core.accessor_dt.DatetimeAccessor.strftime:\n  Return an array of formatted strings specified by date_format, which\n        supports the same string format as the python standard library. Details\n        of the string format can be found in `python string format doc\n        <https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior>`__\n\n        Parameters\n        ----------\n        date_format : str\n            date format string (e.g. \"%Y-%m-%d\")\n\n        Returns\n        -------\n        formatted strings : same type as values\n            Array-like of strings formatted for each element in values\n\n        Examples\n        --------\n        >>> import datetime\n        >>> rng = xr.Dataset({\"time\": datetime.datetime(2000, 1, 1)})\n        >>> rng[\"time\"].dt.strftime(\"%B %d, %Y, %r\")\n        <xarray.DataArray 'strftime' ()> Size: 8B\n        array('January 01, 2000, 12:00:00 AM', dtype=object)\n\nxarray.core.accessor_dt.DatetimeAccessor.isocalendar:\n  Dataset containing ISO year, week number, and weekday.\n\n        Notes\n        -----\n        The iso year and weekday differ from the nominal year and weekday.\n\nxarray.core.accessor_dt.DatetimeAccessor.year:\n  The year of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.month:\n  The month as January=1, December=12\n\nxarray.core.accessor_dt.DatetimeAccessor.day:\n  The days of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.hour:\n  The hours of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.minute:\n  The minutes of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.second:\n  The seconds of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.microsecond:\n  The microseconds of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.nanosecond:\n  The nanoseconds of the datetime\n\nxarray.core.accessor_dt.DatetimeAccessor.weekofyear:\n  The week ordinal of the year\n\nxarray.core.accessor_dt.DatetimeAccessor.dayofweek:\n  The day of the week with Monday=0, Sunday=6\n\nxarray.core.accessor_dt.DatetimeAccessor.dayofyear:\n  The ordinal day of the year\n\nxarray.core.accessor_dt.DatetimeAccessor.quarter:\n  The quarter of the date\n\nxarray.core.accessor_dt.DatetimeAccessor.days_in_month:\n  The number of days in the month\n\nxarray.core.accessor_dt.DatetimeAccessor.season:\n  Season of the year\n\nxarray.core.accessor_dt.DatetimeAccessor.time:\n  Timestamps corresponding to datetimes\n\nxarray.core.accessor_dt.DatetimeAccessor.date:\n  Date corresponding to datetimes\n\nxarray.core.accessor_dt.DatetimeAccessor.is_month_start:\n  Indicate whether the date is the first day of the month\n\nxarray.core.accessor_dt.DatetimeAccessor.is_month_end:\n  Indicate whether the date is the last day of the month\n\nxarray.core.accessor_dt.DatetimeAccessor.is_quarter_start:\n  Indicate whether the date is the first day of a quarter\n\nxarray.core.accessor_dt.DatetimeAccessor.is_quarter_end:\n  Indicate whether the date is the last day of a quarter\n\nxarray.core.accessor_dt.DatetimeAccessor.is_year_start:\n  Indicate whether the date is the first day of a year\n\nxarray.core.accessor_dt.DatetimeAccessor.is_year_end:\n  Indicate whether the date is the last day of the year\n\nxarray.core.accessor_dt.DatetimeAccessor.is_leap_year:\n  Indicate if the date belongs to a leap year\n\nxarray.core.accessor_dt.DatetimeAccessor.calendar:\n  The name of the calendar of the dates.\n\n        Only relevant for arrays of :py:class:`cftime.datetime` objects,\n        returns \"proleptic_gregorian\" for arrays of :py:class:`numpy.datetime64` values.\n\nxarray.core.accessor_dt.DatetimeAccessor.days_in_year:\n  Each datetime as the year plus the fraction of the year elapsed.\n\nxarray.core.accessor_dt.DatetimeAccessor.decimal_year:\n  Convert the dates as a fractional year.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states `days_in_year` is described as 'Each datetime as the year plus the fraction of the year elapsed' which seems to be a documentation error or mismatch. The answer claims `days_in_year` computes leap year information and returns days count, which contradicts the docstring but may reflect actual implementation.",
      "The documentation describes `decimal_year` as 'Convert the dates as a fractional year' while `days_in_year` is documented as 'Each datetime as the year plus the fraction of the year elapsed' - these descriptions appear swapped in the documentation itself, making verification of the answer's claims about these properties ambiguous.",
      "Most implementation details (class hierarchy, dispatch mechanisms, season computation formula, cftime handling, dask support, etc.) cannot be verified from the provided documentation as they go beyond the docstring-level information provided."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few relied on documentation for rationale (issue #8718 reference), implementation details of functions in truncated files (infer_calendar_name), and architectural explanations (PandasIndexingAdapter).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_12",
    "repo": "xarray",
    "question": "How does DatasetView enforce immutability and ensure that inherited Dataset methods return plain Dataset objects instead of DatasetView instances?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `DatasetView` class in `xarray/core/datatree.py` is a subclass of `Dataset` that prevents mutation through several key mechanisms:\n\n**Immutability enforcement:**\n1. `__init__` raises `AttributeError` with message \"DatasetView objects are not to be initialized directly\" - preventing direct construction.\n2. `__setitem__` raises `AttributeError` suggesting users use `.__setitem__` on the wrapping DataTree node, `dt.to_dataset()`, or `.copy()` within `map_over_datasets`.\n3. `update` raises `AttributeError` with similar guidance.\n4. `_replace` raises `AttributeError(\"In-place mutation of the DatasetView is not allowed\")` when `inplace=True`.\n5. `set_close` raises `AttributeError(\"cannot modify a DatasetView()\")`. \n6. `close` raises `AttributeError(\"cannot close a DatasetView(). Close the associated DataTree node instead\")`. \n\n**Return-type coercion to Dataset:**\n- `_construct_direct` (classmethod) creates `object.__new__(Dataset)` instead of `DatasetView`, ensuring any inherited method that calls `_construct_direct` returns a plain `Dataset`.\n- `_replace` delegates to `Dataset._replace(self, ...)` which internally calls `_construct_direct`, which on `DatasetView` returns a `Dataset`.\n- `map` explicitly returns `Dataset(variables, attrs=attrs)` instead of using `type(self)(...)` as the parent `Dataset.map` does.\n\n**Key difference between `DatasetView.map` and `Dataset.map`:**\nThe `DatasetView.map` is a simplified version that skips coordinate merging and directly calls `Dataset(variables, attrs=attrs)`.\n\n**Construction path:**\n`DatasetView` instances are created via the `_constructor` classmethod (using `object.__new__(cls)`) which is called from `DataTree._to_dataset_view()`. The `_resolve_inherit` method on DataTree handles resolving inherited coordinates/indexes from parent nodes.\n\n**`__slots__` difference:**\n`DatasetView.__slots__` does NOT include `\"__weakref__\"` while `Dataset.__slots__` does include it.\n\n**Repr behavior:**\nSince `DatasetView` does not override `__repr__`, it inherits `Dataset.__repr__`. Because the repr mechanism uses `type(ds).__name__`, instances display as `<xarray.DatasetView>`.",
    "rubric": [
      "Mentions that __init__ raises AttributeError to prevent direct construction",
      "Mentions that __setitem__ and update raise AttributeError to block mutation",
      "Mentions that _replace raises AttributeError when inplace=True",
      "Mentions that set_close and/or close raise AttributeError",
      "Explains that _construct_direct creates object.__new__(Dataset) instead of DatasetView, so inherited methods return plain Datasets",
      "Explains that _replace delegates to Dataset._replace (which uses _construct_direct) to return a Dataset",
      "Mentions that the map method explicitly returns Dataset(variables, attrs=attrs) instead of type(self)(...)",
      "Mentions that DatasetView instances are created via the _constructor classmethod (using object.__new__(cls)) called from DataTree._to_dataset_view"
    ],
    "key_files": [
      "xarray/core/datatree.py",
      "xarray/core/dataset.py",
      "xarray/core/formatting.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/core/datatree.py] xarray.core.datatree.DatasetView\nxarray.core.datatree.DatasetView:\n  An immutable Dataset-like view onto the data in a single DataTree node.\n\n    In-place operations modifying this object should raise an AttributeError.\n    This requires overriding all inherited constructors.\n\n    Operations returning a new result will return a new xarray.Dataset object.\n    This includes all API on Dataset, which will be inherited.\n\nxarray.core.datatree.DatasetView._constructor:\n  Private constructor, from Dataset attributes.\n\nxarray.core.datatree.DatasetView._construct_direct:\n  Overriding this method (along with ._replace) and modifying it to return a Dataset object\n        should hopefully ensure that the return type of any method on this object is a Dataset.\n\nxarray.core.datatree.DatasetView._replace:\n  Overriding this method (along with ._construct_direct) and modifying it to return a Dataset object\n        should hopefully ensure that the return type of any method on this object is a Dataset.\n\nxarray.core.datatree.DatasetView.map:\n  Apply a function to each data variable in this dataset\n\n        Parameters\n        ----------\n        func : callable\n            Function which can be called in the form `func(x, *args, **kwargs)`\n            to transform each DataArray `x` in this dataset into another\n            DataArray.\n        keep_attrs : bool | None, optional\n            If True, both the dataset's and variables' attributes (`attrs`) will be\n            copied from the original objects to the new ones. If False, the new dataset\n            and variables will be returned without copying the attributes.\n        args : iterable, optional\n            Positional arguments passed on to `func`.\n        **kwargs : Any\n            Keyword arguments passed on to `func`.\n\n        Returns\n        -------\n        applied : Dataset\n            Resulting dataset from applying ``func`` to each data variable.\n\n        Examples\n        --------\n        >>> da = xr.DataArray(np.random.randn(2, 3))\n        >>> ds = xr.Dataset({\"foo\": da, \"bar\": (\"x\", [-1, 2])})\n        >>> ds\n        <xarray.Dataset> Size: 64B\n        Dimensions:  (dim_0: 2, dim_1: 3, x: 2)\n        Dimensions without coordinates: dim_0, dim_1, x\n        Data variables:\n            foo      (dim_0, dim_1) float64 48B 1.764 0.4002 0.9787 2.241 1.868 -0.9773\n            bar      (x) int64 16B -1 2\n        >>> ds.map(np.fabs)\n        <xarray.Dataset> Size: 64B\n        Dimensions:  (dim_0: 2, dim_1: 3, x: 2)\n        Dimensions without coordinates: dim_0, dim_1, x\n        Data variables:\n            foo      (dim_0, dim_1) float64 48B 1.764 0.4002 0.9787 2.241 1.868 0.9773\n            bar      (x) float64 16B 1.0 2.0",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not explicitly state that `_replace` raises `AttributeError('In-place mutation of the DatasetView is not allowed')` when `inplace=True` - this is a plausible implementation detail but not documented.",
      "The documentation does not mention `__init__`, `__setitem__`, `update`, `set_close`, or `close` methods or their error messages - these are plausible extrapolations but not supported by the provided documentation.",
      "The documentation does not describe `DatasetView.map` as explicitly returning `Dataset(variables, attrs=attrs)` or skipping coordinate merging - the doc only says it returns a Dataset. The claim about 'referencing GitHub issue datatree#188' is unsupported.",
      "The documentation does not mention `_to_dataset_view`, `_resolve_inherit`, `rebuild_dims`, or `inherit` parameters - these are implementation details not in the provided documentation.",
      "The documentation does not mention `__slots__` or `__weakref__` differences.",
      "The claim about repr displaying `<xarray.DatasetView>` is not in the documentation.",
      "The documentation states that `_replace` is modified 'to return a Dataset object' but the answer claims it 'delegates to Dataset._replace(self, ...)' - this is a plausible implementation but the specific mechanism is not documented.",
      "The documentation states `_construct_direct` returns a Dataset object, which is consistent with the answer's claim about `object.__new__(Dataset)`, though the specific mechanism is not documented."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor leakage from line number references, a GitHub issue reference, and some design rationale language.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_11",
    "repo": "xarray",
    "question": "Where is `zeros_like` defined and how does it flow through the codebase to actually create the array?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `zeros_like` function is defined in `xarray/core/common.py`. It delegates entirely to `full_like` by calling `full_like(other, 0, dtype, ...)`. The `full_like` function (also in `xarray/core/common.py`) handles three types of input: Dataset, DataArray, and Variable. For DataArray and Variable inputs, it calls the internal helper `_full_like_variable`. For Dataset inputs, it iterates over data variables and calls `_full_like_variable` for each one, constructing a new Dataset with the original coords and attrs.\n\n`_full_like_variable` has two code paths:\n1. **Chunked path**: If the input data is a chunked array (checked via `is_chunked_array` from `xarray/namedarray/pycompat.py`), or if `chunks` or `chunked_array_type` are specified, it uses `chunkmanager.array_api.full()` to create the data.\n2. **Non-chunked path**: Otherwise, it calls `duck_array_ops.full_like(other.data, fill_value, dtype=dtype)` from `xarray/core/duck_array_ops.py`, which uses `get_array_namespace(a)` to dispatch to the appropriate array API (numpy, cupy, or any array with `__array_namespace__`).\n\nThe function is publicly exported via `xarray/__init__.py` (`from xarray.core.common import ALL_DIMS, full_like, ones_like, zeros_like`). It is also used within xarray itself, notably in `xarray/computation/computation.py`, inside the `polyval` function: `res = zeros_like(coord) + coeffs.isel({degree_dim: max_deg}, drop=True)`.\n\nFor the DataArray path, `full_like` preserves the input's `dims`, `coords`, `attrs`, and `name` attributes. For the Variable path, it preserves `dims` and `attrs`. The `dtypes.NA` sentinel (defined as `utils.ReprObject('<NA>')` in `xarray/core/dtypes.py`) is used when a Dataset's fill_value dict doesn't contain a key for a variable, and `get_fill_value` uses `maybe_promote` to determine an appropriate fill value for that dtype.",
    "rubric": [
      "Identifies that zeros_like is defined in xarray/core/common.py",
      "Explains that zeros_like delegates to full_like by passing 0 as the fill_value",
      "Describes that full_like handles Dataset, DataArray, and Variable types differently",
      "Mentions the internal helper _full_like_variable is called for DataArray/Variable inputs",
      "Explains the chunked path uses chunkmanager.array_api.full() for chunked arrays",
      "Explains the non-chunked path uses duck_array_ops.full_like which dispatches via get_array_namespace",
      "Notes that zeros_like is exported via xarray/__init__.py",
      "Mentions at least one internal usage such as in polyval within computation.py"
    ],
    "key_files": [
      "xarray/core/common.py",
      "xarray/core/duck_array_ops.py",
      "xarray/namedarray/parallelcompat.py",
      "xarray/core/dtypes.py",
      "xarray/compat/array_api_compat.py"
    ],
    "source_doc": "[docstring: xarray/core/common.py] xarray.core.common.zeros_like\nReturn a new object of zeros with the same shape and\n    type as a given dataarray or dataset.\n\n    Parameters\n    ----------\n    other : DataArray, Dataset or Variable\n        The reference object. The output will have the same dimensions and coordinates as this object.\n    dtype : dtype, optional\n        dtype of the new array. If omitted, it defaults to other.dtype.\n    chunks : int, \"auto\", tuple of int or mapping of Hashable to int, optional\n        Chunk sizes along each dimension, e.g., ``5``, ``\"auto\"``, ``(5, 5)`` or\n        ``{\"x\": 5, \"y\": 5}``.\n    chunked_array_type: str, optional\n        Which chunked array type to coerce the underlying data array to.\n        Defaults to 'dask' if installed, else whatever is registered via the `ChunkManagerEnetryPoint` system.\n        Experimental API that should not be relied upon.\n    from_array_kwargs: dict, optional\n        Additional keyword arguments passed on to the `ChunkManagerEntrypoint.from_array` method used to create\n        chunked arrays, via whichever chunk manager is specified through the `chunked_array_type` kwarg.\n        For example, with dask as the default chunked array type, this method would pass additional kwargs\n        to :py:func:`dask.array.from_array`. Experimental API that should not be relied upon.\n\n    Returns\n    -------\n    out : DataArray, Dataset or Variable\n        New object of zeros with the same shape and type as other.\n\n    Examples\n    --------\n    >>> x = xr.DataArray(\n    ...     np.arange(6).reshape(2, 3),\n    ...     dims=[\"lat\", \"lon\"],\n    ...     coords={\"lat\": [1, 2], \"lon\": [0, 1, 2]},\n    ... )\n    >>> x\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.zeros_like(x)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0, 0, 0],\n           [0, 0, 0]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    >>> xr.zeros_like(x, dtype=float)\n    <xarray.DataArray (lat: 2, lon: 3)> Size: 48B\n    array([[0., 0., 0.],\n           [0., 0., 0.]])\n    Coordinates:\n      * lat      (lat) int64 16B 1 2\n      * lon      (lon) int64 24B 0 1 2\n\n    See Also\n    --------\n    ones_like\n    full_like",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage from line numbers and a 'Horner's method' description likely from comments; most claims are verifiable from code structure and logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_13",
    "repo": "xarray",
    "question": "What is the architecture of xarray's custom accessor registration and caching system, and how do the pieces fit together?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `register_dataset_accessor` function is part of xarray's extension system defined in `xarray/core/extensions.py`. Here's how the architecture works:\n\n**Core Components:**\n\n1. **`_register_accessor(name, cls)`** - The private factory function that all three public registration functions delegate to. It takes a `name` string and a target class (`Dataset`, `DataArray`, or `DataTree`), returns a decorator that:\n   - Checks if the class already has an attribute with the given name using `hasattr(cls, name)`\n   - If so, issues an `AccessorRegistrationWarning` (a subclass of `Warning`, not `UserWarning`)\n   - Uses `setattr(cls, name, _CachedAccessor(name, accessor))` to install a descriptor on the class\n   - Returns the original accessor class unchanged (important: the decorator does not wrap the class)\n\n2. **`_CachedAccessor`** - A descriptor class (implements `__get__`) that provides caching behavior:\n   - When accessed on the class itself (obj is None), it returns the raw accessor class\n   - When accessed on an instance, it looks up `obj._cache` dictionary (creating it as `{}` if missing via `AttributeError` handling)\n   - On first access, it instantiates the accessor by calling `self._accessor(obj)`, stores it in the cache, and returns it\n   - If the accessor's `__init__` raises `AttributeError`, it's caught and re-raised as `RuntimeError` with the message `\"error initializing {name!r} accessor.\"`\n\n3. **`_cache` in `__slots__`** - All three target classes explicitly declare `_cache` in their `__slots__`:\n   - `Dataset.__slots__` includes `\"_cache\"`\n   - `DataArray.__slots__` includes `\"_cache\"`\n   - `DataTree.__slots__` includes `\"_cache\"`\n\n4. **Contrast with `UncachedAccessor`** - xarray also has `utils.UncachedAccessor` (in `xarray/core/utils.py`). Unlike `_CachedAccessor`, `UncachedAccessor` creates a new accessor instance on every access (calls `self._accessor(obj)` without caching).\n\n5. **Three public registration functions** exist: `register_dataset_accessor` (targets `Dataset`), `register_dataarray_accessor` (targets `DataArray`), and `register_datatree_accessor` (targets `DataTree`). All are exported from `xarray/__init__.py`.\n\n6. **`AttrAccessMixin.__init_subclass__`** enforces that all xarray core subclasses must explicitly define `__slots__`, raising `AttributeError` for internal modules (`cls.__module__.startswith(\"xarray.\")`) and a `FutureWarning` for third-party extensions.\n\n7. **Pickling support** - The tests verify that accessor state is preserved through pickling: if you set `ds.example_accessor.value = \"foo\"`, then pickle and unpickle the dataset, `ds_restored.example_accessor.value` equals `\"foo\"` because the accessor instance is stored in `_cache` which gets pickled with the object.",
    "rubric": [
      "Identifies `_register_accessor(name, cls)` as the private factory function that all three public registration functions (`register_dataset_accessor`, `register_dataarray_accessor`, `register_datatree_accessor`) delegate to",
      "Explains that `_register_accessor` checks for existing attributes with `hasattr(cls, name)` and issues an `AccessorRegistrationWarning` (subclass of `Warning`, not `UserWarning`) if a conflict exists",
      "Explains that `_register_accessor` installs a `_CachedAccessor` descriptor on the target class via `setattr` and returns the original accessor class unchanged",
      "Describes `_CachedAccessor` as a descriptor (implements `__get__`) that returns the raw accessor class when accessed on the class (obj is None) and caches accessor instances in `obj._cache` when accessed on an instance",
      "Explains the `_cache` initialization: if `obj._cache` raises `AttributeError`, it's created as an empty dict; accessor instances are stored and retrieved from this cache dict by name",
      "Mentions that `AttributeError` raised during accessor `__init__` is caught and re-raised as `RuntimeError` with the message about error initializing the accessor",
      "Notes that `Dataset`, `DataArray`, and `DataTree` all declare `_cache` in their `__slots__`",
      "Mentions `UncachedAccessor` in utils.py as a contrast \u2014 it creates a new accessor instance on every access without caching"
    ],
    "key_files": [
      "xarray/core/extensions.py",
      "xarray/core/utils.py",
      "xarray/tests/test_extensions.py",
      "xarray/core/common.py"
    ],
    "source_doc": "[docstring: xarray/core/extensions.py] xarray.core.extensions.register_dataset_accessor\nRegister a custom property on xarray.Dataset objects.\n\n    Parameters\n    ----------\n    name : str\n        Name under which the accessor should be registered. A warning is issued\n        if this name conflicts with a preexisting attribute.\n\n    Examples\n    --------\n    In your library code:\n\n    >>> @xr.register_dataset_accessor(\"geo\")\n    ... class GeoAccessor:\n    ...     def __init__(self, xarray_obj):\n    ...         self._obj = xarray_obj\n    ...\n    ...     @property\n    ...     def center(self):\n    ...         # return the geographic center point of this dataset\n    ...         lon = self._obj.latitude\n    ...         lat = self._obj.longitude\n    ...         return (float(lon.mean()), float(lat.mean()))\n    ...\n    ...     def plot(self):\n    ...         # plot this array's data on a map, e.g., using Cartopy\n    ...         pass\n    ...\n\n    Back in an interactive IPython session:\n\n    >>> ds = xr.Dataset(\n    ...     {\"longitude\": np.linspace(0, 10), \"latitude\": np.linspace(0, 20)}\n    ... )\n    >>> ds.geo.center\n    (10.0, 5.0)\n    >>> ds.geo.plot()  # plots data on a map\n\n    See Also\n    --------\n    register_dataarray_accessor",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation's example shows that `ds.geo.center` returns `(10.0, 5.0)`, where `center` computes `lon = self._obj.latitude` and `lat = self._obj.longitude`. The generated answer does not mention this specific example but doesn't contradict it either.",
      "Most claims in the answer (about `_register_accessor`, `_CachedAccessor`, `UncachedAccessor`, `__slots__`, `AttrAccessMixin.__init_subclass__`, pickling support, specific line numbers, `AccessorRegistrationWarning` details, GH933 reference) go far beyond what the documentation chunk provides. These appear to be plausible code-level details derived from the actual source code, but they cannot be verified from the provided documentation alone.",
      "The answer claims `AccessorRegistrationWarning` is a subclass of `Warning`, not `UserWarning` \u2014 this is not verifiable from the documentation and could be a hallucination about the specific class hierarchy.",
      "The answer's claim about `register_datatree_accessor` is not mentioned in the documentation chunk at all \u2014 the doc only mentions `register_dataarray_accessor` in 'See Also'. This is plausible but unverifiable from the given doc."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage: GH933 reference and a comment about _CachedAccessor usage are documentation-only; all other claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_16",
    "repo": "xarray",
    "question": "How does the Resample class use a proxy dimension to handle time-resampling, and how do its upsampling methods and overrides work?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `Resample` class in `xarray/core/resample.py` extends `GroupBy[T_Xarray]` and provides time-resampling methods. Here's how it works in detail:\n\n**Class Hierarchy:**\n- `Resample` inherits from `GroupBy[T_Xarray]`\n- `DataArrayResample` inherits from `Resample[\"DataArray\"]`, `DataArrayGroupByBase`, and `DataArrayResampleAggregations`\n- `DatasetResample` inherits from `Resample[\"Dataset\"]`, `DatasetGroupByBase`, and `DatasetResampleAggregations`\n\n**Proxy Dimension Mechanism:**\nA proxy dimension constant `RESAMPLE_DIM = \"__resample_dim__\"` is defined in `xarray/groupers.py`. In `TimeResampler._get_index_and_items()`, the `full_index` is renamed to `\"__resample_dim__\"`. The `Resample.__init__` validates that `dim != resample_dim`, raising a `ValueError` if they are equal.\n\n**Up-sampling Methods Implementation:**\n- `pad()`, `backfill()`, and `nearest()` all follow the same pattern: they call `self._drop_coords()` to remove non-dimension coordinates, extract the `grouper.full_index` from the single grouper, and then call `obj.reindex()` with the appropriate method ('pad', 'backfill', or 'nearest').\n- `_interpolate()` calls `obj.interp()` with `assume_sorted=True` and passes `kwargs` (with a default of `bounds_error=False`) as the `kwargs` parameter to the interpolation function.\n- `ffill` is a direct alias for `pad`, and `bfill` is a direct alias for `backfill`.\n\n**`shuffle_to_chunks` Override:**\nThe `Resample` version overrides the parent `GroupBy.shuffle_to_chunks` and calls `.drop_vars(RESAMPLE_DIM)` on the result of `_shuffle_obj(chunks)` to remove the proxy dimension variable.\n\n**`_flox_reduce` Override:**\nThe `Resample._flox_reduce` calls `super()._flox_reduce()` and then renames `RESAMPLE_DIM` back to `self._group_dim`.\n\n**`_first_or_last` Override:**\nFor Dataset results, the override transposes each data variable back to its original dimension order from `self._obj._variables[var].dims`.\n\n**`asfreq()` Implementation:**\nBoth `DataArrayResample.asfreq()` and `DatasetResample.asfreq()` first set `self._obj = self._drop_coords()`, then call `self.mean(None if self._dim is None else [self._dim])`.\n\n**TimeResampler Factorization:**\nThe `TimeResampler.factorize()` (in `xarray/groupers.py`) creates `group_indices` as a tuple of slices derived from `first_items.values.astype(np.int64)` using `itertools.starmap(slice, pairwise(sbins))`, plus a final `slice(sbins[-1], None)`. It renames the full_index to `\"__resample_dim__\"`.",
    "rubric": [
      "Explains the class hierarchy: Resample extends GroupBy[T_Xarray], DataArrayResample/DatasetResample extend Resample plus their respective GroupByBase and Aggregations classes",
      "Explains the RESAMPLE_DIM proxy dimension constant ('__resample_dim__') defined in groupers.py and how TimeResampler._get_index_and_items renames the full_index to it",
      "Explains that __init__ validates dim != resample_dim and raises ValueError if equal",
      "Explains that pad/backfill/nearest call _drop_coords, extract grouper.full_index, then call obj.reindex with the appropriate method",
      "Explains that _interpolate calls obj.interp with assume_sorted=True and passes kwargs (defaulting bounds_error=False)",
      "Explains ffill is an alias for pad, and bfill is an alias for backfill",
      "Explains _flox_reduce override calls super()._flox_reduce then renames RESAMPLE_DIM back to self._group_dim",
      "Explains shuffle_to_chunks override calls _shuffle_obj(chunks).drop_vars(RESAMPLE_DIM)",
      "Explains asfreq sets self._obj = self._drop_coords() then calls self.mean on the resampling dimension"
    ],
    "key_files": [
      "xarray/core/resample.py",
      "xarray/core/groupby.py",
      "xarray/groupers.py",
      "xarray/core/common.py"
    ],
    "source_doc": "[docstring: xarray/core/resample.py] xarray.core.resample.Resample\nxarray.core.resample.Resample:\n  An object that extends the `GroupBy` object with additional logic\n    for handling specialized re-sampling operations.\n\n    You should create a `Resample` object by using the `DataArray.resample` or\n    `Dataset.resample` methods. The dimension along re-sampling\n\n    See Also\n    --------\n    DataArray.resample\n    Dataset.resample\n\nxarray.core.resample.Resample.shuffle_to_chunks:\n  Sort or \"shuffle\" the underlying object.\n\n        \"Shuffle\" means the object is sorted so that all group members occur sequentially,\n        in the same chunk. Multiple groups may occur in the same chunk.\n        This method is particularly useful for chunked arrays (e.g. dask, cubed).\n        particularly when you need to map a function that requires all members of a group\n        to be present in a single chunk. For chunked array types, the order of appearance\n        is not guaranteed, but will depend on the input chunking.\n\n        Parameters\n        ----------\n        chunks : int, tuple of int, \"auto\" or mapping of hashable to int or tuple of int, optional\n            How to adjust chunks along dimensions not present in the array being grouped by.\n\n        Returns\n        -------\n        DataArrayGroupBy or DatasetGroupBy\n\n        Examples\n        --------\n        >>> import dask.array\n        >>> da = xr.DataArray(\n        ...     dims=\"time\",\n        ...     data=dask.array.arange(10, chunks=1),\n        ...     coords={\"time\": xr.date_range(\"2001-01-01\", freq=\"12h\", periods=10)},\n        ...     name=\"a\",\n        ... )\n        >>> shuffled = da.resample(time=\"2D\").shuffle_to_chunks()\n        >>> shuffled\n        <xarray.DataArray 'a' (time: 10)> Size: 80B\n        dask.array<shuffle, shape=(10,), dtype=int64, chunksize=(4,), chunktype=numpy.ndarray>\n        Coordinates:\n          * time     (time) datetime64[ns] 80B 2001-01-01 ... 2001-01-05T12:00:00\n\n        See Also\n        --------\n        dask.dataframe.DataFrame.shuffle\n        dask.array.shuffle\n\nxarray.core.resample.Resample._drop_coords:\n  Drop non-dimension coordinates along the resampled dimension.\n\nxarray.core.resample.Resample.pad:\n  Forward fill new values at up-sampled frequency.\n\n        Parameters\n        ----------\n        tolerance : float | Iterable[float] | str | None, default: None\n            Maximum distance between original and new labels to limit\n            the up-sampling method.\n            Up-sampled data with indices that satisfy the equation\n            ``abs(index[indexer] - target) <= tolerance`` are filled by\n            new values. Data with indices that are outside the given\n            tolerance are filled with ``NaN`` s.\n\n        Returns\n        -------\n        padded : DataArray or Dataset\n\nxarray.core.resample.Resample.backfill:\n  Backward fill new values at up-sampled frequency.\n\n        Parameters\n        ----------\n        tolerance : float | Iterable[float] | str | None, default: None\n            Maximum distance between original and new labels to limit\n            the up-sampling method.\n            Up-sampled data with indices that satisfy the equation\n            ``abs(index[indexer] - target) <= tolerance`` are filled by\n            new values. Data with indices that are outside the given\n            tolerance are filled with ``NaN`` s.\n\n        Returns\n        -------\n        backfilled : DataArray or Dataset\n\nxarray.core.resample.Resample.nearest:\n  Take new values from nearest original coordinate to up-sampled\n        frequency coordinates.\n\n        Parameters\n        ----------\n        tolerance : float | Iterable[float] | str | None, default: None\n            Maximum distance between original and new labels to limit\n            the up-sampling method.\n            Up-sampled data with indices that satisfy the equation\n            ``abs(index[indexer] - target) <= tolerance`` are filled by\n            new values. Data with indices that are outside the given\n            tolerance are filled with ``NaN`` s.\n\n        Returns\n        -------\n        upsampled : DataArray or Dataset\n\nxarray.core.resample.Resample.interpolate:\n  Interpolate up-sampled data using the original data as knots.\n\n        Parameters\n        ----------\n        kind : {\"linear\", \"nearest\", \"zero\", \"slinear\",                 \"quadratic\", \"cubic\", \"polynomial\"}, default: \"linear\"\n            The method used to interpolate. The method should be supported by\n            the scipy interpolator:\n\n            - ``interp1d``: {\"linear\", \"nearest\", \"zero\", \"slinear\",\n              \"quadratic\", \"cubic\", \"polynomial\"}\n            - ``interpn``: {\"linear\", \"nearest\"}\n\n            If ``\"polynomial\"`` is passed, the ``order`` keyword argument must\n            also be provided.\n\n        Returns\n        -------\n        interpolated : DataArray or Dataset\n\n        See Also\n        --------\n        DataArray.interp\n        Dataset.interp\n        scipy.interpolate.interp1d\n\nxarray.core.resample.Resample._interpolate:\n  Apply scipy.interpolate.interp1d along resampling dimension.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer reference specific line numbers, internal implementation details, and code structures (e.g., 'defined in xarray/core/groupby.py at line 580', 'defined in xarray/groupers.py line 64', 'in xarray/core/common.py line 1103-1104') that are not present in the provided documentation and cannot be verified from it.",
      "The class hierarchy details (DataArrayResample, DatasetResample inheriting from specific bases) are not mentioned in the documentation provided.",
      "The RESAMPLE_DIM = '__resample_dim__' mechanism is not described in the documentation.",
      "The claim that 'ffill is a direct alias for pad (line 155), and bfill is a direct alias for backfill (line 182)' is not in the documentation.",
      "The asfreq() implementation details are not in the documentation.",
      "The _flox_reduce override details are not in the documentation.",
      "The _first_or_last override details are not in the documentation.",
      "The TimeResampler.factorize() details are not in the documentation.",
      "However, the claims that ARE verifiable against the documentation are consistent: Resample extends GroupBy, shuffle_to_chunks sorts so group members are sequential, pad/backfill/nearest have tolerance parameters, interpolate supports various methods from scipy."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale explanations and one reference to common.py line numbers were removed or rewritten.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_14",
    "repo": "xarray",
    "question": "Why is xarray's `set_options` designed the way it is \u2014 combining global state mutation, context management, validators, setters, and the LRU cache connection?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `set_options` class in `xarray/core/options.py` is designed as a plain class (not inheriting from any base class) that doubles as both a global option setter and a context manager.\n\n**1. Mutable module-level global dict with snapshot-restore pattern for context management:**\nThe `OPTIONS` dict is a plain mutable module-level dictionary (typed as `T_Options` for type checking). The `set_options.__init__` saves old values in `self.old` before applying updates, and `__exit__` restores them via `_apply_update(self.old)`. This supports nested context managers correctly \u2014 each nesting level saves/restores only the keys it changed. The `__enter__` method deliberately returns `None` (just `return` with no value), meaning the `as` clause in `with set_options(...) as x` yields `None`.\n\n**2. Separation of validation, side-effects, and storage via `_VALIDATORS` and `_SETTERS` dicts:**\nNot all options have validators \u2014 `chunk_manager`, `cmap_divergent`, `cmap_sequential`, and `display_expand_groups` are in `OPTIONS` but lack entries in `_VALIDATORS`. The validation check in `__init__` is guarded by `if k in _VALIDATORS`, allowing unvalidated options through. Only two options have side-effect setters in `_SETTERS`: `file_cache_maxsize` (which directly modifies `FILE_CACHE.maxsize` on the global `LRUCache` instance from `xarray/backends/file_manager.py`) and `enable_cftimeindex` (which emits a `FutureWarning` stating it's a no-op). Critically, `_apply_update` runs setters *before* updating OPTIONS, ensuring side effects fire on both set and restore.\n\n**3. The `LRUCache` connection for `file_cache_maxsize`:**\nThe `_set_file_cache_maxsize` setter lazily imports `FILE_CACHE` from `xarray.backends.file_manager` and sets its `maxsize` property. The `LRUCache` (in `xarray/backends/lru_cache.py`) is thread-safe (uses `threading.RLock`) and its `maxsize` setter calls `_enforce_size_limit(size)` which evicts oldest items (calling `on_evict` callback, which is `lambda k, v: v.close()` for FILE_CACHE) to shrink the cache when the size limit decreases. This means changing `file_cache_maxsize` has an immediate side effect of closing files.\n\n**4. `_get_boolean_with_default` pattern for tri-state options:**\nOptions like `keep_attrs` accept `True`, `False`, or `\"default\"`. The helper `_get_boolean_with_default` resolves these: when the global value is `\"default\"`, it falls back to a caller-supplied `default` parameter. The `_get_keep_attrs` function wraps this for the `keep_attrs` option specifically.\n\n**5. `use_new_combine_kwarg_defaults` uses `CombineKwargDefault` sentinel objects:**\nThe `use_new_combine_kwarg_defaults` option is consumed via `CombineKwargDefault` objects in `xarray/util/deprecation_helpers.py`. These sentinel objects have a `_value` property that reads `OPTIONS[\"use_new_combine_kwarg_defaults\"]` at access time, returning `self._new` if True, `self._old` if False. This enables a gradual deprecation: old defaults work normally, but setting the option to `True` switches to new defaults without changing function signatures.\n\n**6. Internal options exist:**\nThe `OPTIONS` dict includes `display_expand_groups` (default: `\"default\"`), `display_default_indexes` (default: `False`), and `enable_cftimeindex` (default: `True`). `enable_cftimeindex` is explicitly a no-op with a FutureWarning. `display_expand_groups` has no validator in `_VALIDATORS`, meaning any value can be set for it without raising an error.\n\n**7. `get_options()` returns an immutable `Frozen` wrapper:**\nThe `get_options()` function wraps the `OPTIONS` dict in a `FrozenDict` (which is actually a `Frozen` class from `xarray/core/utils.py`, a `Mapping` wrapper that makes the underlying dict read-only). This prevents external code from accidentally mutating the global options, though internally `OPTIONS` remains a plain mutable dict.\n\n**8. `arithmetic_broadcast` and related options:**\nThe `arithmetic_broadcast`, `arithmetic_join`, and `arithmetic_compat` options are defined in `OPTIONS` with defaults `True`, `\"inner\"`, and `\"minimal\"` respectively. `OPTIONS` is imported in `variable.py`, where `arithmetic_broadcast` is used to control whether broadcasting is allowed during arithmetic operations.",
    "rubric": [
      "Explains that OPTIONS is a mutable module-level dict and set_options saves old values in self.old before applying updates, restoring them in __exit__ (snapshot-restore pattern for context management)",
      "Notes that __enter__ returns None (just `return` with no value), so the `as` clause yields None",
      "Explains the separation of concerns: _VALIDATORS for validation, _SETTERS for side effects, and OPTIONS for storage \u2014 and that not all options have validators (e.g., chunk_manager, cmap_divergent, cmap_sequential lack validators)",
      "Describes that _apply_update runs setters before updating OPTIONS, ensuring side effects fire on both set and restore",
      "Explains the file_cache_maxsize setter lazily imports FILE_CACHE and sets its maxsize property, which triggers _enforce_size_limit in the thread-safe LRUCache to evict items (calling on_evict which closes files)",
      "Mentions get_options() returns a FrozenDict wrapper to prevent external mutation while OPTIONS remains mutable internally",
      "Describes the CombineKwargDefault sentinel pattern: its _value property reads OPTIONS['use_new_combine_kwarg_defaults'] at access time to switch between old and new default values for gradual deprecation",
      "Mentions the _get_boolean_with_default pattern for tri-state options like keep_attrs that accept True, False, or 'default'"
    ],
    "key_files": [
      "xarray/core/options.py",
      "xarray/backends/lru_cache.py",
      "xarray/backends/file_manager.py",
      "xarray/util/deprecation_helpers.py",
      "xarray/core/variable.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/core/options.py] xarray.core.options.set_options\nxarray.core.options.set_options:\n  Set options for xarray in a controlled context.\n\n    Parameters\n    ----------\n    arithmetic_broadcast : bool, default: True\n        Whether to perform automatic broadcasting in binary operations.\n    arithmetic_compat: {\"identical\", \"equals\", \"broadcast_equals\", \"no_conflicts\", \"override\", \"minimal\"}, default: \"minimal\"\n        How to compare non-index coordinates of the same name for potential\n        conflicts when performing binary operations. (For the alignment of index\n        coordinates in binary operations, see `arithmetic_join`.)\n\n        - \"identical\": all values, dimensions and attributes of the coordinates\n          must be the same.\n        - \"equals\": all values and dimensions of the coordinates must be the\n          same.\n        - \"broadcast_equals\": all values of the coordinates must be equal after\n          broadcasting to ensure common dimensions.\n        - \"no_conflicts\": only values which are not null in both coordinates\n          must be equal. The returned coordinate then contains the combination\n          of all non-null values.\n        - \"override\": skip comparing and take the coordinates from the first\n          operand.\n        - \"minimal\": drop conflicting coordinates.\n    arithmetic_join : {\"inner\", \"outer\", \"left\", \"right\", \"exact\"}, default: \"inner\"\n        DataArray/Dataset index alignment in binary operations:\n\n        - \"outer\": use the union of object indexes\n        - \"inner\": use the intersection of object indexes\n        - \"left\": use indexes from the first object with each dimension\n        - \"right\": use indexes from the last object with each dimension\n        - \"exact\": instead of aligning, raise `ValueError` when indexes to be\n          aligned are not equal\n    chunk_manager : str, default: \"dask\"\n        Chunk manager to use for chunked array computations when multiple\n        options are installed.\n    cmap_divergent : str or matplotlib.colors.Colormap, default: \"RdBu_r\"\n        Colormap to use for divergent data plots. If string, must be\n        matplotlib built-in colormap. Can also be a Colormap object\n        (e.g. mpl.colormaps[\"magma\"])\n    cmap_sequential : str or matplotlib.colors.Colormap, default: \"viridis\"\n        Colormap to use for nondivergent data plots. If string, must be\n        matplotlib built-in colormap. Can also be a Colormap object\n        (e.g. mpl.colormaps[\"magma\"])\n    display_expand_attrs : {\"default\", True, False}\n        Whether to expand the attributes section for display of\n        ``DataArray`` or ``Dataset`` objects. Can be\n\n        * ``True`` : to always expand attrs\n        * ``False`` : to always collapse attrs\n        * ``default`` : to expand unless over a pre-defined limit\n    display_expand_coords : {\"default\", True, False}\n        Whether to expand the coordinates section for display of\n        ``DataArray`` or ``Dataset`` objects. Can be\n\n        * ``True`` : to always expand coordinates\n        * ``False`` : to always collapse coordinates\n        * ``default`` : to expand unless over a pre-defined limit\n    display_expand_data : {\"default\", True, False}\n        Whether to expand the data section for display of ``DataArray``\n        objects. Can be\n\n        * ``True`` : to always expand data\n        * ``False`` : to always collapse data\n        * ``default`` : to expand unless over a pre-defined limit\n    display_expand_data_vars : {\"default\", True, False}\n        Whether to expand the data variables section for display of\n        ``Dataset`` objects. Can be\n\n        * ``True`` : to always expand data variables\n        * ``False`` : to always collapse data variables\n        * ``default`` : to expand unless over a pre-defined limit\n    display_expand_indexes : {\"default\", True, False}\n        Whether to expand the indexes section for display of\n        ``DataArray`` or ``Dataset``. Can be\n\n        * ``True`` : to always expand indexes\n        * ``False`` : to always collapse indexes\n        * ``default`` : to expand unless over a pre-defined limit (always collapse for html style)\n    display_max_children : int, default: 12\n        Maximum number of children to display for each node in a DataTree.\n    display_max_html_elements : int, default: 300\n        Maximum number of HTML elements to include in DataTree HTML displays.\n        Additional items are truncated.\n    display_max_rows : int, default: 12\n        Maximum display rows.\n    display_max_items : int, default 20\n        Maximum number of items to display for a DataTree before collapsing\n        child nodes, across all levels.\n    display_values_threshold : int, default: 200\n        Total number of array elements which trigger summarization rather\n        than full repr for variable data views (numpy arrays).\n    display_style : {\"text\", \"html\"}, default: \"html\"\n        Display style to use in jupyter for xarray objects.\n    display_width : int, default: 80\n        Maximum display width for ``repr`` on xarray objects.\n    file_cache_maxsize : int, default: 128\n        Maximum number of open files to hold in xarray's\n        global least-recently-usage cached. This should be smaller than\n        your system's per-process file descriptor limit, e.g.,\n        ``ulimit -n`` on Linux.\n    keep_attrs : {\"default\", True, False}\n        Whether to keep attributes on xarray Datasets/dataarrays after\n        operations. Can be\n\n        * ``True`` : to always keep attrs\n        * ``False`` : to always discard attrs\n        * ``default`` : to use original logic that attrs should only\n          be kept in unambiguous circumstances\n    netcdf_engine_order : sequence, default ['netcdf4', 'h5netcdf', 'scipy']\n        Preference order of backend engines to use when reading or writing\n        netCDF files with ``open_dataset()`` and ``to_netcdf()`` if ``engine``\n        is not explicitly specified. May be any permutation or subset of\n        ``['netcdf4', 'h5netcdf', 'scipy']``.\n    use_bottleneck : bool, default: True\n        Whether to use ``bottleneck`` to accelerate 1D reductions and\n        1D rolling reduction operations.\n    use_flox : bool, default: True\n        Whether to use ``numpy_groupies`` and `flox`` to\n        accelerate groupby and resampling reductions.\n    use_new_combine_kwarg_defaults : bool, default False\n        Whether to use new kwarg default values for combine functions:\n        :py:func:`~xarray.concat`, :py:func:`~xarray.merge`,\n        :py:func:`~xarray.open_mfdataset`. New values are:\n\n        * ``data_vars``: None\n        * ``coords``: \"minimal\"\n        * ``compat``: \"override\"\n        * ``join``: \"exact\"\n    use_numbagg : bool, default: True\n        Whether to use ``numbagg`` to accelerate reductions.\n        Takes precedence over ``use_bottleneck`` when both are True.\n    use_opt_einsum : bool, default: True\n        Whether to use ``opt_einsum`` to accelerate dot products.\n    warn_for_unclosed_files : bool, default: False\n        Whether or not to issue a warning when unclosed files are\n        deallocated. This is mostly useful for debugging.\n\n    Examples\n    --------\n    It is possible to use ``set_options`` either as a context manager:\n\n    >>> ds = xr.Dataset({\"x\": np.arange(1000)})\n    >>> with xr.set_options(display_width=40):\n    ...     print(ds)\n    ...\n    <xarray.Dataset> Size: 8kB\n    Dimensions:  (x: 1000)\n    Coordinates:\n      * x        (x) int64 8kB 0 1 ... 999\n    Data variables:\n        *empty*\n\n    Or to set global options:\n\n    >>> xr.set_options(display_width=80)  # doctest: +ELLIPSIS\n    <xarray.core.options.set_options object at 0x...>",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims `keep_attrs` default behavior (when set to 'default') with `_get_keep_attrs(default=True)` is 'to keep attributes', but the documentation states the default logic is 'attrs should only be kept in unambiguous circumstances' - the claim about always using `default=True` in the codebase is an implementation detail that cannot be verified from this documentation alone.",
      "The answer claims `arithmetic_compat` default is 'minimal' which matches the documentation, but the detailed description of how it's used in specific files (dataarray.py:4900-4911, dataset.py:7723, 7793) cannot be verified from this documentation.",
      "The answer states `__enter__` returns `None` (just `return` with no value) - this is a code-level implementation detail not verifiable from the documentation.",
      "The answer mentions `display_expand_groups`, `display_default_indexes`, and `enable_cftimeindex` as undocumented options - these cannot be verified from the provided documentation but are plausible internal implementation details.",
      "The answer states specific line numbers (e.g., line 78, line 3057) which cannot be verified from the documentation alone.",
      "The answer claims `_apply_update` runs setters *before* updating OPTIONS - this is an implementation detail not verifiable from this documentation.",
      "The answer discusses `CombineKwargDefault` sentinel objects in `xarray/util/deprecation_helpers.py` - this is beyond the scope of the documentation but is a plausible implementation detail."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor adjustments needed for claims about docstring content, specific line numbers in truncated files, and usage across unshown files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_15",
    "repo": "xarray",
    "question": "Where does data flow through `map_blocks` \u2014 from its entry points to dask graph construction and result assembly?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `map_blocks` function in `xarray/core/parallel.py` orchestrates a complex data flow that connects multiple internal systems:\n\n**Call Chain (Callers \u2192 map_blocks \u2192 Callees):**\n\n1. **Entry points**: Both `DataArray.map_blocks` and `Dataset.map_blocks` are thin wrappers that call `xarray.core.parallel.map_blocks(func, self, args, kwargs, template)`. The function is also exported as `xr.map_blocks` via `__init__.py` (`from xarray.core.parallel import map_blocks`).\n\n2. **Early exit for non-dask data**: If `obj` is not a dask collection, `map_blocks` short-circuits and returns `func(obj, *args, **kwargs)` directly without any dask graph construction.\n\n3. **DataArray-to-Dataset conversion** (`dataarray_to_dataset`): All DataArrays are internally converted to Datasets. If the DataArray has `name is None`, it uses `_to_temp_dataset()` which names the variable `_THIS_ARRAY` (a `ReprObject('<this-array>')` defined in `dataarray.py`). If the DataArray has a name, it uses `to_dataset()` instead.\n\n4. **Template inference** (`infer_template`): When no template is provided, `make_meta` creates mock objects using `dask.array.utils.meta_from_array` for each variable, preserving dims and attrs. The function is then run on these mocked objects to infer return type, dtype, variable names, and new indexes.\n\n5. **Alignment**: All xarray objects (obj + args) are aligned using `align(*xarray_objs, join='exact')`. After alignment, objects are rechunked via `arg.chunk(arg.chunksizes)`.\n\n6. **Dask graph construction**: The function builds a `dask.highlevelgraph.HighLevelGraph` with multiple layers:\n   - A main graph layer (named `{funcname}-{token}`) containing `_wrapper` tasks for each chunk combination (produced by `itertools.product` over all chunked dimensions)\n   - Per-variable extraction layers (named `{varname}-{gname}`) that use `operator.getitem` to extract individual variable data from the wrapper's dict return value\n   - `subset_dataset_to_block` creates dask task tuples that reconstruct a `Dataset` from its constituent chunk tasks. For dask-backed variables, it references the underlying dask array's chunk key directly. For non-dask variables (dimension coordinates), it uses `variable.isel()` with slice bounds derived from `np.cumsum` of chunk sizes, and deduplicates graph entries (checking `if variable.ndim == 0 or chunk_variable_task not in graph`).\n\n7. **The `_wrapper` inner function**: This is the function placed in the dask graph for execution. It: (a) converts Dataset args back to DataArrays where needed, (b) calls the user function, (c) validates that returned dimensions, shapes, indexes, coordinate variables, and data variables match the `expected` dict, and (d) returns `make_dict(result)` \u2014 a plain dict mapping variable names to their data arrays (`{k: v.data for k, v in x.variables.items()}`).\n\n8. **Result assembly**: A new Dataset is constructed with the template's coordinates, attrs, and encoding. Each computed variable gets a `dask.array.Array` constructed from the HighLevelGraph with appropriate chunks and dtype from the template. If the original input was a DataArray, the result Dataset is converted back via `dataset_to_dataarray` and the original name is restored.\n\n9. **Computed variables**: `computed_variables = set(template.variables) - set(coordinates.indexes)` determines which variables need dask array construction (excluding index coordinates).\n\n10. **Chunk compatibility check**: `assert_chunks_compatible` calls `unify_chunks()` on both datasets before comparing chunk tuples along shared dimensions.",
    "rubric": [
      "Identifies that DataArray.map_blocks and Dataset.map_blocks are thin wrappers that delegate to xarray.core.parallel.map_blocks",
      "Explains the early exit path: if obj is not a dask collection, func(obj, *args, **kwargs) is returned directly",
      "Describes DataArray-to-Dataset conversion via dataarray_to_dataset (using _to_temp_dataset for unnamed arrays, to_dataset for named ones)",
      "Explains template inference via infer_template and make_meta when no template is provided",
      "Mentions alignment of xarray objects using align(*xarray_objs, join='exact') and subsequent rechunking",
      "Describes dask HighLevelGraph construction with the main _wrapper task layer and per-variable getitem layers",
      "Explains subset_dataset_to_block's role in creating tasks that subset datasets to blocks (referencing dask chunk keys for dask variables, using variable.isel for non-dask variables)",
      "Describes the _wrapper inner function: converts datasets back to dataarrays, calls user func, validates result, returns make_dict output",
      "Explains result assembly: Dataset constructed with template coords/attrs, dask arrays created from the HighLevelGraph, and DataArray conversion if input was DataArray"
    ],
    "key_files": [
      "xarray/core/parallel.py",
      "xarray/core/dataarray.py",
      "xarray/core/dataset.py",
      "xarray/__init__.py"
    ],
    "source_doc": "[docstring: xarray/core/parallel.py] xarray.core.parallel.map_blocks\nApply a function to each block of a DataArray or Dataset.\n\n    .. warning::\n        This function is experimental and its signature may change.\n\n    Parameters\n    ----------\n    func : callable\n        User-provided function that accepts a DataArray or Dataset as its first\n        parameter ``obj``. The function will receive a subset or 'block' of ``obj`` (see below),\n        corresponding to one chunk along each chunked dimension. ``func`` will be\n        executed as ``func(subset_obj, *subset_args, **kwargs)``.\n\n        This function must return either a single DataArray or a single Dataset.\n\n        This function cannot add a new chunked dimension.\n    obj : DataArray, Dataset\n        Passed to the function as its first argument, one block at a time.\n    args : sequence\n        Passed to func after unpacking and subsetting any xarray objects by blocks.\n        xarray objects in args must be aligned with obj, otherwise an error is raised.\n    kwargs : mapping\n        Passed verbatim to func after unpacking. xarray objects, if any, will not be\n        subset to blocks. Passing dask collections in kwargs is not allowed.\n    template : DataArray or Dataset, optional\n        xarray object representing the final result after compute is called. If not provided,\n        the function will be first run on mocked-up data, that looks like ``obj`` but\n        has sizes 0, to determine properties of the returned object such as dtype,\n        variable names, attributes, new dimensions and new indexes (if any).\n        ``template`` must be provided if the function changes the size of existing dimensions.\n        When provided, ``attrs`` on variables in `template` are copied over to the result. Any\n        ``attrs`` set by ``func`` will be ignored.\n\n    Returns\n    -------\n    obj : same as obj\n        A single DataArray or Dataset with dask backend, reassembled from the outputs of the\n        function.\n\n    Notes\n    -----\n    This function is designed for when ``func`` needs to manipulate a whole xarray object\n    subset to each block. Each block is loaded into memory. In the more common case where\n    ``func`` can work on numpy arrays, it is recommended to use ``apply_ufunc``.\n\n    If none of the variables in ``obj`` is backed by dask arrays, calling this function is\n    equivalent to calling ``func(obj, *args, **kwargs)``.\n\n    See Also\n    --------\n    dask.array.map_blocks, xarray.apply_ufunc, xarray.Dataset.map_blocks\n    xarray.DataArray.map_blocks\n\n    Examples\n    --------\n    Calculate an anomaly from climatology using ``.groupby()``. Using\n    ``xr.map_blocks()`` allows for parallel operations with knowledge of ``xarray``,\n    its indices, and its methods like ``.groupby()``.\n\n    >>> def calculate_anomaly(da, groupby_type=\"time.month\"):\n    ...     gb = da.groupby(groupby_type)\n    ...     clim = gb.mean(dim=\"time\")\n    ...     return gb - clim\n    ...\n    >>> time = xr.date_range(\"1990-01\", \"1992-01\", freq=\"ME\", use_cftime=True)\n    >>> month = xr.DataArray(time.month, coords={\"time\": time}, dims=[\"time\"])\n    >>> np.random.seed(123)\n    >>> array = xr.DataArray(\n    ...     np.random.rand(len(time)),\n    ...     dims=[\"time\"],\n    ...     coords={\"time\": time, \"month\": month},\n    ... ).chunk()\n    >>> array.map_blocks(calculate_anomaly, template=array).compute()\n    <xarray.DataArray (time: 24)> Size: 192B\n    array([ 0.12894847,  0.11323072, -0.0855964 , -0.09334032,  0.26848862,\n            0.12382735,  0.22460641,  0.07650108, -0.07673453, -0.22865714,\n           -0.19063865,  0.0590131 , -0.12894847, -0.11323072,  0.0855964 ,\n            0.09334032, -0.26848862, -0.12382735, -0.22460641, -0.07650108,\n            0.07673453,  0.22865714,  0.19063865, -0.0590131 ])\n    Coordinates:\n      * time     (time) object 192B 1990-01-31 00:00:00 ... 1991-12-31 00:00:00\n        month    (time) int64 192B 1 2 3 4 5 6 7 8 9 10 ... 3 4 5 6 7 8 9 10 11 12\n\n    Note that one must explicitly use ``args=[]`` and ``kwargs={}`` to pass arguments\n    to the function being applied in ``xr.map_blocks()``:\n\n    >>> array.map_blocks(\n    ...     calculate_anomaly,\n    ...     kwargs={\"groupby_type\": \"time.year\"},\n    ...     template=array,\n    ... )  # doctest: +ELLIPSIS\n    <xarray.DataArray (time: 24)> Size: 192B\n    dask.array<<this-array>-calculate_anomaly, shape=(24,), dtype=float64, chunksize=(24,), chunktype=numpy.ndarray>\n    Coordinates:\n      * time     (time) object 192B 1990-01-31 00:00:00 ... 1991-12-31 00:00:00\n        month    (time) int64 192B dask.array<chunksize=(24,), meta=np.ndarray>",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for line numbers and a few rationale-based explanations.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_17",
    "repo": "xarray",
    "question": "What does the `did_you_mean` utility function do, and how does it handle non-string hashable values like `None` or tuples?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `did_you_mean` function in `xarray/core/utils.py` is a string similarity suggestion utility that uses Python's `difflib.get_close_matches` to find close matches between a user-provided word and a list of valid possibilities. Here's what the implementation reveals:\n\n**Algorithm and Implementation Details:**\n1. It converts all hashable possibilities to strings using `str(k)` and stores them in a `dict[str, Hashable]` mapping (`possibilities_str`).\n2. It uses a walrus operator (`:=`) inside a `len()` check to both compute and test the result of `difflib.get_close_matches` in a single expression.\n3. It passes `n=n` (default 10) to `get_close_matches` but does NOT pass a `cutoff` parameter, meaning Python's default cutoff of 0.6 (from `difflib.SequenceMatcher` ratio) is used for filtering.\n4. After finding matches as strings, it maps them back to their original hashable values using the dictionary, preserving the original type (e.g., `None` stays as `None`, not `'None'`).\n5. The function returns an empty string `\"\"` when no close matches are found, which is falsy \u2014 enabling boolean truthiness checks by callers.\n\n**Usage in the Codebase:**\nThe function is likely called in `Dataset.__getitem__` (in dataset.py) to suggest similar variable names when a non-existent key is accessed, though the specific usage details cannot be fully confirmed from the truncated code.\n\n**Potential Edge Case:**\nSince `possibilities_str` is built as `{str(k): k for k in possibilities}`, if two possibilities have the same string representation (e.g., the string `'None'` and `None`), only the last one in iteration order will be preserved in the dictionary, potentially losing one of the possibilities.",
    "rubric": [
      "Explains that it uses difflib.get_close_matches to find similar words from a list of possibilities",
      "Mentions that all hashable possibilities are converted to strings via str(k) and stored in a dict mapping back to original values",
      "Notes that matches are mapped back to their original hashable types (e.g., None stays as None, not 'None')",
      "Mentions the default n=10 parameter for max suggestions and that no custom cutoff is passed (so difflib's default 0.6 is used)",
      "Explains that it returns an empty string when no matches are found, enabling falsy/truthy checks by callers",
      "Identifies the potential edge case where two possibilities with the same string representation (e.g., 'None' and None) would collide in the dict",
      "Notes that it is used in Dataset.__getitem__ to suggest similar variable names when a key is not found"
    ],
    "key_files": [
      "xarray/core/utils.py",
      "xarray/core/dataset.py",
      "xarray/core/formatting.py"
    ],
    "source_doc": "[docstring: xarray/core/utils.py] xarray.core.utils.did_you_mean\nSuggest a few correct words based on a list of possibilities\n\n    Parameters\n    ----------\n    word : Hashable\n        Word to compare to a list of possibilities.\n    possibilities : Iterable of Hashable\n        The iterable of Hashable that contains the correct values.\n    n : int, default: 10\n        Maximum number of suggestions to show.\n\n    Examples\n    --------\n    >>> did_you_mean(\"bluch\", (\"blech\", \"gray_r\", 1, None, (2, 56)))\n    \"Did you mean one of ('blech',)?\"\n    >>> did_you_mean(\"none\", (\"blech\", \"gray_r\", 1, None, (2, 56)))\n    'Did you mean one of (None,)?'\n\n    See Also\n    --------\n    https://en.wikipedia.org/wiki/String_metric",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for truncated dataset.py usage details and one rationale statement.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_18",
    "repo": "xarray",
    "question": "Why does `maybe_promote` promote integer dtypes to float, and how does it decide which float size to use?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `maybe_promote` function in `xarray/core/dtypes.py` determines, for a given dtype, the appropriate promoted dtype that can hold missing values (NaN/NaT) and the corresponding fill value. NumPy integer and other non-floating dtypes cannot natively represent missing values, so the function handles this by promoting dtypes as needed.\n\n**How it works (from the code):**\n\n1. **Integer types are promoted to float**: When xarray needs to introduce missing values into integer arrays (e.g., during `shift`, `pad`, `rolling`, or `unstack` operations), it must promote the dtype to a floating-point type first. The function promotes integers with `itemsize <= 2` (e.g., int8/int16/uint8/uint16) to float32, and larger integers (int32/int64/uint32/uint64) to float64 (`np.float32 if dtype.itemsize <= 2 else np.float64`).\n\n2. **Special handling of timedelta64 before integer check**: The function checks for `np.timedelta64` *before* checking for integers (`isdtype(dtype, \"integral\")`), which is important because timedelta64 could otherwise match the integer check. This ordering ensures timedelta arrays are not incorrectly promoted to float.\n\n3. **Support for pandas ExtensionDtype**: The function first checks `utils.is_allowed_extension_array_dtype(dtype)`, and if true, returns the dtype's own `na_value` (e.g., `pd.NA` for nullable integer types), avoiding unnecessary promotion.\n\n4. **Typed fill values**: At the end, the function casts the fill_value with `dtype_out.type(fill_value)` to ensure the fill value is of the exact scalar type matching the promoted dtype, not a generic Python float or similar.\n\n**Where it's used in the system:**\n- `Variable._shift_one_dim`: To fill shifted positions with NaN after promoting integer arrays\n- `Variable.pad`: When padding with constant values in mode 'constant'\n- `Variable._unstack_once`: When unstacking introduces missing values\n- `Variable.rolling_window`: To support NaN fill in rolling windows\n- `NamedArray._to_sparse`: To determine fill_value for sparse arrays\n- `dask_rolling_wrapper`: To determine output dtype for dask rolling operations\n- `nanops._maybe_null_out`: To null out results where min_count is not satisfied\n- `CFMaskCoder.decode`: To determine fill value when decoding masked NetCDF data\n\nThe `NA` sentinel (a `ReprObject(\"<NA>\")` instance) is used as a default parameter in these methods, signaling that `maybe_promote` should be called to determine the appropriate fill value automatically.",
    "rubric": [
      "Explains that integer types cannot natively represent NaN/missing values, so promotion to float is necessary",
      "States that integers with itemsize <= 2 (int8/int16/uint8/uint16) are promoted to float32",
      "States that larger integers (int32/int64/uint32/uint64) are promoted to float64",
      "Mentions that timedelta64 is checked before the integer check to avoid incorrect promotion to float",
      "Mentions that pandas ExtensionDtype types are handled separately, returning their own na_value without promotion",
      "Notes that the fill_value is cast to the promoted dtype's scalar type at the end",
      "Identifies at least two concrete usage sites where maybe_promote is called (e.g., shift, pad, unstack, rolling_window, sparse, nanops, or CFMaskCoder)"
    ],
    "key_files": [
      "xarray/core/dtypes.py",
      "xarray/core/variable.py",
      "xarray/namedarray/dtypes.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/core/dtypes.py] xarray.core.dtypes.maybe_promote\nSimpler equivalent of pandas.core.common._maybe_promote\n\n    Parameters\n    ----------\n    dtype : np.dtype\n\n    Returns\n    -------\n    dtype : Promoted dtype that can hold missing values.\n    fill_value : Valid missing value for the promoted dtype.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (specific promotion rules like int8\u2192float32, timedelta64 handling, ExtensionDtype support, specific usage locations, etc.) that are not present in the provided documentation. While these are plausible extrapolations based on likely source code knowledge, they cannot be verified from the given docstring alone.",
      "The claim about promoting int8/int16/uint8/uint16 to float32 and int32/int64/uint32/uint64 to float64 cannot be verified from the docstring.",
      "The specific list of where the function is used in the system (Variable._shift_one_dim, Variable.pad, etc.) cannot be verified from the docstring.",
      "The claim about timedelta64 being a subclass of np.integer and the referenced NumPy issue cannot be verified from the docstring.",
      "The claim about `utils.is_allowed_extension_array_dtype` handling cannot be verified from the docstring."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Removed claims about pandas comparison rationale and comment references; rewrote claims mixing code logic with doc-only rationale.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_19",
    "repo": "xarray",
    "question": "Where is the SeasonGrouper class implemented and how does it handle overlapping seasons differently from SeasonResampler?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `SeasonGrouper` class is located in `xarray/groupers.py` and inherits from the abstract base class `Grouper` (which uses `ABC`). Unlike its sibling `SeasonResampler` (which inherits from `Resampler`, itself a subclass of `Grouper`), `SeasonGrouper` does not restrict overlapping seasons.\n\nKey implementation details:\n\n1. **Location in call chain**: When `da.groupby(time=SeasonGrouper(...))` is called, `DataArray.groupby()` calls `_parse_group_and_groupers()` which creates a `ResolvedGrouper` object. The `ResolvedGrouper.__post_init__()` method calls `self.grouper.factorize(self.group)` (as `self.encoded = self.grouper.factorize(self.group)`), which invokes `SeasonGrouper.factorize()`.\n\n2. **Overlapping season handling via `find_independent_seasons()`**: When seasons overlap, `find_independent_seasons()` splits them into multiple `SeasonsGroup` objects containing non-overlapping subsets.\n\n3. **Multi-dimensional codes with `__season_dim__`**: When `len(seasons_groups) > 1` (overlapping seasons exist), `SeasonGrouper.factorize()` creates a codes array with an extra leading dimension called `\"__season_dim__\"`. The codes array dtype is `np.int8` and is initialized to `-1` (sentinel for unassigned).\n\n4. **Month resolution via `season_to_month_tuple()`**: This helper builds a `starts` dictionary from 2-character sliding windows over `\"JFMAMJJASONDJ\"` mapping to months 1-12. E.g., `\"DJ\"` maps to 12, `\"JF\"` maps to 1. For single-letter seasons, the suffix is taken from the next season's first character.\n\n5. **Key difference from `SeasonResampler`**: `SeasonResampler` explicitly raises a `ValueError` in `__post_init__` if seasons overlap (`\"Overlapping seasons are not allowed\"`), while `SeasonGrouper` has no `__post_init__` and performs no such check.\n\n6. **The output codes DataArray is named `\"season\"`** (hardcoded), and the `unique_coord` is a `Variable` with dimension name `\"season\"`.\n\n7. **The `reset()` method** simply creates a new instance with the same seasons: `type(self)(self.seasons)`.",
    "rubric": [
      "SeasonGrouper is located in xarray/groupers.py and inherits from the Grouper ABC",
      "SeasonResampler raises a ValueError if seasons overlap, while SeasonGrouper does not restrict overlapping seasons",
      "SeasonGrouper.factorize() uses find_independent_seasons() to split overlapping seasons into multiple SeasonsGroup objects",
      "When seasons overlap (len(seasons_groups) > 1), factorize() creates a codes array with an extra leading dimension called '__season_dim__'",
      "The codes array uses np.int8 dtype and is initialized to -1 as a sentinel value",
      "season_to_month_tuple() resolves month numbers using 2-character sliding windows over 'JFMAMJJASONDJ'",
      "The output codes DataArray is named 'season' and the unique_coord is a Variable with dimension name 'season'"
    ],
    "key_files": [
      "xarray/groupers.py",
      "xarray/core/groupby.py",
      "xarray/compat/toolzcompat.py"
    ],
    "source_doc": "[docstring: xarray/groupers.py] xarray.groupers.SeasonGrouper\nxarray.groupers.SeasonGrouper:\n  Allows grouping using a custom definition of seasons.\n\n    Parameters\n    ----------\n    seasons: sequence of str\n        List of strings representing seasons. E.g. ``\"JF\"`` or ``\"JJA\"`` etc.\n        Overlapping seasons are allowed (e.g. ``[\"DJFM\", \"MAMJ\", \"JJAS\", \"SOND\"]``)\n\n    Examples\n    --------\n    >>> SeasonGrouper([\"JF\", \"MAM\", \"JJAS\", \"OND\"])\n    SeasonGrouper(seasons=['JF', 'MAM', 'JJAS', 'OND'])\n\n    The ordering is preserved\n\n    >>> SeasonGrouper([\"MAM\", \"JJAS\", \"OND\", \"JF\"])\n    SeasonGrouper(seasons=['MAM', 'JJAS', 'OND', 'JF'])\n\n    Overlapping seasons are allowed\n\n    >>> SeasonGrouper([\"DJFM\", \"MAMJ\", \"JJAS\", \"SOND\"])\n    SeasonGrouper(seasons=['DJFM', 'MAMJ', 'JJAS', 'SOND'])",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code structure and logic; removed comment-dependent claim about drop_incomplete and a documentation example.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_20",
    "repo": "xarray",
    "question": "How does `SeasonResampler.compute_chunks` determine chunk sizes, and how does it differ from `TimeResampler.compute_chunks`?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `SeasonResampler.compute_chunks` method implements the following algorithm to determine chunk sizes for a datetime variable when resampling by custom seasons:\n\n1. **Validation**: First checks that the variable contains datetime-like objects (via `_contains_datetime_like_objects`). If not, raises a `ValueError`.\n\n2. **Coverage check**: Validates that all 12 months are covered by checking `len(\"\".join(self.seasons)) != 12`. If the seasons don't cover all 12 months, it raises a `ValueError` with message \"Cannot rechunk with a SeasonResampler that does not cover all 12 months.\"\n\n3. **Ignoring `drop_incomplete`**: Creates a *temporary* `SeasonResampler` with `drop_incomplete=False` for chunking purposes.\n\n4. **Counting elements per season**: Creates a `DataArray` of ones with the same shape as the variable, attaches the variable as a coordinate on the given dimension, then calls `.resample({dim: resampler_for_chunking}).sum()`.\n\n5. **Handling NaN periods**: If the result has float dtype, it drops NaN values along the dimension and casts to int.\n\n6. **Returns**: Converts the resulting DataArray to a tuple of integers representing chunk sizes.\n\nThe method is called from `Dataset.chunk()` via the `_resolve_resampler` helper (in `xarray/core/dataset.py`), which additionally validates that the variable is 1D and that `sum(newchunks) == variable.shape[0]`.\n\nKey differences from `TimeResampler.compute_chunks`: The `TimeResampler` version uses `self` directly for resampling, while `SeasonResampler` creates a new instance with `drop_incomplete=False`. Also, `SeasonResampler` adds the 12-month coverage validation that `TimeResampler` does not have.\n\nThe `SeasonResampler.__post_init__` validates that seasons don't overlap (using `functools.reduce(operator.add, ...)` to concatenate all month indices and checking for duplicates) and that they are sorted periodically (using `is_sorted_periodic`).",
    "rubric": [
      "Mentions validation that the variable contains datetime-like objects, raising ValueError if not",
      "Mentions the 12-month coverage check (all months must be covered by seasons) with appropriate error message",
      "Explains that a temporary SeasonResampler with drop_incomplete=False is created for chunking, unlike TimeResampler which uses self directly",
      "Describes the core algorithm: creating a DataArray of ones, attaching the variable as a coordinate, then calling .resample().sum() to count elements per season",
      "Mentions handling of NaN values: checking for float dtype, dropping NaNs, and casting to int",
      "Mentions that the result is converted to a tuple of integers",
      "Notes how _resolve_resampler in Dataset.chunk() additionally validates the variable is 1D and that chunk sizes sum to the variable's shape"
    ],
    "key_files": [
      "xarray/groupers.py",
      "xarray/core/dataset.py",
      "xarray/core/common.py"
    ],
    "source_doc": "[docstring: xarray/groupers.py] xarray.groupers.SeasonResampler\nxarray.groupers.SeasonResampler:\n  Allows grouping using a custom definition of seasons.\n\n    Parameters\n    ----------\n    seasons: Sequence[str]\n        An ordered list of seasons.\n    drop_incomplete: bool\n        Whether to drop seasons that are not completely included in the data.\n        For example, if a time series starts in Jan-2001, and seasons includes `\"DJF\"`\n        then observations from Jan-2001, and Feb-2001 are ignored in the grouping\n        since Dec-2000 isn't present.\n\n    Examples\n    --------\n    >>> SeasonResampler([\"JF\", \"MAM\", \"JJAS\", \"OND\"])\n    SeasonResampler(seasons=['JF', 'MAM', 'JJAS', 'OND'], drop_incomplete=True)\n\n    >>> SeasonResampler([\"DJFM\", \"AM\", \"JJA\", \"SON\"])\n    SeasonResampler(seasons=['DJFM', 'AM', 'JJA', 'SON'], drop_incomplete=True)\n\nxarray.groupers.SeasonResampler.compute_chunks:\n  Compute chunk sizes for this season resampler.\n\n        This method is used during chunking operations to determine appropriate\n        chunk sizes for the given variable when using this resampler.\n\n        Parameters\n        ----------\n        name : Hashable\n            The name of the dimension being chunked.\n        variable : Variable\n            The variable being chunked.\n\n        Returns\n        -------\n        tuple[int, ...]\n            A tuple of chunk sizes for the dimension.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the `compute_chunks` method's parameters (`name: Hashable`, `variable: Variable`) and return type (`tuple[int, ...]`), plus a brief description. The generated answer provides extensive implementation details (validation steps, coverage checks, temporary resampler creation, NaN handling, `__post_init__` validation, comparisons with `TimeResampler`, references to `_resolve_resampler` helper) that are not present in the documentation and appear to be derived from reading the actual source code rather than the provided documentation.",
      "The claim about `_contains_datetime_like_objects` validation is not supported by the documentation \u2014 it's plausible but unverifiable from the given doc.",
      "The claim about 12-month coverage check with specific error message is not in the documentation.",
      "The claim about creating a temporary SeasonResampler with `drop_incomplete=False` is not in the documentation.",
      "The claim about `_resolve_resampler` helper in `xarray/core/dataset.py` is not in the documentation.",
      "The claims about `__post_init__` validation (overlap checking, `is_sorted_periodic`) are not in the documentation.",
      "None of these claims directly contradict the documentation \u2014 they are plausible implementation details but cannot be verified from the provided doc alone."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code; minor rationale/explanation phrases from documentation were trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_21",
    "repo": "xarray",
    "question": "What is the architecture of RangeIndex and its associated coordinate transform, including their class hierarchy, key methods, and how lazy coordinate computation works?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `RangeIndex` class in `xarray/indexes/range_index.py` is an xarray index for 1D evenly-spaced floating-point coordinate ranges. Here are the key architectural and implementation details discovered from the source code:\n\n**Class Hierarchy:**\n- `RangeIndex` extends `CoordinateTransformIndex` (defined in `xarray/core/indexes.py`), which extends the base `Index` class (also in `xarray/core/indexes.py`).\n- `RangeIndex` wraps a `RangeCoordinateTransform` object, which extends `CoordinateTransform` (defined in `xarray/core/coordinate_transform.py`).\n\n**RangeCoordinateTransform internals:**\n- Uses `__slots__ = (\"_step\", \"start\", \"stop\")`.\n- The `_step` field is initialized to `None` and lazily computed via a property: `step = (self.stop - self.start) / self.size` when `size > 0`, defaulting to `1.0` for empty arrays.\n- The `forward()` method computes coordinate labels as `start + positions * step`.\n- The `reverse()` method computes positions as `(labels - start) / step`.\n- The `slice()` method creates a new transform by mapping through Python's `range(self.size)[sl]`, using `new_range.start` and `new_range.stop` to compute new start/stop values. For empty slices (size=0), it explicitly preserves the parent's step via `result._step = self.step`.\n- The `equals()` method on the transform compares `start`, `stop`, and `size`. When `exact=False` (default), it uses `np.isclose()` for start/stop comparison. When `exact=True`, it uses strict `==` comparison.\n\n**Key implementation details in RangeIndex:**\n- `arange()` computes size using `math.ceil((stop - start) / step)` (from Python's `math` module).\n- `linspace()` with `endpoint=True` adjusts stop by computing `stop += (stop - start) / (num - 1)` before creating the transform, so the internal stop is always exclusive.\n- `from_variables()` raises `NotImplementedError` with a message directing users to use `arange()` or `linspace()` instead, meaning `set_xindex()` cannot be used with RangeIndex.\n- `isel()` overrides the parent (which simply returns `None`): slices return a new `RangeIndex` via `self.transform.slice(idxer)`, scalars and multi-dimensional Variables return `None` (dropping the index), and 1D array/Variable indexers materialize values via `self.transform.forward()` and return a `PandasIndex` wrapping a `pd.Index`.\n- `sel()` requires `method='nearest'` (raises `ValueError` otherwise) and does not yet support `tolerance` (raises `ValueError`). For step-less slices, it uses `reverse()` to convert labels to positions, rounds them, and clips to `[0, self.size]`. For slices with a step, it converts to `np.arange(label.start, label.stop, label.step)` for array indexing. For scalar/array labels, it wraps them in `Variable` objects and delegates to `super().sel()` (i.e., `CoordinateTransformIndex.sel()`), which uses the transform's `reverse()` method and rounds the positions.\n- `to_pandas_index()` materializes coordinates by calling `self.transform.generate_coords()`, which is inherited from `CoordinateTransform` and uses `np.meshgrid` with `np.arange` over the dimension size, then calls `forward()`.\n- The lazy coordinate data is stored via `CoordinateTransformIndexingAdapter` (in `xarray/core/indexing.py`), which computes values on demand through the transform's `forward()` method.\n\n**Imports and dependencies:**\n- The module imports `math`, `numpy`, `pandas`, `duck_array_ops`, `CoordinateTransform`, `DataArray`, `CoordinateTransformIndex`, `Index`, `PandasIndex`, `IndexSelResult`, and `Variable`.\n- `CoordinateTransformIndexingAdapter` is imported in `xarray/core/indexes.py` and is used for creating lazy coordinate variables.\n- `copy` is imported in `xarray/core/indexes.py` and is used in the `Index._copy()` method via `copy.deepcopy`.",
    "rubric": [
      "RangeIndex extends CoordinateTransformIndex, which extends the base Index class",
      "RangeIndex wraps a RangeCoordinateTransform, which extends CoordinateTransform",
      "RangeCoordinateTransform uses __slots__ with _step, start, and stop fields, where step is lazily computed as (stop - start) / size",
      "The forward() method computes labels as start + positions * step, and reverse() computes positions as (labels - start) / step",
      "The slice() method uses Python's range(self.size)[sl] to compute new start/stop values for the sliced transform",
      "arange() computes size using math.ceil((stop - start) / step)",
      "linspace() with endpoint=True adjusts the stop value so the internal stop is always exclusive",
      "from_variables() raises NotImplementedError, meaning set_xindex() cannot be used with RangeIndex",
      "isel() returns a new RangeIndex for slices, None for scalars/multi-dimensional indexers, and a PandasIndex for 1D array indexers by materializing via forward()",
      "sel() requires method='nearest', uses reverse() for label-to-position conversion, and delegates scalar/array labels to super().sel()",
      "Lazy coordinate data is provided through CoordinateTransformIndexingAdapter which calls forward() on demand",
      "equals() on the transform uses np.isclose() by default but strict == when exact=True"
    ],
    "key_files": [
      "xarray/indexes/range_index.py",
      "xarray/core/indexes.py",
      "xarray/core/coordinate_transform.py",
      "xarray/core/indexing.py",
      "xarray/tests/test_range_index.py"
    ],
    "source_doc": "[docstring: xarray/indexes/range_index.py] xarray.indexes.range_index.RangeIndex\nxarray.indexes.range_index.RangeIndex:\n  Xarray index implementing a simple bounded 1-dimension interval with\n    evenly spaced, monotonic floating-point values.\n\n    This index is memory-saving, i.e., the values of its associated coordinate\n    variable are not materialized in memory.\n\n    Do not use :py:meth:`~xarray.indexes.RangeIndex.__init__` directly. Instead\n    use :py:meth:`~xarray.indexes.RangeIndex.arange` or\n    :py:meth:`~xarray.indexes.RangeIndex.linspace`, which are similar to\n    :py:func:`numpy.arange` and :py:func:`numpy.linspace`.\n\n    In the case of a monotonic integer range, it is better using a\n    :py:class:`~xarray.indexes.PandasIndex` that wraps a\n    :py:class:`pandas.RangeIndex`.\n\nxarray.indexes.range_index.RangeIndex.equals:\n  Check equality with another RangeIndex.\n\n        Parameters\n        ----------\n        other : Index\n            The other index to compare with.\n        exclude : frozenset of hashable, optional\n            Dimensions excluded from checking (unused for 1D RangeIndex).\n        exact : bool, default False\n            If False (default), use np.isclose() for floating point comparisons\n            to handle accumulated floating point errors from slicing operations.\n            If True, require exact equality of start and stop values.\n\n        Returns\n        -------\n        bool\n            True if the indexes are equal, False otherwise.\n\nxarray.indexes.range_index.RangeIndex.arange:\n  Create a new RangeIndex from given start, stop and step values.\n\n        ``RangeIndex.arange`` can be called with a varying number of positional arguments:\n\n        - ``RangeIndex.arange(stop)``: the index is within the half-open interval [0, stop)\n          (in other words, the interval including start but excluding stop).\n\n        - ``RangeIndex.arange(start, stop)``: the index is within the half-open interval\n          [start, stop).\n\n        - ``RangeIndex.arange(start, stop, step)``: the index is within the half-open interval\n          [start, stop), with spacing between values given by step.\n\n        .. note::\n           When using a non-integer step, such as 0.1, it is often better to use\n           :py:meth:`~xarray.indexes.RangeIndex.linspace`.\n\n        .. note::\n           ``RangeIndex.arange(start=4.0)`` returns a range index in the [0.0, 4.0)\n           interval, i.e., ``start`` is interpreted as ``stop`` even when it is given\n           as a unique keyword argument.\n\n        Parameters\n        ----------\n        start : float, optional\n            Start of interval. The interval includes this value. The default start\n            value is 0. If ``stop`` is not given, the value given here is interpreted\n            as the end of the interval.\n        stop : float\n            End of interval. In general the interval does not include this value,\n            except floating point round-off affects the size of the dimension.\n        step : float, optional\n            Spacing between values (default: 1.0).\n        coord_name : Hashable, optional\n            Name of the (lazy) coordinate variable that will be created and\n            associated with the new index. If ``None``, the coordinate is named\n            as the dimension name.\n        dim : str\n            Dimension name.\n        dtype : dtype, optional\n            The dtype of the coordinate variable (default: float64).\n\n        Examples\n        --------\n        >>> from xarray.indexes import RangeIndex\n\n        >>> index = RangeIndex.arange(0.0, 1.0, 0.2, dim=\"x\")\n        >>> ds = xr.Dataset(coords=xr.Coordinates.from_xindex(index))\n\n        >>> ds\n        <xarray.Dataset> Size: 40B\n        Dimensions:  (x: 5)\n        Coordinates:\n          * x        (x) float64 40B 0.0 0.2 0.4 0.6 0.8\n        Data variables:\n            *empty*\n        Indexes:\n            x        RangeIndex (start=0, stop=1, step=0.2)\n\nxarray.indexes.range_index.RangeIndex.linspace:\n  Create a new RangeIndex from given start / stop values and number of\n        values.\n\n        Parameters\n        ----------\n        start : float\n            Start of interval. The interval includes this value.\n        stop : float, optional\n            End of interval. The interval includes this value if ``endpoint=True``.\n        num : float, optional\n            Number of values in the interval, i.e., dimension size (default: 50).\n        endpoint : bool, optional\n            If True (default), the ``stop`` value is included in the interval.\n        coord_name : Hashable, optional\n            Name of the (lazy) coordinate variable that will be created and\n            associated with the new index. If ``None``, the coordinate is named\n            as the dimension name.\n        dim : str\n            Dimension name.\n        dtype : dtype, optional\n            The dtype of the coordinate variable (default: float64).\n\n        Examples\n        --------\n        >>> from xarray.indexes import RangeIndex\n\n        >>> index = RangeIndex.linspace(0.0, 1.0, 5, dim=\"x\")\n        >>> ds = xr.Dataset(coords=xr.Coordinates.from_xindex(index))\n\n        >>> ds\n        <xarray.Dataset> Size: 40B\n        Dimensions:  (x: 5)\n        Coordinates:\n          * x        (x) float64 40B 0.0 0.25 0.5 0.75 1.0\n        Data variables:\n            *empty*\n        Indexes:\n            x        RangeIndex (start=0, stop=1.25, step=0.25)\n\nxarray.indexes.range_index.RangeIndex.start:\n  Returns the start of the interval (the interval includes this value).\n\nxarray.indexes.range_index.RangeIndex.stop:\n  Returns the end of the interval (the interval does not include this value).\n\nxarray.indexes.range_index.RangeIndex.step:\n  Returns the spacing between values.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.09,
    "strip_verify_summary": "Minor doc leakage in design rationale descriptions (memory efficiency, floating-point error handling) but nearly all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_22",
    "repo": "xarray",
    "question": "Why does `_prepare_plot1d_data` insert NaN values and transpose dimensions before stacking multi-dimensional data?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `_prepare_plot1d_data` function uses NaN insertion combined with stacking for multi-dimensional data. When `darray.ndim > 1`, the function flattens all dimensions into a single 1D array via `darray.stack(_stacked_dim=darray.dims)`.\n\nFor floating-point data (checked via `np.issubdtype(darray.dtype, np.floating)`), NaN values are inserted along relevant dimensions. This is done by concatenating a NaN-filled slice (`np.nan * darray.isel({dim: -1})`) along the relevant dimension before stacking. The NaN insertion is applied for dimensions corresponding to the `\"z\"` and `\"x\"` keys in `coords_to_plot`, provided those dimensions exist in `darray.dims`.\n\nThe NaN insertion only applies to floating-point dtypes (`np.issubdtype(darray.dtype, np.floating)`).\n\nThe transpose before stacking (`darray.transpose(..., *dims_T)`) places the dimensions that had NaN slices appended at the end of the dimension order. The NaN slice is appended to the end of the dimension (via `isel({dim: -1})`), and the dimensions involved are collected into `dims_T` and transposed to the trailing positions before the stack operation flattens all dimensions.",
    "rubric": [
      "Explains that NaN insertion prevents plot lines from continuously connecting across different slices when flattened/stacked into 1D",
      "Mentions that NaN insertion only applies to floating-point data (checked via np.issubdtype with np.floating)",
      "Notes that NaN slices are appended along dimensions corresponding to the 'z' and 'x' keys in coords_to_plot",
      "Explains that the transpose places the NaN-appended dimensions at the end so that when stacking flattens the array, the NaN separators correctly break connections between different coordinate lines"
    ],
    "key_files": [
      "xarray/plot/dataarray_plot.py",
      "xarray/plot/utils.py",
      "xarray/structure/alignment.py"
    ],
    "source_doc": "[docstring: xarray/plot/dataarray_plot.py] xarray.plot.dataarray_plot._prepare_plot1d_data\nPrepare data for usage with plt.scatter.\n\n    Parameters\n    ----------\n    darray : T_DataArray\n        Base DataArray.\n    coords_to_plot : MutableMapping[str, Hashable]\n        Coords that will be plotted.\n    plotfunc_name : str | None\n        Name of the plotting function that will be used.\n\n    Returns\n    -------\n    plts : dict[str, T_DataArray]\n        Dict of DataArrays that will be sent to matplotlib.\n\n    Examples\n    --------\n    >>> # Make sure int coords are plotted:\n    >>> a = xr.DataArray(\n    ...     data=[1, 2],\n    ...     coords={1: (\"x\", [0, 1], {\"units\": \"s\"})},\n    ...     dims=(\"x\",),\n    ...     name=\"a\",\n    ... )\n    >>> plts = xr.plot.dataarray_plot._prepare_plot1d_data(\n    ...     a, coords_to_plot={\"x\": 1, \"z\": None, \"hue\": None, \"size\": None}\n    ... )\n    >>> # Check which coords to plot:\n    >>> print({k: v.name for k, v in plts.items()})\n    {'y': 'a', 'x': 1}",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the function's signature, parameters, return type, and a brief example. It says the function 'Prepare[s] data for usage with plt.scatter.' The generated answer makes extensive claims about NaN insertion, stacking, transposing, dask array handling, and performance optimization that are not present in the provided documentation.",
      "The answer's claims about internal implementation details (NaN insertion, stacking, transpose ordering, dask compute calls) cannot be verified from the provided documentation alone \u2014 they appear to be derived from reading the actual source code rather than the docstring shown.",
      "The answer does not contradict anything in the documentation, but it goes far beyond what the documentation states, making it impossible to verify most claims from the given doc chunk."
    ],
    "strip_verify_leakage": 0.4,
    "strip_verify_summary": "Several claims about design rationale (performance optimization, matplotlib NaN behavior, dask limitation comments) are not recoverable from stripped code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_23",
    "repo": "xarray",
    "question": "Where is the FacetGrid class defined and what are the main entry points that instantiate it across the codebase?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `FacetGrid` class is defined in `xarray/plot/facetgrid.py` and is the central class for creating multi-panel (small multiples/trellis) plots in xarray. It is a `Generic[T_DataArrayOrSet]` class (not inheriting from any base class other than `Generic`). Here is where key parts are located and how they connect:\n\n**Location and instantiation:**\n- `FacetGrid` is defined at line 62 of `xarray/plot/facetgrid.py`.\n- It is instantiated primarily through the `_easy_facetgrid()` convenience function (line 1024 of the same file), which is the common entry point for all faceted plotting.\n\n**Callers of `_easy_facetgrid`:**\n- `xarray/plot/dataarray_plot.py` calls it for `kind=\"line\"` (line 487), `kind=\"plot1d\"` (line 900), and `kind=\"dataarray\"` (line 1512).\n- `xarray/plot/dataset_plot.py` calls it for `kind=\"dataset\"` (line 238).\n- `_easy_facetgrid` raises `ValueError` if both `size` and `figsize` are provided. It also sets `sharex=False, sharey=False` when `z` is not None (3D plots).\n\n**Key internal details not in documentation:**\n1. The `__init__` method raises a `ValueError` if coordinates used for faceting contain repeated (nonunique) values (lines 172-178). It checks `data[col].to_index().is_unique` and `data[row].to_index().is_unique`.\n2. When `col_wrap` is specified alongside both `row` and `col`, a warning is issued (`\"Ignoring col_wrap since both col and row were passed\"`) rather than an error (line 188-190).\n3. The default figure size calculation uses a `cbar_space = 1` inch added to the width: `figsize = (ncol * size * aspect + cbar_space, nrow * size)` (lines 217-218).\n4. The `axes` property (line 280) is deprecated since 2022.11 with a `FutureWarning`, directing users to use `axs` instead (to align with matplotlib's `plt.subplots` naming).\n5. In `map_dataarray`, the function passes `_is_facetgrid=True` as a keyword argument to the underlying plotting function (line 376). This flag is also used in `_process_cmap_cbar_kwargs` (in `xarray/plot/utils.py` line 964) to skip calling `_determine_cmap_params` when inside a facetgrid.\n6. The `map` method (unlike `map_dataarray`) calls `plt.sca(ax)` to set the current axes before calling the user-provided function (line 1011), and it checks for mappables using `hasattr(maybe_mappable, \"autoscale_None\")` (line 1016).\n7. `map_plot1d` copies `self.data` at the start (line 427) to allow converting categoricals to integers. It also constructs a 3D boolean array `add_labels_` of shape `(nrow, ncol, 3)` to control which subplots get x/y/z labels (lines 488-497).\n8. `_finalize_grid` uses a `_finalized` boolean guard (line 682) so it only runs once \u2014 it calls `set_axis_labels`, `set_titles`, `fig.tight_layout()`, and hides empty axes via `ax.set_visible(False)` for any `namedict` that is `None`.\n9. The `_nicetitle` helper function (line 46) uses `format_item` from `xarray.core.formatting` with `quote_strings=False` to format facet title values.\n10. `_FONTSIZE = \"small\"` and `_NTICKS = 5` are module-level constants used as defaults for `set_ticks`.\n11. `_MARKERSIZE_RANGE = (18.0, 36.0, 72.0)` and `_LINEWIDTH_RANGE = (1.5, 1.5, 6.0)` are defined in `xarray/plot/utils.py` (copied from seaborn) and used in `map_plot1d` based on whether the function is `scatter` or not.\n12. `_adjust_fig_for_guide` (line 693) draws the figure twice to compute guide width, then calls `fig.subplots_adjust(right=...)` to make room.\n13. `add_colorbar` uses `self.fig.colorbar(self._mappables[-1], ax=list(self.axs.flat), **kwargs)` \u2014 passing all axes as the `ax` parameter to steal space from all of them.\n14. `_set_lims` calls `_get_largest_lims` first and uses the largest computed limits as defaults, only overriding them if explicit limits are passed.\n15. The `_determine_guide` function in `utils.py` returns `(False, False)` immediately if `plotfunc_name == \"hist\"`.\n16. `set_titles` places row titles on the right edge of the grid (at `xy=(1.02, 0.5)` in axes fraction, rotated 270\u00b0) and column titles on the top row using `ax.set_title`.\n17. `label_from_attrs` in `xarray/plot/utils.py` checks for `long_name`, `standard_name`, then `name` from DataArray attrs/properties, and wraps text at 30 characters (or 60 for LaTeX sequences).",
    "rubric": [
      "FacetGrid is defined in xarray/plot/facetgrid.py",
      "It is primarily instantiated through the _easy_facetgrid() convenience function in the same file",
      "xarray/plot/dataarray_plot.py calls _easy_facetgrid for line, plot1d, and/or dataarray plot kinds",
      "xarray/plot/dataset_plot.py calls _easy_facetgrid for dataset plot kind",
      "FacetGrid is a Generic[T_DataArrayOrSet] class (no other base class besides Generic)",
      "_easy_facetgrid raises ValueError if both size and figsize are provided"
    ],
    "key_files": [
      "xarray/plot/facetgrid.py",
      "xarray/plot/utils.py",
      "xarray/plot/dataarray_plot.py",
      "xarray/plot/dataset_plot.py",
      "xarray/core/formatting.py"
    ],
    "source_doc": "[docstring: xarray/plot/facetgrid.py] xarray.plot.facetgrid.FacetGrid\nxarray.plot.facetgrid.FacetGrid:\n  Initialize the Matplotlib figure and FacetGrid object.\n\n    The :class:`FacetGrid` is an object that links an xarray DataArray to\n    a Matplotlib figure with a particular structure.\n\n    In particular, :class:`FacetGrid` is used to draw plots with multiple\n    axes, where each axes shows the same relationship conditioned on\n    different levels of some dimension. It's possible to condition on up to\n    two variables by assigning variables to the rows and columns of the\n    grid.\n\n    The general approach to plotting here is called \"small multiples\",\n    where the same kind of plot is repeated multiple times, and the\n    specific use of small multiples to display the same relationship\n    conditioned on one or more other variables is often called a \"trellis\n    plot\".\n\n    The basic workflow is to initialize the :class:`FacetGrid` object with\n    the DataArray and the variable names that are used to structure the grid.\n    Then plotting functions can be applied to each subset by calling\n    :meth:`FacetGrid.map_dataarray` or :meth:`FacetGrid.map`.\n\n    Attributes\n    ----------\n    axs : ndarray of matplotlib.axes.Axes\n        Array containing axes in corresponding position, as returned from\n        :py:func:`matplotlib.pyplot.subplots`.\n    col_labels : list of matplotlib.text.Annotation\n        Column titles.\n    row_labels : list of matplotlib.text.Annotation\n        Row titles.\n    fig : matplotlib.figure.Figure\n        The figure containing all the axes.\n    name_dicts : ndarray of dict\n        Array containing dictionaries mapping coordinate names to values. ``None`` is\n        used as a sentinel value for axes that should remain empty, i.e.,\n        sometimes the rightmost grid positions in the bottom row.\n\nxarray.plot.facetgrid.FacetGrid.__init__:\n  Parameters\n        ----------\n        data : DataArray or Dataset\n            DataArray or Dataset to be plotted.\n        row, col : str\n            Dimension names that define subsets of the data, which will be drawn\n            on separate facets in the grid.\n        col_wrap : int, optional\n            \"Wrap\" the grid the for the column variable after this number of columns,\n            adding rows if ``col_wrap`` is less than the number of facets.\n        sharex : bool, optional\n            If true, the facets will share *x* axes.\n        sharey : bool, optional\n            If true, the facets will share *y* axes.\n        figsize : Iterable of float or None, optional\n            A tuple (width, height) of the figure in inches.\n            If set, overrides ``size`` and ``aspect``.\n        aspect : scalar, default: 1\n            Aspect ratio of each facet, so that ``aspect * size`` gives the\n            width of each facet in inches.\n        size : scalar, default: 3\n            Height (in inches) of each facet. See also: ``aspect``.\n        subplot_kws : dict, optional\n            Dictionary of keyword arguments for Matplotlib subplots\n            (:py:func:`matplotlib.pyplot.subplots`).\n\nxarray.plot.facetgrid.FacetGrid.map_dataarray:\n  Apply a plotting function to a 2d facet's subset of the data.\n\n        This is more convenient and less general than ``FacetGrid.map``\n\n        Parameters\n        ----------\n        func : callable\n            A plotting function with the same signature as a 2d xarray\n            plotting method such as `xarray.plot.imshow`\n        x, y : string\n            Names of the coordinates to plot on x, y axes\n        **kwargs\n            additional keyword arguments to func\n\n        Returns\n        -------\n        self : FacetGrid object\n\nxarray.plot.facetgrid.FacetGrid.map_plot1d:\n  Apply a plotting function to a 1d facet's subset of the data.\n\n        This is more convenient and less general than ``FacetGrid.map``\n\n        Parameters\n        ----------\n        func :\n            A plotting function with the same signature as a 1d xarray\n            plotting method such as `xarray.plot.scatter`\n        x, y :\n            Names of the coordinates to plot on x, y axes\n        **kwargs\n            additional keyword arguments to func\n\n        Returns\n        -------\n        self : FacetGrid object\n\nxarray.plot.facetgrid.FacetGrid._finalize_grid:\n  Finalize the annotations and layout.\n\nxarray.plot.facetgrid.FacetGrid.add_colorbar:\n  Draw a colorbar.\n\nxarray.plot.facetgrid.FacetGrid._get_largest_lims:\n  Get largest limits in the facetgrid.\n\n        Returns\n        -------\n        lims_largest : dict[str, tuple[float, float]]\n            Dictionary with the largest limits along each axis.\n\n        Examples\n        --------\n        >>> ds = xr.tutorial.scatter_example_dataset(seed=42)\n        >>> fg = ds.plot.scatter(x=\"A\", y=\"B\", hue=\"y\", row=\"x\", col=\"w\")\n        >>> round(fg._get_largest_lims()[\"x\"][0], 3)\n        np.float64(-0.334)\n\nxarray.plot.facetgrid.FacetGrid._set_lims:\n  Set the same limits for all the subplots in the facetgrid.\n\n        Parameters\n        ----------\n        x : tuple[float, float] or None, optional\n            x axis limits.\n        y : tuple[float, float] or None, optional\n            y axis limits.\n        z : tuple[float, float] or None, optional\n            z axis limits.\n\n        Examples\n        --------\n        >>> ds = xr.tutorial.scatter_example_dataset(seed=42)\n        >>> fg = ds.plot.scatter(x=\"A\", y=\"B\", hue=\"y\", row=\"x\", col=\"w\")\n        >>> fg._set_lims(x=(-0.3, 0.3), y=(0, 2), z=(0, 4))\n        >>> fg.axs[0, 0].get_xlim(), fg.axs[0, 0].get_ylim()\n        ((np.float64(-0.3), np.float64(0.3)), (np.float64(0.0), np.float64(2.0)))\n\nxarray.plot.facetgrid.FacetGrid.set_axis_labels:\n  Set axis labels on the left column and bottom row of the grid.\n\nxarray.plot.facetgrid.FacetGrid.set_xlabels:\n  Label the x axis on the bottom row of the grid.\n\nxarray.plot.facetgrid.FacetGrid.set_ylabels:\n  Label the y axis on the left column of the grid.\n\nxarray.plot.facetgrid.FacetGrid.set_zlabels:\n  Label the z axis.\n\nxarray.plot.facetgrid.FacetGrid.set_titles:\n  Draw titles either above each facet or on the grid margins.\n\n        Parameters\n        ----------\n        template : str, default: \"{coord} = {value}\"\n            Template for plot titles containing {coord} and {value}\n        maxchar : int, default: 30\n            Truncate titles at maxchar\n        **kwargs : keyword args\n            additional arguments to matplotlib.text\n\n        Returns\n        -------\n        self: FacetGrid object\n\nxarray.plot.facetgrid.FacetGrid.set_ticks:\n  Set and control tick behavior.\n\n        Parameters\n        ----------\n        max_xticks, max_yticks : int, optional\n            Maximum number of labeled ticks to plot on x, y axes\n        fontsize : string or int\n            Font size as used by matplotlib text\n\n        Returns\n        -------\n        self : FacetGrid object\n\nxarray.plot.facetgrid.FacetGrid.map:\n  Apply a plotting function to each facet's subset of the data.\n\n        Parameters\n        ----------\n        func : callable\n            A plotting function that takes data and keyword arguments. It\n            must plot to the currently active matplotlib Axes and take a\n            `color` keyword argument. If faceting on the `hue` dimension,\n            it must also take a `label` keyword argument.\n        *args : Hashable\n            Column names in self.data that identify variables with data to\n            plot. The data for each variable is passed to `func` in the\n            order the variables are specified in the call.\n        **kwargs : keyword arguments\n            All keyword arguments are passed to the plotting function.\n\n        Returns\n        -------\n        self : FacetGrid object",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims (line numbers, internal implementation details like `_is_facetgrid=True`, `_finalized` guard, `_nicetitle`, `_FONTSIZE`, `_NTICKS`, `_MARKERSIZE_RANGE`, `_LINEWIDTH_RANGE`, `_adjust_fig_for_guide`, `_determine_guide`, `label_from_attrs`, specific default values, etc.) are not verifiable from the provided documentation. These are plausible code-level details but could be hallucinations since we cannot confirm them from the doc alone.",
      "The claim that `col_wrap` with both `row` and `col` produces a warning rather than an error is not mentioned in the documentation and cannot be verified.",
      "The claim about `cbar_space = 1` inch and the specific figsize formula is not in the documentation.",
      "The claim about `axes` property being deprecated since 2022.11 with `FutureWarning` is not in the documentation.",
      "The claim that `_easy_facetgrid` raises `ValueError` if both `size` and `figsize` are provided is not in the documentation (the doc only says figsize overrides size and aspect).",
      "The claim about `set_titles` placing row titles at `xy=(1.02, 0.5)` rotated 270\u00b0 is not verifiable from the documentation, which only says 'Draw titles either above each facet or on the grid margins'.",
      "The documentation says the default template for set_titles is '{coord} = {value}', which is consistent with the answer (not contradicted).",
      "The documentation describes `map` as taking `*args` as 'Column names in self.data', and the answer's claim about `plt.sca(ax)` and `hasattr(maybe_mappable, 'autoscale_None')` is not verifiable.",
      "The documentation says `aspect` default is 1 and `size` default is 3, which are not contradicted by the answer."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_24",
    "repo": "xarray",
    "question": "How does the `_Normalize` class handle both numeric and categorical data normalization for xarray's plotting system?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `_Normalize` class in `xarray/plot/utils.py` implements a normalization mechanism for both numerical and categorical data used in xarray's plotting system. Here's how it works in detail:\n\n**Class Design:**\n- `_Normalize` inherits from `collections.abc.Sequence` (imported at the top of the file), implementing `__len__` (returns `len(self._data_unique)`) and `__getitem__` (delegates to `self._data_unique[key]`).\n- It uses `__slots__` with 6 attributes: `_data`, `_data_is_numeric`, `_data_unique`, `_data_unique_index`, `_data_unique_inverse`, and `_width`.\n\n**Initialization Algorithm:**\n1. Stores the data and width. If `_is_facetgrid=True`, the width is forced to `None`.\n2. Checks if the data is a pint Quantity using `DuckArrayModule(\"pint\").type` \u2014 if so, converts to numpy before processing with `data.to_numpy()`.\n3. Calls `np.unique(to_unique, return_inverse=True)` to get unique values and inverse indices.\n4. Determines if data is numeric using `_is_numeric()`, which checks if the dtype is a subtype of `np.floating` or `np.integer` only (not datetime64 or timedelta64 via `np.issubdtype`).\n\n**Width Normalization (`_calc_widths`):**\n- When `width` is `None`, returns values unchanged.\n- When all values are constant (`diff_maxy_miny == 0`), returns the default (middle) width value for all elements: `xdefault + 0 * y`.\n- Otherwise applies linear interpolation: `k = (y - min(y)) / (max(y) - min(y))`, then `widths = xmin + k * (xmax - xmin)`.\n\n**Categorical vs Numeric Path:**\n- For categorical data, `_indexes_centered` maps indices via `x * 2 + 1` (e.g., indices [0,1,2] \u2192 [1,3,5]), centering values between `levels` which are computed as `index * 2` (e.g., [0,2,4,6]).\n- For numeric data, the `values` property returns the original data directly (optionally scaled through `_calc_widths`).\n\n**Inverse Lookup:**\n- `_lookup` creates a `pd.Series` mapping normalized unique values \u2192 original unique values.\n- `_lookup_arr` uses `pd.Series.sort_index().reindex(x, method=\"nearest\")` to handle floating-point imprecision when mapping back from normalized to original values.\n\n**Integration with Plot System:**\n- Used in `dataarray_plot.py` and `facetgrid.py` for both hue and size normalization.\n- `_MARKERSIZE_RANGE = (18.0, 36.0, 72.0)` and `_LINEWIDTH_RANGE = (1.5, 1.5, 6.0)` are the default width tuples.\n- `_determine_guide()` uses `_Normalize` instances to decide whether to add a colorbar (for numeric hue) or legend (for non-numeric hue or when size is specified).\n- The `format` property returns a `matplotlib.pyplot.FuncFormatter` for colorbar tick labeling, and `func` returns a callable for `ax.legend_elements`.\n- The custom `legend_elements` function in `xarray/plot/utils.py` accepts a `func` parameter that receives `_Normalize.func`'s output to map normalized values back to original values for legend labels.",
    "rubric": [
      "Mentions that _Normalize inherits from collections.abc.Sequence and implements __len__ and __getitem__",
      "Explains that __slots__ defines 6 attributes: _data, _data_is_numeric, _data_unique, _data_unique_index, _data_unique_inverse, and _width",
      "Describes initialization: stores data/width, handles pint Quantities via to_numpy(), calls np.unique with return_inverse=True, and checks numeric status",
      "Explains that _is_numeric checks only np.floating and np.integer subtypes (excluding datetime64)",
      "Describes _calc_widths: returns unchanged when width is None, uses default value when data is constant, otherwise applies linear interpolation between xmin and xmax",
      "Explains categorical path: _indexes_centered maps via x * 2 + 1, and levels are computed as index * 2 to evenly bound the centered values",
      "Explains numeric path: values property returns original data directly (passed through _calc_widths)",
      "Describes _lookup/_lookup_arr: creates pd.Series mapping normalized\u2192original values and uses reindex with method='nearest' for float tolerance",
      "Mentions format property returns FuncFormatter for colorbar labeling and func property returns callable for legend_elements",
      "Explains integration: _determine_guide uses data_is_numeric to decide colorbar vs legend, and width defaults include _MARKERSIZE_RANGE (18, 36, 72) and _LINEWIDTH_RANGE (1.5, 1.5, 6.0)"
    ],
    "key_files": [
      "xarray/plot/utils.py",
      "xarray/plot/dataarray_plot.py",
      "xarray/namedarray/pycompat.py",
      "xarray/plot/facetgrid.py"
    ],
    "source_doc": "[docstring: xarray/plot/utils.py] xarray.plot.utils._Normalize\nxarray.plot.utils._Normalize:\n  Normalize numerical or categorical values to numerical values.\n\n    The class includes helper methods that simplifies transforming to\n    and from normalized values.\n\n    Parameters\n    ----------\n    data : DataArray\n        DataArray to normalize.\n    width : Sequence of three numbers, optional\n        Normalize the data to these (min, default, max) values.\n        The default is None.\n\nxarray.plot.utils._Normalize.data_is_numeric:\n  Check if data is numeric.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([\"b\", \"a\", \"a\", \"b\", \"c\"])\n        >>> _Normalize(a).data_is_numeric\n        False\n\n        >>> a = xr.DataArray([0.5, 0, 0, 0.5, 2, 3])\n        >>> _Normalize(a).data_is_numeric\n        True\n\n        >>> # TODO: Datetime should be numeric right?\n        >>> a = xr.DataArray(pd.date_range(\"2000-1-1\", periods=4))\n        >>> _Normalize(a).data_is_numeric\n        False\n\n        # TODO: Timedelta should be numeric right?\n        >>> a = xr.DataArray(pd.timedelta_range(\"-1D\", periods=4, freq=\"D\"))\n        >>> _Normalize(a).data_is_numeric\n        True\n\nxarray.plot.utils._Normalize._calc_widths:\n  Normalize the values so they're in between self._width.\n\nxarray.plot.utils._Normalize._indexes_centered:\n  Offset indexes to make sure being in the center of self.levels.\n        [\"a\", \"b\", \"c\"] -> [1, 3, 5]\n\nxarray.plot.utils._Normalize.values:\n  Return a normalized number array for the unique levels.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([\"b\", \"a\", \"a\", \"b\", \"c\"])\n        >>> _Normalize(a).values\n        <xarray.DataArray (dim_0: 5)> Size: 40B\n        array([3, 1, 1, 3, 5])\n        Dimensions without coordinates: dim_0\n\n        >>> _Normalize(a, width=(18, 36, 72)).values\n        <xarray.DataArray (dim_0: 5)> Size: 40B\n        array([45., 18., 18., 45., 72.])\n        Dimensions without coordinates: dim_0\n\n        >>> a = xr.DataArray([0.5, 0, 0, 0.5, 2, 3])\n        >>> _Normalize(a).values\n        <xarray.DataArray (dim_0: 6)> Size: 48B\n        array([0.5, 0. , 0. , 0.5, 2. , 3. ])\n        Dimensions without coordinates: dim_0\n\n        >>> _Normalize(a, width=(18, 36, 72)).values\n        <xarray.DataArray (dim_0: 6)> Size: 48B\n        array([27., 18., 18., 27., 54., 72.])\n        Dimensions without coordinates: dim_0\n\n        >>> _Normalize(a * 0, width=(18, 36, 72)).values\n        <xarray.DataArray (dim_0: 6)> Size: 48B\n        array([36., 36., 36., 36., 36., 36.])\n        Dimensions without coordinates: dim_0\n\nxarray.plot.utils._Normalize._values_unique:\n  Return unique values.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([\"b\", \"a\", \"a\", \"b\", \"c\"])\n        >>> _Normalize(a)._values_unique\n        array([1, 3, 5])\n\n        >>> _Normalize(a, width=(18, 36, 72))._values_unique\n        array([18., 45., 72.])\n\n        >>> a = xr.DataArray([0.5, 0, 0, 0.5, 2, 3])\n        >>> _Normalize(a)._values_unique\n        array([0. , 0.5, 2. , 3. ])\n\n        >>> _Normalize(a, width=(18, 36, 72))._values_unique\n        array([18., 27., 54., 72.])\n\nxarray.plot.utils._Normalize.ticks:\n  Return ticks for plt.colorbar if the data is not numeric.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([\"b\", \"a\", \"a\", \"b\", \"c\"])\n        >>> _Normalize(a).ticks\n        array([1, 3, 5])\n\nxarray.plot.utils._Normalize.levels:\n  Return discrete levels that will evenly bound self.values.\n        [\"a\", \"b\", \"c\"] -> [0, 2, 4, 6]\n\n        Examples\n        --------\n        >>> a = xr.DataArray([\"b\", \"a\", \"a\", \"b\", \"c\"])\n        >>> _Normalize(a).levels\n        array([0, 2, 4, 6])\n\nxarray.plot.utils._Normalize.format:\n  Return a FuncFormatter that maps self.values elements back to\n        the original value as a string. Useful with plt.colorbar.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([0.5, 0, 0, 0.5, 2, 3])\n        >>> aa = _Normalize(a, width=(0, 0.5, 1))\n        >>> aa._lookup\n        0.000000    0.0\n        0.166667    0.5\n        0.666667    2.0\n        1.000000    3.0\n        dtype: float64\n        >>> aa.format(1)\n        '3.0'\n\nxarray.plot.utils._Normalize.func:\n  Return a lambda function that maps self.values elements back to\n        the original value as a numpy array. Useful with ax.legend_elements.\n\n        Examples\n        --------\n        >>> a = xr.DataArray([0.5, 0, 0, 0.5, 2, 3])\n        >>> aa = _Normalize(a, width=(0, 0.5, 1))\n        >>> aa._lookup\n        0.000000    0.0\n        0.166667    0.5\n        0.666667    2.0\n        1.000000    3.0\n        dtype: float64\n        >>> aa.func([0.16, 1])\n        array([0.5, 3. ])",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that `_Normalize` inherits from `collections.abc.Sequence` with `__len__` and `__getitem__` is not mentioned in the documentation \u2014 plausible but unverifiable from the provided doc.",
      "The claim about `__slots__` with 6 specific attributes is not in the documentation \u2014 plausible implementation detail but unverifiable.",
      "The claim about `_is_facetgrid=True` forcing width to `None` is not in the documentation \u2014 plausible but unverifiable.",
      "The claim about pint Quantity handling is not in the documentation \u2014 plausible but unverifiable.",
      "The detailed `_calc_widths` algorithm (linear interpolation formula) is not explicitly in the documentation, though examples are consistent with it.",
      "The claim that `_is_numeric()` checks `np.issubdtype` for `np.floating` or `np.integer` only is partially consistent with doc examples: timedelta returns True in the doc (contradicting the claim that datetime64/timedelta64 are excluded), and datetime returns False. The answer says 'not datetime64 or timedelta64 via np.issubdtype' which contradicts the documented behavior where timedelta returns True.",
      "The `_indexes_centered` formula `x * 2 + 1` is consistent with the doc example ['a','b','c'] -> [1,3,5].",
      "The `levels` formula as `index * 2` is consistent with the doc example ['a','b','c'] -> [0,2,4,6].",
      "The `_lookup` and `_lookup_arr` details about pd.Series are plausible but not explicitly in the doc, though the `format` example shows `_lookup` output.",
      "Claims about `_MARKERSIZE_RANGE`, `_LINEWIDTH_RANGE`, `_determine_guide()`, `legend_elements`, and integration with `dataarray_plot.py`/`facetgrid.py` are not in the documentation \u2014 plausible but unverifiable.",
      "The claim about `_is_numeric()` behavior with timedelta is potentially contradictory: the doc shows timedelta `data_is_numeric` returns True, but the answer implies timedelta64 is excluded from numeric check."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the 'copied from seaborn' comment and 'deferring width normalization' rationale were documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_25",
    "repo": "xarray",
    "question": "What is the role of `_guess_coords_to_plot` and how does it fit into the 1D plotting architecture?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "`_guess_coords_to_plot` is a function in `xarray/plot/utils.py` that fills in `None` values in a `coords_to_plot` mapping by assigning available coordinates from the DataArray in order. Here is how it fits into the broader architecture:\n\n**Core Algorithm:**\n1. It first builds `coords_to_plot_exist` \u2014 a filtered dict of user-specified (non-None) coordinates.\n2. It computes `available_coords` as a tuple of coordinate keys from `darray.coords` that are NOT already in the user-specified values.\n3. It uses `zip(default_guess, available_coords, ignore_guess_kwargs, strict=False)` to iterate over the guess targets, available coords, and ignore kwargs simultaneously. The `strict=False` means the zip stops at the shortest iterable \u2014 if there are fewer available coords than default guesses, extra guesses are simply skipped.\n4. For each triplet `(k, dim, ign_kws)`, it assigns the coord only if `coords_to_plot[k]` is None AND none of the matplotlib ignore kwargs are present in `kwargs`.\n5. After guessing, it calls `_assert_valid_xy(darray, dim, k)` on every entry in `coords_to_plot` to validate.\n\n**Default Parameters:**\n- `default_guess` defaults to `(\"x\",)` \u2014 so by default only the `x` coordinate is guessed.\n- `ignore_guess_kwargs` defaults to `((),)` \u2014 an empty tuple meaning no matplotlib kwargs are checked.\n\n**Callers (both use only default parameters, i.e., only guess 'x'):**\n1. `dataarray_plot.py` in the `newplotfunc` inner function of the `_plot1d` decorator \u2014 called only when `_is_facetgrid` is False.\n2. `facetgrid.py` in `FacetGrid.map_plot1d` \u2014 called unconditionally.\n\nBoth callers construct `coords_to_plot` as `dict(x=x, z=z, hue=hue, size=size_)`, where `size_` is `markersize` for scatter plots and `linewidth` for other plot types.\n\n**Validation via `_assert_valid_xy`:**\nThe `_assert_valid_xy` function validates each coordinate against `(set(darray.dims) | set(darray.coords)) - multiindex_dims`, where `multiindex_dims` are dimensions associated with `PandasMultiIndex` instances from `darray.xindexes.get_unique()`. MultiIndex dimensions are explicitly excluded from valid coordinates.\n\n**Downstream consumer:**\nThe result of `_guess_coords_to_plot` is passed to `_prepare_plot1d_data` (in `dataarray_plot.py`), which:\n- Stacks multi-dimensional arrays (ndim > 1) by inserting NaN separators for float data along z/x dims, transposing, then calling `.stack(_stacked_dim=darray.dims)`.\n- Creates a dict starting with `y=darray`, then adds entries for each non-None coordinate from `coords_to_plot`, and broadcasts them together.\n\n**`scatter` is the function decorated by `_plot1d`**. The older `line` function uses `_infer_line_data` instead and does NOT call `_guess_coords_to_plot`.\n\n**Example dataset context:**\n`scatter_example_dataset(seed=42)` creates a Dataset with DataArrays of shape `[3, 11, 4, 4]` and dims `['x', 'y', 'z', 'w']`, where `w` is overwritten with string values `['one', 'two', 'three', 'five']`. The coords have attributes like `units: 'xunits'`, `units: 'yunits'`, etc.\n\n**Related constants:**\n`_MARKERSIZE_RANGE = (18.0, 36.0, 72.0)` and `_LINEWIDTH_RANGE = (1.5, 1.5, 6.0)` are used to normalize size data via the `_Normalize` class after coordinate guessing.",
    "rubric": [
      "Explains that _guess_coords_to_plot fills in None values in a coords_to_plot mapping by assigning available coordinates from the DataArray",
      "Describes the algorithm: builds existing coords dict, computes available_coords excluding already-specified ones, then uses zip with strict=False over default_guess/available_coords/ignore_guess_kwargs",
      "Mentions that assignment only happens if coords_to_plot[k] is None AND none of the ignore kwargs are present in kwargs",
      "Notes that _assert_valid_xy is called on every entry in coords_to_plot for validation",
      "Identifies that default_guess defaults to ('x',) meaning only the x coordinate is guessed by default",
      "Identifies the two callers: the _plot1d decorator's inner function (when not facetgrid) and FacetGrid.map_plot1d",
      "Explains that coords_to_plot includes keys like x, z, hue, and size (markersize for scatter, linewidth for others)",
      "Mentions that results feed into _prepare_plot1d_data which handles stacking multi-dimensional arrays and broadcasting",
      "Notes that scatter is decorated by _plot1d while the older line function uses _infer_line_data instead",
      "Mentions _assert_valid_xy excludes MultiIndex dimensions from valid coordinates"
    ],
    "key_files": [
      "xarray/plot/utils.py",
      "xarray/plot/dataarray_plot.py",
      "xarray/plot/facetgrid.py",
      "xarray/tutorial.py"
    ],
    "source_doc": "[docstring: xarray/plot/utils.py] xarray.plot.utils._guess_coords_to_plot\nGuess what coords to plot if some of the values in coords_to_plot are None which\n    happens when the user has not defined all available ways of visualizing\n    the data.\n\n    Parameters\n    ----------\n    darray : DataArray\n        The DataArray to check for available coords.\n    coords_to_plot : MutableMapping[str, Hashable]\n        Coords defined by the user to plot.\n    kwargs : dict\n        Extra kwargs that will be sent to matplotlib.\n    default_guess : Iterable[str], optional\n        Default values and order to retrieve dims if values in dims_plot is\n        missing, default: (\"x\", \"hue\", \"size\").\n    ignore_guess_kwargs : tuple[tuple[str, ...], ...]\n        Matplotlib arguments to ignore.\n\n    Examples\n    --------\n    >>> ds = xr.tutorial.scatter_example_dataset(seed=42)\n    >>> # Only guess x by default:\n    >>> xr.plot.utils._guess_coords_to_plot(\n    ...     ds.A,\n    ...     coords_to_plot={\"x\": None, \"z\": None, \"hue\": None, \"size\": None},\n    ...     kwargs={},\n    ... )\n    {'x': 'x', 'z': None, 'hue': None, 'size': None}\n\n    >>> # Guess all plot dims with other default values:\n    >>> xr.plot.utils._guess_coords_to_plot(\n    ...     ds.A,\n    ...     coords_to_plot={\"x\": None, \"z\": None, \"hue\": None, \"size\": None},\n    ...     kwargs={},\n    ...     default_guess=(\"x\", \"hue\", \"size\"),\n    ...     ignore_guess_kwargs=((), (\"c\", \"color\"), (\"s\",)),\n    ... )\n    {'x': 'x', 'z': None, 'hue': 'y', 'size': 'z'}\n\n    >>> # Don't guess \u00b4size\u00b4, since the matplotlib kwarg \u00b4s\u00b4 has been defined:\n    >>> xr.plot.utils._guess_coords_to_plot(\n    ...     ds.A,\n    ...     coords_to_plot={\"x\": None, \"z\": None, \"hue\": None, \"size\": None},\n    ...     kwargs={\"s\": 5},\n    ...     default_guess=(\"x\", \"hue\", \"size\"),\n    ...     ignore_guess_kwargs=((), (\"c\", \"color\"), (\"s\",)),\n    ... )\n    {'x': 'x', 'z': None, 'hue': 'y', 'size': None}\n\n    >>> # Prioritize \u00b4size\u00b4 over \u00b4s\u00b4:\n    >>> xr.plot.utils._guess_coords_to_plot(\n    ...     ds.A,\n    ...     coords_to_plot={\"x\": None, \"z\": None, \"hue\": None, \"size\": \"x\"},\n    ...     kwargs={\"s\": 5},\n    ...     default_guess=(\"x\", \"hue\", \"size\"),\n    ...     ignore_guess_kwargs=((), (\"c\", \"color\"), (\"s\",)),\n    ... )\n    {'x': 'y', 'z': None, 'hue': 'z', 'size': 'x'}",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that 'default_guess defaults to (\"x\",)' appears to contradict the docstring which states 'default: (\"x\", \"hue\", \"size\")'. However, looking more carefully at the docstring, it says the default is '(\"x\", \"hue\", \"size\")' but the first example shows that 'Only guess x by default' - this suggests the actual default parameter value is just (\"x\",) and the docstring's 'default: (\"x\", \"hue\", \"size\")' describes what values are possible, not the actual default. The first example confirms that with no default_guess specified, only x is guessed. So this is actually consistent with the documentation's examples, though the docstring parameter description is ambiguous.",
      "The answer claims 'zip(..., strict=False)' but this is an implementation detail not in the documentation - it could be a hallucination about the specific implementation mechanism.",
      "Many implementation details (line numbers, _prepare_plot1d_data, _Normalize class, _MARKERSIZE_RANGE, _LINEWIDTH_RANGE, scatter_example_dataset shape details, facetgrid.py line 443, etc.) are not verifiable from the provided documentation chunk alone, though they are plausible code-level extrapolations.",
      "The docstring says default_guess default is '(\"x\", \"hue\", \"size\")' but the answer claims it defaults to '(\"x\",)'. The examples actually support the answer's claim since the first example (without default_guess specified) only guesses x. This is a contradiction between the docstring's parameter description and its examples - the answer sides with the examples."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage from line numbers, 'copied from seaborn' comment, and 'docstring examples' reference; core algorithmic claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_26",
    "repo": "xarray",
    "question": "Why is xarray's Aligner class designed with so many specialized code paths and matching strategies for its different join modes?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `align` function in xarray is designed around a sophisticated `Aligner` class that encapsulates all complex alignment logic. Several key design aspects emerge from the code:\n\n**1. Performance optimization through early identity checks:**\nThe `indexes_all_equal` function (in `xarray/core/indexes.py`) is used in `_need_reindex` to determine if reindexing can be skipped when indexes are already equal.\n\n**2. Avoidance of reindexing when indexes are equal:**\nThe `_need_reindex` method returns `False` when `indexes_all_equal` returns `True`, avoiding reindexing when indexes are already equal. This is the first check performed in the method.\n\n**3. Delegation pattern via `_reindex_callback`:**\nThe Aligner uses a protocol-based callback (`_reindex_callback` defined in the `Alignable` protocol in `xarray/core/types.py`). The protocol defines `_reindex_callback` accepting the aligner, dim_pos_indexers, variables, indexes, fill_value, exclude_dims, and exclude_vars parameters.\n\n**4. `join='override'` uses a separate code path:**\nIn `align_indexes()`, when `join == 'override'`, the code sets `joined_index = matching_indexes[0]` and `need_reindex = False`, replacing indexes on other objects with those from the first object using `_overwrite_indexes` (defined in the Alignable protocol). The `find_matching_indexes` method additionally validates that all indexes have the same *size* when override is used (raising `AlignmentError` otherwise).\n\n**5. Negative values in positional indexers signal missing values:**\nIn `reindex_variables`, negative values in `dim_pos_indexers` are detected with `(indxr < 0).any()` to classify dimensions as `masked_dims`, which then use `_getitem_with_mask` to fill with `fill_value`.\n\n**6. The `MatchingIndexKey` design:**\nIndexes are matched across objects using a key that is a tuple of `(CoordNamesAndDims, type[Index])`, as defined by `MatchingIndexKey = tuple[CoordNamesAndDims, type[Index]]`. This means indexes are only compared with other indexes of the same type that are associated with the same coordinate names and dimensions.\n\n**7. Default fill value uses a sentinel, not literal NaN:**\n`dtypes.NA` is defined as `utils.ReprObject('<NA>')` (not `np.nan`). The `reindex_variables` function uses this as the default `fill_value` parameter, and downstream code in `maybe_promote` chooses the appropriate NA value based on dtype.\n\n**8. `_get_index_joiner` uses `functools.reduce` for outer/inner joins:**\nFor 'outer' and 'inner' joins, the joiner is constructed as `functools.reduce(functools.partial(index_cls.join, how=self.join))`, which progressively merges multiple indexes pairwise. For 'left'/'right', it simply picks the first/last index using `operator.itemgetter`.",
    "rubric": [
      "Explains that `indexes_all_equal` is used in `_need_reindex` to skip reindexing when indexes are already equal, serving as a performance optimization for the common case of pre-aligned objects",
      "Describes how `join='override'` follows a distinct code path: it sets `need_reindex = False`, takes the first object's index as `joined_index`, and uses `_overwrite_indexes` instead of reindexing, while `find_matching_indexes` validates that override indexes have the same size",
      "Explains the `_reindex_callback` delegation pattern: the Aligner delegates actual reindexing to each object via the `Alignable` protocol's `_reindex_callback` method, which accepts the aligner, dim_pos_indexers, variables, indexes, fill_value, exclude_dims, and exclude_vars",
      "Describes the `MatchingIndexKey` design: indexes are matched across objects using a key of `(CoordNamesAndDims, type[Index])`, ensuring only indexes of the same type with the same coordinate names and dimensions are compared",
      "Explains that `_get_index_joiner` uses `functools.reduce` with `functools.partial` for outer/inner joins to progressively merge indexes pairwise, while left/right joins use `operator.itemgetter` to simply pick the first or last index",
      "Mentions that negative values in positional indexers signal missing values: `reindex_variables` detects `(indxr < 0).any()` to identify `masked_dims` that need `_getitem_with_mask` for fill value handling",
      "Notes that the default fill value is `dtypes.NA` (a `ReprObject('<NA>')` sentinel), not literal `np.nan`, allowing downstream code to choose the appropriate NA value per dtype"
    ],
    "key_files": [
      "xarray/structure/alignment.py",
      "xarray/core/indexes.py",
      "xarray/core/types.py",
      "xarray/core/dtypes.py",
      "xarray/core/dataset.py"
    ],
    "source_doc": "[docstring: xarray/structure/alignment.py] xarray.structure.alignment.align\nGiven any number of Dataset and/or DataArray objects, returns new\n    objects with aligned indexes and dimension sizes.\n\n    Array from the aligned objects are suitable as input to mathematical\n    operators, because along each dimension they have the same index and size.\n\n    Missing values (if ``join != 'inner'``) are filled with ``fill_value``.\n    The default fill value is NaN.\n\n    Parameters\n    ----------\n    *objects : Dataset or DataArray\n        Objects to align.\n    join : {\"outer\", \"inner\", \"left\", \"right\", \"exact\", \"override\"}, optional\n        Method for joining the indexes of the passed objects along each\n        dimension:\n\n        - \"outer\": use the union of object indexes\n        - \"inner\": use the intersection of object indexes\n        - \"left\": use indexes from the first object with each dimension\n        - \"right\": use indexes from the last object with each dimension\n        - \"exact\": instead of aligning, raise `ValueError` when indexes to be\n          aligned are not equal\n        - \"override\": if indexes are of same size, rewrite indexes to be\n          those of the first object with that dimension. Indexes for the same\n          dimension must have the same size in all objects.\n\n    copy : bool, default: True\n        If ``copy=True``, data in the return values is always copied. If\n        ``copy=False`` and reindexing is unnecessary, or can be performed with\n        only slice operations, then the output may share memory with the input.\n        In either case, new xarray objects are always returned.\n    indexes : dict-like, optional\n        Any indexes explicitly provided with the `indexes` argument should be\n        used in preference to the aligned indexes.\n    exclude : str, iterable of hashable or None, optional\n        Dimensions that must be excluded from alignment\n    fill_value : scalar or dict-like, optional\n        Value to use for newly missing values. If a dict-like, maps\n        variable names to fill values. Use a data array's name to\n        refer to its values.\n\n    Returns\n    -------\n    aligned : tuple of DataArray or Dataset\n        Tuple of objects with the same type as `*objects` with aligned\n        coordinates.\n\n    Raises\n    ------\n    AlignmentError\n        If any dimensions without labels on the arguments have different sizes,\n        or a different size than the size of the aligned dimension labels.\n\n    Examples\n    --------\n    >>> x = xr.DataArray(\n    ...     [[25, 35], [10, 24]],\n    ...     dims=(\"lat\", \"lon\"),\n    ...     coords={\"lat\": [35.0, 40.0], \"lon\": [100.0, 120.0]},\n    ... )\n    >>> y = xr.DataArray(\n    ...     [[20, 5], [7, 13]],\n    ...     dims=(\"lat\", \"lon\"),\n    ...     coords={\"lat\": [35.0, 42.0], \"lon\": [100.0, 120.0]},\n    ... )\n\n    >>> x\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[25, 35],\n           [10, 24]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 40.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> y\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[20,  5],\n           [ 7, 13]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y)\n    >>> a\n    <xarray.DataArray (lat: 1, lon: 2)> Size: 16B\n    array([[25, 35]])\n    Coordinates:\n      * lat      (lat) float64 8B 35.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 1, lon: 2)> Size: 16B\n    array([[20,  5]])\n    Coordinates:\n      * lat      (lat) float64 8B 35.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y, join=\"outer\")\n    >>> a\n    <xarray.DataArray (lat: 3, lon: 2)> Size: 48B\n    array([[25., 35.],\n           [10., 24.],\n           [nan, nan]])\n    Coordinates:\n      * lat      (lat) float64 24B 35.0 40.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 3, lon: 2)> Size: 48B\n    array([[20.,  5.],\n           [nan, nan],\n           [ 7., 13.]])\n    Coordinates:\n      * lat      (lat) float64 24B 35.0 40.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y, join=\"outer\", fill_value=-999)\n    >>> a\n    <xarray.DataArray (lat: 3, lon: 2)> Size: 48B\n    array([[  25,   35],\n           [  10,   24],\n           [-999, -999]])\n    Coordinates:\n      * lat      (lat) float64 24B 35.0 40.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 3, lon: 2)> Size: 48B\n    array([[  20,    5],\n           [-999, -999],\n           [   7,   13]])\n    Coordinates:\n      * lat      (lat) float64 24B 35.0 40.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y, join=\"left\")\n    >>> a\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[25, 35],\n           [10, 24]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 40.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[20.,  5.],\n           [nan, nan]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 40.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y, join=\"right\")\n    >>> a\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[25., 35.],\n           [nan, nan]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[20,  5],\n           [ 7, 13]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 42.0\n      * lon      (lon) float64 16B 100.0 120.0\n\n    >>> a, b = xr.align(x, y, join=\"exact\")\n    Traceback (most recent call last):\n    ...\n    xarray.structure.alignment.AlignmentError: cannot align objects with join='exact' ...\n\n    >>> a, b = xr.align(x, y, join=\"override\")\n    >>> a\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[25, 35],\n           [10, 24]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 40.0\n      * lon      (lon) float64 16B 100.0 120.0\n    >>> b\n    <xarray.DataArray (lat: 2, lon: 2)> Size: 32B\n    array([[20,  5],\n           [ 7, 13]])\n    Coordinates:\n      * lat      (lat) float64 16B 35.0 40.0\n      * lon      (lon) float64 16B 100.0 120.0",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims contain design rationales from documentation (why certain patterns exist), while the code-level mechanics are verifiable from stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_27",
    "repo": "xarray",
    "question": "Where does data flow through the `concat` function, and how does it dispatch to different internal handlers based on input type?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `concat` function in `xarray/structure/concat.py` is the main entry point that dispatches to three internal functions based on the type of the first object: `_datatree_concat`, `_dataarray_concat`, or `_dataset_concat`. Here is the detailed data/control flow:\n\n**Dispatch Flow:**\nThe function uses `utils.peek_at(objs)` to get the first element without consuming the iterator. It then does an `isinstance` check on `first_obj` \u2014 checking `DataTree` first, then `DataArray`, then `Dataset`. If none match, it raises `TypeError`.\n\n**DataArray path (`_dataarray_concat`):**\nThis function handles DataArray concatenation. It is called when the first object is a DataArray instance.\n\n**Dataset path (`_dataset_concat`):**\n1. Calls `_calc_concat_dim_index(dim)` to determine `dim_name` and optionally create a `PandasIndex`. If `dim` is a `pd.Index` without a name, it defaults to `\"concat_dim\"`. If `dim` is a `Variable`/`DataArray`, it extracts the dim name from its `.dims` attribute.\n2. Calls `_parse_datasets` to gather all dims, dim_coords, dims_sizes, coord_names, data_vars names, and variables_order.\n3. Calls `_calc_concat_over` to determine which variables to concatenate vs. merge.\n4. `merge_collected` (imported from `xarray/structure/merge.py`) is available for merging variables not in `concat_over`.\n5. `PandasIndex.concat` uses `_concat_indexes` internally, which appends pd.Index objects via `pd_indexes[0].append(pd_indexes[1:])` and optionally reorders using positions.\n6. `concat_vars` (imported as the module-level `concat` from `xarray/core/variable.py`) is used for concatenating variables without indexes.\n7. `reindex_variables` from `xarray/structure/alignment.py` is available for handling missing variables, and its implementation handles masked dims (where indexer < 0) and unchanged dims.\n\n**DataTree path (`_datatree_concat`):**\n`_dataset_concat` has a `preexisting_dim: bool = False` keyword parameter that supports DataTree concatenation scenarios.\n\n**Deprecation defaults:**\nThe default parameter values use `CombineKwargDefault` objects from `xarray/util/deprecation_helpers.py`. The current (old) defaults are: `data_vars=\"all\"`, `coords=\"different\"`, `compat=\"equals\"`, `join=\"outer\"`. The planned new defaults are: `data_vars=None`, `coords=\"minimal\"`, `compat=\"override\"`, `join=\"exact\"`. Which value is used depends on the `OPTIONS[\"use_new_combine_kwarg_defaults\"]` setting.",
    "rubric": [
      "Mentions that `concat` uses `utils.peek_at(objs)` to inspect the first object and dispatches via `isinstance` checks on the first object",
      "Identifies the three dispatch targets: `_datatree_concat` for DataTree, `_dataarray_concat` for DataArray, and `_dataset_concat` for Dataset (with TypeError for anything else)",
      "Explains that `_dataset_concat` calls `_calc_concat_dim_index` to determine the dimension name and optional PandasIndex (defaulting unnamed pd.Index to 'concat_dim')",
      "Explains that `_dataset_concat` calls `_parse_datasets` to collect dims, dim_coords, dims_sizes, coord_names, data_vars names, and variables_order",
      "Explains that `_calc_concat_over` determines which variables to concatenate vs. merge",
      "Notes that `_dataarray_concat` converts arrays to temp datasets and delegates to `_dataset_concat`",
      "Notes that `_datatree_concat` iterates over tree nodes via `group_subtrees` and calls `_dataset_concat` per node with the `preexisting_dim` keyword",
      "Mentions that default parameter values use `CombineKwargDefault` objects whose resolved value depends on the `OPTIONS['use_new_combine_kwarg_defaults']` setting, with old defaults like data_vars='all', coords='different', compat='equals', join='outer' and new defaults like data_vars=None, coords='minimal', compat='override', join='exact'"
    ],
    "key_files": [
      "xarray/structure/concat.py",
      "xarray/core/variable.py",
      "xarray/core/indexes.py",
      "xarray/util/deprecation_helpers.py",
      "xarray/core/treenode.py",
      "xarray/structure/alignment.py"
    ],
    "source_doc": "[docstring: xarray/structure/concat.py] xarray.structure.concat.concat\nConcatenate xarray objects along a new or existing dimension.\n\n    Parameters\n    ----------\n    objs : sequence of DataArray, Dataset or DataTree\n        xarray objects to concatenate together. Each object is expected to\n        consist of variables and coordinates with matching shapes except for\n        along the concatenated dimension.\n    dim : Hashable or Variable or DataArray or pandas.Index\n        Name of the dimension to concatenate along. This can either be a new\n        dimension name, in which case it is added along axis=0, or an existing\n        dimension name, in which case the location of the dimension is\n        unchanged. If dimension is provided as a Variable, DataArray or Index, its name\n        is used as the dimension to concatenate along and the values are added\n        as a coordinate.\n    data_vars : {\"minimal\", \"different\", \"all\", None} or list of str, default: \"all\"\n        These data variables will be concatenated together:\n          * \"minimal\": Only data variables in which the dimension already\n            appears are included.\n          * \"different\": Data variables which are not equal (ignoring\n            attributes) across all datasets are also concatenated (as well as\n            all for which dimension already appears). Beware: this option may\n            load the data payload of data variables into memory if they are not\n            already loaded.\n          * \"all\": All data variables will be concatenated.\n          * None: Means ``\"all\"`` if ``dim`` is not present in any of the ``objs``,\n            and ``\"minimal\"`` if ``dim`` is present in any of ``objs``.\n          * list of str: The listed data variables will be concatenated, in\n            addition to the \"minimal\" data variables.\n\n        If objects are DataArrays, data_vars must be \"all\" or None.\n    coords : {\"minimal\", \"different\", \"all\"} or list of str, default: \"different\"\n        These coordinate variables will be concatenated together:\n          * \"minimal\": Only coordinates in which the dimension already appears\n            are included.\n          * \"different\": Coordinates which are not equal (ignoring attributes)\n            across all datasets are also concatenated (as well as all for which\n            dimension already appears). Beware: this option may load the data\n            payload of coordinate variables into memory if they are not already\n            loaded.\n          * \"all\": All coordinate variables will be concatenated, except\n            those corresponding to other dimensions.\n          * list of str: The listed coordinate variables will be concatenated,\n            in addition to the \"minimal\" coordinates.\n    compat : {\"identical\", \"equals\", \"broadcast_equals\", \"no_conflicts\", \"override\"},              default: \"equals\"\n        String indicating how to compare non-concatenated variables of the same name for\n        potential conflicts. This is passed down to merge.\n\n        - \"broadcast_equals\": all values must be equal when variables are\n          broadcast against each other to ensure common dimensions.\n        - \"equals\": all values and dimensions must be the same.\n        - \"identical\": all values, dimensions and attributes must be the\n          same.\n        - \"no_conflicts\": only values which are not null in both datasets\n          must be equal. The returned dataset then contains the combination\n          of all non-null values.\n        - \"override\": skip comparing and pick variable from first dataset\n    positions : None or list of integer arrays, optional\n        List of integer arrays which specifies the integer positions to which\n        to assign each dataset along the concatenated dimension. If not\n        supplied, objects are concatenated in the provided order.\n    fill_value : scalar or dict-like, optional\n        Value to use for newly missing values. If a dict-like, maps\n        variable names to fill values. Use a data array's name to\n        refer to its values.\n    join : {\"outer\", \"inner\", \"left\", \"right\", \"exact\"}, default: \"outer\"\n        String indicating how to combine differing indexes\n        (excluding dim) in objects\n\n        - \"outer\": use the union of object indexes\n        - \"inner\": use the intersection of object indexes\n        - \"left\": use indexes from the first object with each dimension\n        - \"right\": use indexes from the last object with each dimension\n        - \"exact\": instead of aligning, raise `ValueError` when indexes to be\n          aligned are not equal\n        - \"override\": if indexes are of same size, rewrite indexes to be\n          those of the first object with that dimension. Indexes for the same\n          dimension must have the same size in all objects.\n    combine_attrs : {\"drop\", \"identical\", \"no_conflicts\", \"drop_conflicts\",                      \"override\"} or callable, default: \"override\"\n        A callable or a string indicating how to combine attrs of the objects being\n        merged:\n\n        - \"drop\": empty attrs on returned Dataset.\n        - \"identical\": all attrs must be the same on every object.\n        - \"no_conflicts\": attrs from all objects are combined, any that have\n          the same name must also have the same value.\n        - \"drop_conflicts\": attrs from all objects are combined, any that have\n          the same name but different values are dropped.\n        - \"override\": skip comparing and copy attrs from the first dataset to\n          the result.\n\n        If a callable, it must expect a sequence of ``attrs`` dicts and a context object\n        as its only parameters.\n    create_index_for_new_dim : bool, default: True\n        Whether to create a new ``PandasIndex`` object when the objects being\n        concatenated contain scalar variables named ``dim``.\n\n    Returns\n    -------\n    concatenated : type of objs\n\n    See Also\n    --------\n    merge\n\n    Examples\n    --------\n    >>> da = xr.DataArray(\n    ...     np.arange(6).reshape(2, 3), [(\"x\", [\"a\", \"b\"]), (\"y\", [10, 20, 30])]\n    ... )\n    >>> da\n    <xarray.DataArray (x: 2, y: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * x        (x) <U1 8B 'a' 'b'\n      * y        (y) int64 24B 10 20 30\n\n    >>> xr.concat([da.isel(y=slice(0, 1)), da.isel(y=slice(1, None))], dim=\"y\")\n    <xarray.DataArray (x: 2, y: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * x        (x) <U1 8B 'a' 'b'\n      * y        (y) int64 24B 10 20 30\n\n    >>> xr.concat([da.isel(x=0), da.isel(x=1)], \"x\", coords=\"minimal\")\n    <xarray.DataArray (x: 2, y: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * x        (x) <U1 8B 'a' 'b'\n      * y        (y) int64 24B 10 20 30\n\n    >>> xr.concat([da.isel(x=0), da.isel(x=1)], \"new_dim\", coords=\"all\")\n    <xarray.DataArray (new_dim: 2, y: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n        x        (new_dim) <U1 8B 'a' 'b'\n      * y        (y) int64 24B 10 20 30\n    Dimensions without coordinates: new_dim\n\n    >>> xr.concat(\n    ...     [da.isel(x=0), da.isel(x=1)],\n    ...     pd.Index([-90, -100], name=\"new_dim\"),\n    ...     coords=\"all\",\n    ... )\n    <xarray.DataArray (new_dim: 2, y: 3)> Size: 48B\n    array([[0, 1, 2],\n           [3, 4, 5]])\n    Coordinates:\n      * new_dim  (new_dim) int64 16B -90 -100\n        x        (new_dim) <U1 8B 'a' 'b'\n      * y        (y) int64 24B 10 20 30\n\n    # Concatenate a scalar variable along a new dimension of the same name with and without creating a new index\n\n    >>> ds = xr.Dataset(coords={\"x\": 0})\n    >>> xr.concat([ds, ds], dim=\"x\")\n    <xarray.Dataset> Size: 16B\n    Dimensions:  (x: 2)\n    Coordinates:\n      * x        (x) int64 16B 0 0\n    Data variables:\n        *empty*\n\n    >>> xr.concat([ds, ds], dim=\"x\").indexes\n    Indexes:\n        x        Index([0, 0], dtype='int64', name='x')\n\n    >>> xr.concat([ds, ds], dim=\"x\", create_index_for_new_dim=False).indexes\n    Indexes:\n        *empty*",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims data_vars default is 'all' but the documentation states it is 'all' as the default parameter value, which is consistent.",
      "The answer mentions specific line numbers and internal implementation details (e.g., lines 286-341, 866-926, etc.) that cannot be verified from the documentation alone - these are plausible but unverifiable extrapolations.",
      "The answer mentions 'CombineKwargDefault' objects and planned new defaults (data_vars=None, coords='minimal', compat='override', join='exact') which cannot be verified from the documentation - this is a plausible implementation detail but could be a hallucination.",
      "The answer states that for DataArrays, data_vars must be 'all' or None, which is consistent with the documentation stating 'If objects are DataArrays, data_vars must be \"all\" or None.'",
      "The answer's description of the dim parameter behavior (e.g., pd.Index without name defaulting to 'concat_dim') cannot be verified from the documentation alone.",
      "The answer's description of Variable.concat using duck_array_ops.concatenate vs stack is plausible but unverifiable from the documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims about truncated function bodies (Variable.concat, _dataarray_concat internals, _datatree_concat internals) cannot be verified from the stripped code shown, while the dispatch logic, _calc_concat_dim_index, _calc_concat_over, _parse_datasets, PandasIndex.concat, and deprecation defaults are all verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_28",
    "repo": "xarray",
    "question": "How does `load_dataset` open a file, load data into memory, and ensure the file handle gets closed?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `load_dataset` function in `xarray/backends/api.py` works through the following mechanism:\n\n1. **Input Validation**: It first checks if 'cache' is passed in kwargs and raises a `TypeError(\"cache has no effect in this context\")` if it is.\n\n2. **Context Manager Pattern**: It uses a `with open_dataset(filename_or_obj, **kwargs) as ds:` statement, leveraging the Dataset's context manager protocol.\n\n3. **Loading into memory**: Inside the context manager, it calls `ds.load()` which is defined in the `Dataset` class in `xarray/core/dataset.py`. This method:\n   - First identifies all variables with chunked (e.g., dask) arrays via `is_chunked_array(v._data)`\n   - If chunked arrays exist, it uses the chunk manager's `compute()` to evaluate them all simultaneously (batch computation)\n   - Then loads all remaining (non-chunked) variables sequentially by calling `v.load()` on each `Variable`\n\n4. **File Closing**: After `ds.load()` returns, the `with` statement exits, triggering the context manager's exit which closes the file. The `_close` callable is set by `_dataset_from_backend_dataset` via `ds.set_close(backend_ds._close)`, connecting it to the backend's resource cleanup.\n\n5. **The `open_dataset` processing pipeline** that runs before `load()` involves:\n   - Engine selection via `plugins.guess_engine()` if no engine is specified\n   - Backend instantiation via `plugins.get_backend(engine)`\n   - Decoder resolution via `_resolve_decoders_kwargs()` which removes None-valued decoder kwargs and sets all to False if `decode_cf=False`\n   - Backend's `open_dataset()` call to get raw data\n   - `_dataset_from_backend_dataset()` which wraps variable data in `CopyOnWriteArray` (and additionally `MemoryCachedArray` if cache=True), creates default pandas indexes for dimension coordinates, optionally chunks with dask, sets the close handler, and stores the source path in `ds.encoding[\"source\"]`",
    "rubric": [
      "Mentions that load_dataset rejects a 'cache' kwarg by raising a TypeError",
      "Explains that it uses open_dataset inside a context manager (with statement)",
      "Explains that ds.load() is called inside the context manager to load data into memory",
      "Describes that ds.load() identifies chunked (e.g., dask) arrays and computes them in batch via the chunk manager's compute method",
      "Describes that ds.load() also loads non-chunked variables sequentially by calling v.load() on each",
      "Explains that exiting the context manager triggers __exit__ which calls close(), which invokes the _close callable set by _dataset_from_backend_dataset via set_close",
      "Mentions that _dataset_from_backend_dataset wraps variable data in CopyOnWriteArray (and MemoryCachedArray if cache=True)",
      "Mentions engine selection via guess_engine or get_backend and _resolve_decoders_kwargs processing in open_dataset"
    ],
    "key_files": [
      "xarray/backends/api.py",
      "xarray/core/dataset.py",
      "xarray/core/common.py",
      "xarray/core/indexing.py"
    ],
    "source_doc": "[docstring: xarray/backends/api.py] xarray.backends.api.load_dataset\nOpen, load into memory, and close a Dataset from a file or file-like\n    object.\n\n    This is a thin wrapper around :py:meth:`~xarray.open_dataset`. It differs\n    from `open_dataset` in that it loads the Dataset into memory, closes the\n    file, and returns the Dataset. In contrast, `open_dataset` keeps the file\n    handle open and lazy loads its contents. All parameters are passed directly\n    to `open_dataset`. See that documentation for further details.\n\n    Returns\n    -------\n    dataset : Dataset\n        The newly created Dataset.\n\n    See Also\n    --------\n    open_dataset",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.18,
    "strip_verify_summary": "A few claims rely on design rationale from docs/comments and internal implementation details not shown in the stripped code, but most claims are directly verifiable from the code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_29",
    "repo": "xarray",
    "question": "What does `load_dataarray` do, and how does its underlying `open_dataarray` function validate and restore DataArray names during roundtripping?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "`load_dataarray` is a function in `xarray/backends/api.py` that wraps `open_dataarray`. It opens a file containing a single data variable, loads all data into memory, closes the file handle, and returns the resulting `DataArray`. Specifically:\n\n1. **Implementation**: The function takes a `filename_or_obj` (typed as `T_PathFileOrDataStore`, imported from `xarray.backends.common`) and arbitrary `**kwargs`. It raises a `TypeError` if \"cache\" is passed in kwargs (the error message states \"cache has no effect in this context\"). It uses a context manager pattern: `with open_dataarray(filename_or_obj, **kwargs) as da: return da.load()`.\n\n2. **Context manager mechanism**: The `with` statement works because `DataArray` inherits from `DataWithCoords` (defined in `xarray/core/common.py`), which has a `_close` attribute defined in `__slots__`. The context manager protocol enables the file to be closed after the data is loaded.\n\n3. **`open_dataarray` internals**: This function actually calls `open_dataset` first, then validates that the dataset has exactly one data variable (raises `ValueError` if zero or more than one). It unpacks the single data variable, transfers the dataset's close function via `data_array.set_close(dataset._close)`, and performs roundtrip name restoration using two sentinel constants:\n   - `DATAARRAY_NAME = \"__xarray_dataarray_name__\"` - checked in dataset.attrs to restore the DataArray's name\n   - `DATAARRAY_VARIABLE = \"__xarray_dataarray_variable__\"` - if the data_array's name equals this constant, the name is set to None\n\n4. **`Dataset.load()` mechanism**: `Dataset.load()` handles chunked (dask) arrays by calling `chunkmanager.compute()` on all chunked variables simultaneously, then sequentially loads all non-chunked variables by calling `v.load()` on each.\n\n5. **Sibling functions**: `load_dataset` and `load_datatree` follow the exact same pattern (raise TypeError on \"cache\" kwarg, use context manager, call `.load()`).",
    "rubric": [
      "Explains that load_dataarray opens a file, loads all data into memory, closes the file handle, and returns the DataArray",
      "Mentions the context manager pattern: `with open_dataarray(...) as da: return da.load()`",
      "Notes that passing 'cache' in kwargs raises a TypeError with the message 'cache has no effect in this context'",
      "Explains that open_dataarray internally calls open_dataset and validates exactly one data variable exists (raises ValueError otherwise)",
      "Describes the name roundtrip logic using DATAARRAY_NAME ('__xarray_dataarray_name__') to restore the original DataArray name from dataset attrs",
      "Describes DATAARRAY_VARIABLE ('__xarray_dataarray_variable__') sentinel: if the data_array's name equals this constant, the name is set to None",
      "Mentions that set_close transfers the dataset's close function to the DataArray",
      "Notes that load_dataset and load_datatree follow the same pattern (TypeError on cache, context manager, .load())"
    ],
    "key_files": [
      "xarray/backends/api.py",
      "xarray/core/common.py",
      "xarray/core/dataarray.py",
      "xarray/core/dataset.py"
    ],
    "source_doc": "[docstring: xarray/backends/api.py] xarray.backends.api.load_dataarray\nOpen, load into memory, and close a DataArray from a file or file-like\n    object containing a single data variable.\n\n    This is a thin wrapper around :py:meth:`~xarray.open_dataarray`. It differs\n    from `open_dataarray` in that it loads the Dataset into memory, closes the\n    file, and returns the Dataset. In contrast, `open_dataarray` keeps the file\n    handle open and lazy loads its contents. All parameters are passed directly\n    to `open_dataarray`. See that documentation for further details.\n\n    Returns\n    -------\n    datarray : DataArray\n        The newly created DataArray.\n\n    See Also\n    --------\n    open_dataarray",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in purpose descriptions for sentinel constants and the DataArray.load() delegation mechanism.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_30",
    "repo": "xarray",
    "question": "Why does CachingFileManager have so many safeguards around file lifecycle management, like reference counting, non-blocking locks in __del__, and the atexit handler?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `CachingFileManager` class exists to solve several interconnected problems related to file management in xarray:\n\n1. **The mode='w' to mode='a' switch**: In `_acquire_with_cache_info`, after a file is opened with `mode='w'`, the mode is immediately changed to `'a'` (`if self._mode == \"w\": self._mode = \"a\"`). This prevents the file from being re-opened in write mode if it needs to be reopened later. The `_make_key` method also maps `'w'` to `'a'` when generating the cache key (`\"a\" if self._mode == \"w\" else self._mode`), ensuring that the cache key remains the same regardless of whether the mode switch has occurred yet.\n\n2. **Non-blocking lock acquisition in `__del__`**: The `__del__` method uses `acquire(self._lock, blocking=False)` and only closes the file if the lock is immediately available. If the lock cannot be acquired, the file is not closed in that call. This avoids potential deadlocks that could occur with blocking acquisition during garbage collection.\n\n3. **The `_remove_del_methods` atexit handler**: The `@atexit.register` decorated function `_remove_del_methods` deletes `CachingFileManager.__del__` and `PickleableFileManager.__del__` at program exit. This prevents the `__del__` methods from executing during interpreter shutdown when globals they depend on (like `OPTIONS`, `acquire`, or the lock) might already be unavailable.\n\n4. **Reference counting with `_RefCounter`**: The `_RefCounter` class tracks how many `CachingFileManager` instances reference the same cache key. In `__init__`, the ref count is incremented; in `__del__`, it's decremented. The file is only closed in `__del__` when the ref count reaches zero (`if not ref_count and self._key in self._cache`). Since `__setstate__` calls `__init__` (which increments the count), unpickling creates a new reference that prevents premature closing by the original instance's `__del__`.\n\n5. **`acquire_context` closes on exception for non-cached files only**: In `acquire_context`, if an exception occurs and the file was newly opened (not previously in the cache, `cached=False`), `self.close()` is called. If the file was already cached, it is not closed on exception, since other references may still need it.\n\n6. **The LRU cache `on_evict` callback**: The global `FILE_CACHE` is initialized with `on_evict=lambda k, v: v.close()`, which ensures files are properly closed when evicted from the cache due to capacity limits (as seen in `LRUCache._enforce_size_limit` which calls `self._on_evict(key, value)`).\n\n7. **Cache and ref_counts excluded from pickle state**: The `__getstate__` method returns only `(self._opener, self._args, self._mode, self._kwargs, lock, self._manager_id)`, omitting `_cache` and `_ref_counter`. On unpickling, `__setstate__` calls `__init__` again, which connects to the global `FILE_CACHE` and `REF_COUNTS` by default.\n\n8. **`_HashedSequence` class**: This class extends `list`, stores the tuple values, and caches the hash value in `self.hashvalue = hash(tuple_value)`. Its `__hash__` method simply returns this cached value, avoiding recomputation of the hash on every cache lookup (which happens on every `acquire()` call since `self._key` is used for cache access).",
    "rubric": [
      "Explains the mode='w' to mode='a' switch in _acquire_with_cache_info and why _make_key maps 'w' to 'a' to keep cache keys consistent",
      "Explains why __del__ uses non-blocking lock acquisition (acquire with blocking=False) to avoid deadlocks during garbage collection",
      "Explains that the _remove_del_methods atexit handler removes __del__ to prevent errors during interpreter shutdown when globals may be unavailable",
      "Explains _RefCounter tracks how many CachingFileManager instances reference the same cache key, and files are only closed when ref count reaches zero, which handles the pickle/unpickle case",
      "Explains that acquire_context only closes the file on exception if it was newly opened (not cached), to avoid closing files still needed by other references",
      "Explains the LRU cache on_evict callback (lambda k, v: v.close()) ensures files are closed when evicted due to capacity limits",
      "Explains that __getstate__ omits cache and ref_counts, and __setstate__ calls __init__ to reconnect to global FILE_CACHE and REF_COUNTS",
      "Explains _HashedSequence caches hash values to speed up repeated cache lookups"
    ],
    "key_files": [
      "xarray/backends/file_manager.py",
      "xarray/backends/lru_cache.py",
      "xarray/backends/locks.py",
      "xarray/core/options.py"
    ],
    "source_doc": "[docstring: xarray/backends/file_manager.py] xarray.backends.file_manager.CachingFileManager\nxarray.backends.file_manager.CachingFileManager:\n  Wrapper for automatically opening and closing file objects.\n\n    Unlike files, CachingFileManager objects can be safely pickled and passed\n    between processes. They should be explicitly closed to release resources,\n    but a per-process least-recently-used cache for open files ensures that you\n    can safely create arbitrarily large numbers of FileManager objects.\n\n    Don't directly close files acquired from a FileManager. Instead, call\n    FileManager.close(), which ensures that closed files are removed from the\n    cache as well.\n\n    Example usage::\n\n        manager = FileManager(open, \"example.txt\", mode=\"w\")\n        f = manager.acquire()\n        f.write(...)\n        manager.close()  # ensures file is closed\n\n    Note that as long as previous files are still cached, acquiring a file\n    multiple times from the same FileManager is essentially free::\n\n        f1 = manager.acquire()\n        f2 = manager.acquire()\n        assert f1 is f2\n\nxarray.backends.file_manager.CachingFileManager.__init__:\n  Initialize a CachingFileManager.\n\n        The cache, manager_id and ref_counts arguments exist solely to\n        facilitate dependency injection, and should only be set for tests.\n\n        Parameters\n        ----------\n        opener : callable\n            Function that when called like ``opener(*args, **kwargs)`` returns\n            an open file object. The file object must implement a ``close()``\n            method.\n        *args\n            Positional arguments for opener. A ``mode`` argument should be\n            provided as a keyword argument (see below). All arguments must be\n            hashable.\n        mode : optional\n            If provided, passed as a keyword argument to ``opener`` along with\n            ``**kwargs``. ``mode='w' `` has special treatment: after the first\n            call it is replaced by ``mode='a'`` in all subsequent function to\n            avoid overriding the newly created file.\n        kwargs : dict, optional\n            Keyword arguments for opener, excluding ``mode``. All values must\n            be hashable.\n        lock : duck-compatible threading.Lock, optional\n            Lock to use when modifying the cache inside acquire() and close().\n            By default, uses a new threading.Lock() object. If set, this object\n            should be pickleable.\n        cache : MutableMapping, optional\n            Mapping to use as a cache for open files. By default, uses xarray's\n            global LRU file cache. Because ``cache`` typically points to a\n            global variable and contains non-picklable file objects, an\n            unpickled FileManager objects will be restored with the default\n            cache.\n        manager_id : hashable, optional\n            Identifier for this CachingFileManager.\n        ref_counts : dict, optional\n            Optional dict to use for keeping track the number of references to\n            the same file.\n\nxarray.backends.file_manager.CachingFileManager._make_key:\n  Make a key for caching files in the LRU cache.\n\nxarray.backends.file_manager.CachingFileManager._optional_lock:\n  Context manager for optionally acquiring a lock.\n\nxarray.backends.file_manager.CachingFileManager.acquire:\n  Acquire a file object from the manager.\n\n        A new file is only opened if it has expired from the\n        least-recently-used cache.\n\n        This method uses a lock, which ensures that it is thread-safe. You can\n        safely acquire a file in multiple threads at the same time, as long as\n        the underlying file object is thread-safe.\n\n        Returns\n        -------\n        file-like\n            An open file object, as returned by ``opener(*args, **kwargs)``.\n\nxarray.backends.file_manager.CachingFileManager.acquire_context:\n  Context manager for acquiring a file.\n\nxarray.backends.file_manager.CachingFileManager._acquire_with_cache_info:\n  Acquire a file, returning the file and whether it was cached.\n\nxarray.backends.file_manager.CachingFileManager.close:\n  Explicitly close any associated file object (if necessary).\n\nxarray.backends.file_manager.CachingFileManager.__getstate__:\n  State for pickling.\n\nxarray.backends.file_manager.CachingFileManager.__setstate__:\n  Restore from a pickle.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., line 226-228, line 171, line 255, lines 445-450, etc.) that cannot be verified from the documentation provided \u2014 these appear to be from source code inspection rather than the given documentation.",
      "The answer describes details about `__del__` method behavior, `_remove_del_methods` atexit handler, `_RefCounter` class, `_HashedSequence` class, `on_evict` callback, and `acquire_context` exception handling \u2014 none of these are documented in the provided docstrings. These are plausible implementation details but cannot be verified against the given documentation.",
      "The claim about mode='w' being replaced by mode='a' is directly supported by the __init__ docstring: '``mode='w'`` has special treatment: after the first call it is replaced by ``mode='a'`` in all subsequent function to avoid overriding the newly created file.'",
      "The claim about cache and ref_counts being excluded from pickle state is partially supported \u2014 the doc mentions 'cache typically points to a global variable and contains non-picklable file objects, an unpickled FileManager objects will be restored with the default cache'.",
      "The claim that 'acquire_context closes on exception for non-cached files only' cannot be verified from the provided documentation, which only says 'Context manager for acquiring a file.'"
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from code but some design rationales (inspiration from functools, deadlock prevention reasoning) rely on documentation knowledge rather than code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_31",
    "repo": "xarray",
    "question": "Where is `_get_lock_maker` defined, what does it return for different schedulers, and where is the resulting lock logic used?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `_get_lock_maker` function is located in `xarray/backends/locks.py`. It is called exclusively from the `get_write_lock` function in the same file, which first calls `get_dask_scheduler()` to determine the current scheduler, then passes the result to `_get_lock_maker` to get the appropriate lock factory function, and finally calls that factory with a key (typically a filename).\n\n`_get_lock_maker` returns one of three lock-creating callables:\n1. `_get_threaded_lock` (for scheduler=None or 'threaded') - looks up/creates a `threading.Lock` in a module-level `WeakValueDictionary` called `_FILE_LOCKS`\n2. `_get_multiprocessing_lock` (for scheduler='multiprocessing') - creates a `multiprocessing.Lock()` (ignoring the key parameter via `del key`)\n3. `dask.distributed.Lock` (for scheduler='distributed') - lazily imported inside the elif block\n\nFor any other scheduler value, it raises a `KeyError`.\n\n`get_write_lock` is called from three backend store modules:\n- `xarray/backends/h5netcdf_.py`: combines HDF5_LOCK with the write lock for write mode\n- `xarray/backends/netCDF4_.py`: combines either NETCDF4_PYTHON_LOCK or NETCDFC_LOCK with the write lock\n- `xarray/backends/scipy_.py`: uses the write lock directly when mode != 'r' and the target is a string filename\n\nThe `Lock` type used throughout is a Protocol class defined in `xarray/core/types.py` requiring `acquire`, `release`, `__enter__`, and `__exit__` methods.",
    "rubric": [
      "_get_lock_maker is defined in xarray/backends/locks.py",
      "It is called from get_write_lock in the same file, which first determines the scheduler via get_dask_scheduler()",
      "Returns _get_threaded_lock for scheduler=None or 'threaded', which uses a WeakValueDictionary (_FILE_LOCKS) of threading.Lock instances",
      "Returns _get_multiprocessing_lock for scheduler='multiprocessing', which creates a multiprocessing.Lock() ignoring the key",
      "Returns dask.distributed.Lock (lazily imported) for scheduler='distributed'",
      "Raises KeyError for any unrecognized scheduler value",
      "get_write_lock is called from h5netcdf_.py, netCDF4_.py, and scipy_.py backend stores",
      "The Lock type is defined as a Protocol in xarray/core/types.py requiring acquire, release, __enter__, and __exit__ methods"
    ],
    "key_files": [
      "xarray/backends/locks.py",
      "xarray/core/types.py",
      "xarray/backends/netCDF4_.py",
      "xarray/backends/h5netcdf_.py",
      "xarray/backends/scipy_.py"
    ],
    "source_doc": "[docstring: xarray/backends/locks.py] xarray.backends.locks._get_lock_maker\nReturns an appropriate function for creating resource locks.\n\n    Parameters\n    ----------\n    scheduler : str or None\n        Dask scheduler being used.\n\n    See Also\n    --------\n    dask.utils.get_scheduler_lock",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage from line numbers and a comment about locket.py/import overhead rationale; core claims are all code-verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_32",
    "repo": "xarray",
    "question": "How does `extract_zarr_variable_encoding` resolve and validate chunk encoding for a zarr variable?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `extract_zarr_variable_encoding` function implements the following algorithm:\n\n1. **Copy encoding**: It starts by copying the variable's encoding dictionary (`variable.encoding.copy()`).\n\n2. **Drop safe keys**: It removes keys from the set `{\"source\", \"original_shape\", \"preferred_chunks\"}`.\n\n3. **Define valid encodings**: The set of valid zarr encoding keys is `{\"chunks\", \"shards\", \"compressor\", \"compressors\", \"filters\", \"serializer\", \"cache_metadata\", \"write_empty_chunks\", \"chunk_key_encoding\"}`. If `zarr_format == 3`, `\"fill_value\"` is also added to the valid set.\n\n4. **Handle invalid keys**: If `raise_on_invalid=True`, it collects all encoding keys not in `valid_encodings` and raises a `ValueError`. There's a special case: if `\"fill_value\"` is among the invalid keys and `zarr_format == 2`, the error message is appended with `\" Use _FillValue to set the Zarr array fill_value\"`. If `raise_on_invalid=False` (the default), invalid keys are silently removed.\n\n5. **Determine chunks**: It calls `_determine_zarr_chunks()` with `enc_chunks` (from encoding), `var_chunks` (the variable's dask chunks, if any), `ndim`, and `name`. This helper function:\n   - Returns `None` if neither encoding chunks nor variable chunks are present.\n   - If only variable (dask) chunks exist, validates that they are uniform (except for the final chunk), and that the final chunk is \u2264 the first chunk. Returns a tuple of the first chunk size per dimension.\n   - If encoding chunks are an integer, expands it to a tuple of length `ndim`. If the length of `enc_chunks_tuple` doesn't match `ndim`, it recursively calls itself with `enc_chunks=None`. It also validates that each element is an `int` (the check uses `isinstance(x, int)`).\n\n6. **Zarr v3 auto-chunk fallback**: After `_determine_zarr_chunks` returns, if `_zarr_v3()` is True (zarr version \u2265 3 is installed) and chunks is `None`, it sets chunks to `\"auto\"`.\n\n7. **Return**: The encoding dict is returned with the `\"chunks\"` key set to the resolved value.\n\nIn the calling context (`ZarrStore.set_variables`), the returned encoding is further augmented: for zarr v3, `encoding[\"dimension_names\"] = dims` is added; for zarr v2, dimensions are stored as an attribute under the key `\"_ARRAY_DIMENSIONS\"`. Also, `encoding[\"overwrite\"]` is set based on the write mode. The encoding's `\"shards\"` key (if present) takes precedence over `\"chunks\"` when determining `effective_write_chunks` for chunk alignment validation.",
    "rubric": [
      "Mentions copying the variable's encoding and removing safe-to-drop keys (source, original_shape, preferred_chunks)",
      "Lists the valid encoding keys (chunks, shards, compressor, compressors, filters, serializer, cache_metadata, write_empty_chunks, chunk_key_encoding) and notes fill_value is added for zarr_format==3",
      "Explains the raise_on_invalid behavior: raises ValueError for invalid keys when True, silently removes them when False; special error message for fill_value when zarr_format==2",
      "Describes _determine_zarr_chunks: returns None if no encoding or variable chunks exist; uses first chunk per dimension from uniform dask chunks; expands integer enc_chunks to a tuple of ndim length; recurses with None if length mismatches ndim",
      "Mentions that for zarr v3 (when _zarr_v3() is True), chunks defaults to 'auto' if _determine_zarr_chunks returns None",
      "Explains that in set_variables, encoding is further augmented with dimension_names (v3) or _ARRAY_DIMENSIONS attribute (v2), and encoding['overwrite'] is set based on mode",
      "Notes that shards takes precedence over chunks for effective_write_chunks used in chunk alignment validation"
    ],
    "key_files": [
      "xarray/backends/zarr.py",
      "xarray/namedarray/pycompat.py",
      "xarray/namedarray/utils.py"
    ],
    "source_doc": "[docstring: xarray/backends/zarr.py] xarray.backends.zarr.extract_zarr_variable_encoding\nExtract zarr encoding dictionary from xarray Variable\n\n    Parameters\n    ----------\n    variable : Variable\n    raise_on_invalid : bool, optional\n    name: str | Hashable, optional\n    zarr_format: Literal[2,3]\n    Returns\n    -------\n    encoding : dict\n        Zarr encoding for `variable`",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage from a TODO comment and one rationale explanation; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_33",
    "repo": "xarray",
    "question": "What is the full dependency chain when `open_zarr` is called, from chunk defaulting through store opening to dataset construction?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "`open_zarr` is a top-level convenience function (exposed as `xarray.open_zarr` via `xarray/__init__.py`) that delegates entirely to `xarray.backends.api.open_dataset` with `engine=\"zarr\"`. Here is how its dependency chain works:\n\n1. **Chunking default logic**: When `chunks` is not specified (uses the sentinel `_default = Default.token` from `xarray.core.utils`, where `Default` is an `Enum` with a single member `token = 0`), `open_zarr` calls `guess_chunkmanager(chunked_array_type)` from `xarray.namedarray.parallelcompat`. If a chunk manager is available (e.g., dask), it sets `chunks = {}`; if `ImportError` or `ValueError` is raised, it sets `chunks = None`. The `guess_chunkmanager` function checks `KNOWN_CHUNKMANAGERS` which maps `\"dask\"` \u2192 `\"dask\"`, `\"cubed\"` \u2192 `\"cubed-xarray\"`, and `\"arkouda\"` \u2192 `\"arkouda-xarray\"`.\n\n2. **Backend kwargs assembly**: `open_zarr` packages `synchronizer`, `consolidated`, `overwrite_encoded_chunks`, `chunk_store`, `storage_options`, `zarr_version`, and `zarr_format` into a `backend_kwargs` dict. Notably, `use_zarr_fill_value_as_mask` is passed directly as a keyword argument to `open_dataset`, not inside `backend_kwargs`.\n\n3. **ZarrBackendEntrypoint**: This class inherits from `BackendEntrypoint` (in `xarray/backends/common.py`) and sets `supports_groups = True`. Its `guess_can_open` method checks if the filename has a `.zarr` extension (stripping trailing slashes). Its `open_dataset` method calls `ZarrStore.open_group(...)` and then `StoreBackendEntrypoint().open_dataset(store, ...)`. Importantly, in `ZarrBackendEntrypoint.open_dataset`, `use_zarr_fill_value_as_mask` is passed as `None` to `ZarrStore.open_group` regardless of the user-specified value \u2014 the actual fill value masking is handled later.\n\n4. **ZarrStore** extends `AbstractWritableDataStore` and has 14 `__slots__` including `_cache_members`, `_members`, `_align_chunks`, and `_use_zarr_fill_value_as_mask`. When `cache_members=True` (the default), the constructor calls `_fetch_members()` to populate `_members` once; this cache is never updated. For zarr v3+, `_fetch_members` uses `dict(self.zarr_group.members())`, for v2 it uses `dict(self.zarr_group.items())`.\n\n5. **`_get_open_params`**: This function handles consolidated metadata with a fallback pattern: when `consolidated=None`, it tries `zarr.open_consolidated()` first, catches `ValueError` (zarr v3) or `KeyError` (zarr v2), then falls back to `zarr.open_group()` with a `RuntimeWarning`. The `mode='a-'` is treated specially \u2014 it's converted to `'a'` for zarr's `open_kwargs`. For zarr v3, if the store doesn't support consolidated metadata (`supports_consolidated_metadata` attribute is False), both `consolidated` and `consolidate_on_close` are set to `False`.\n\n6. **`_get_zarr_dims_and_attrs`**: Dimension discovery follows a priority chain: first tries zarr V3 native `metadata.dimension_names`, then falls back to the `_ARRAY_DIMENSIONS` attribute key, and finally tries NCZarr format (reading `.zarray` JSON for `_NCZARR_ARRAY.dimrefs`) but only when `try_nczarr` is True (which is set when `mode == 'r'`). NCZarr dimensions use Fully Qualified Names, extracting just `os.path.basename(dim)`. Attributes are wrapped in a `HiddenKeyDict` that hides both the `_ARRAY_DIMENSIONS` key and any attributes starting with `_nc` (case-insensitive).\n\n7. **`ZarrArrayWrapper`** extends `BackendArray` and wraps a zarr array. It preserves vlen string object dtype by checking for `vlen-utf8` codec (filter in v2, serializer in v3). It supports async indexing methods (`_async_getitem`, `_async_oindex`, `_async_vindex`) that access `self._array._async_array` for zarr v3. `_async_oindex` and `_async_vindex` require zarr >= 3.1.2 (checked via `has_zarr_async_index` which tests for `zarr.AsyncArray.oindex`).\n\n8. **StoreBackendEntrypoint.open_dataset** (in `xarray/backends/store.py`): calls `filename_or_obj.load()` and `get_encoding()`, then applies `conventions.decode_cf_variables()`. Variables that appear in `coord_names` or are 1D with their name matching their dimension are promoted to coordinates. The resulting `Coordinates` object is created with `indexes={}` (no indexes), and the Dataset's close method is set to `store.close()`.\n\n9. **ZarrStore.close**: Only calls `self.zarr_group.store.close()` if `_close_store_on_close` is True. This flag is set in `_get_open_params` based on `zarr_group.store is not store` (i.e., if zarr created a new store wrapper, it should be closed).\n\n10. **`open_store_variable`**: When `_use_zarr_fill_value_as_mask` is True, the zarr array's `fill_value` is injected as the `_FillValue` attribute (triggering CF decoding). When False, if `_FillValue` is already present in the original attributes, it's decoded via `FillValueCoder.decode()`. The encoding dict includes `preferred_chunks` as a dict mapping dimension names to chunk sizes.",
    "rubric": [
      "Explains that open_zarr delegates to open_dataset with engine='zarr', packaging parameters like synchronizer, consolidated, storage_options into backend_kwargs while passing use_zarr_fill_value_as_mask directly",
      "Describes the chunk defaulting logic: when chunks is the sentinel default, calls guess_chunkmanager and sets chunks={} if available, else chunks=None on ImportError/ValueError",
      "Mentions KNOWN_CHUNKMANAGERS mapping (dask\u2192dask, cubed\u2192cubed-xarray, arkouda\u2192arkouda-xarray)",
      "Explains ZarrBackendEntrypoint (supports_groups=True, guess_can_open checks .zarr extension) creates a ZarrStore via open_group and passes use_zarr_fill_value_as_mask=None to ZarrStore.open_group regardless of user input",
      "Describes _get_open_params: consolidated=None tries zarr.open_consolidated first then falls back to zarr.open_group on ValueError/KeyError with RuntimeWarning; mode='a-' is converted to 'a'",
      "Explains ZarrStore has 14 __slots__, caches members via _fetch_members on init when cache_members=True; _fetch_members uses .members() for zarr v3 and .items() for v2",
      "Describes _get_zarr_dims_and_attrs dimension priority: zarr V3 metadata.dimension_names first, then _ARRAY_DIMENSIONS attribute, then NCZarr (.zarray JSON dimrefs) when try_nczarr=True; attributes hidden via HiddenKeyDict",
      "Explains open_store_variable: when _use_zarr_fill_value_as_mask is True, injects fill_value as _FillValue; when False and _FillValue exists, decodes it via FillValueCoder.decode(); encoding includes preferred_chunks",
      "Describes StoreBackendEntrypoint.open_dataset: calls load() and get_encoding(), applies decode_cf_variables, promotes coord_names and 1D variables matching their dimension to coordinates with indexes={}",
      "Explains ZarrStore.close only calls zarr_group.store.close() when _close_store_on_close is True, which is set based on whether zarr_group.store is not the original store"
    ],
    "key_files": [
      "xarray/backends/zarr.py",
      "xarray/backends/store.py",
      "xarray/backends/common.py",
      "xarray/namedarray/parallelcompat.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/backends/zarr.py] xarray.backends.zarr.open_zarr\nLoad and decode a dataset from a Zarr store.\n\n    The `store` object should be a valid store for a Zarr group. `store`\n    variables must contain dimension metadata encoded in the\n    `_ARRAY_DIMENSIONS` attribute or must have NCZarr format.\n\n    Parameters\n    ----------\n    store : MutableMapping or str\n        A MutableMapping where a Zarr Group has been stored or a path to a\n        directory in file system where a Zarr DirectoryStore has been stored.\n    synchronizer : object, optional\n        Array synchronizer provided to zarr\n    group : str, optional\n        Group path. (a.k.a. `path` in zarr terminology.)\n    chunks : int, dict, \"auto\" or None, optional\n        Used to load the data into dask arrays. Default behavior is to use\n        ``chunks={}`` if dask is available, otherwise ``chunks=None``.\n\n        - ``chunks='auto'`` will use dask ``auto`` chunking taking into account the\n          engine preferred chunks.\n        - ``chunks=None`` skips using dask. This uses xarray's internally private\n          :ref:`lazy indexing classes <internal design.lazy indexing>`,\n          but data is eagerly loaded into memory as numpy arrays when accessed.\n          This can be more efficient for smaller arrays, though results may vary.\n        - ``chunks=-1`` loads the data with dask using a single chunk for all arrays.\n        - ``chunks={}`` loads the data with dask using engine preferred chunks if\n          exposed by the backend, otherwise with a single chunk for all arrays.\n\n        See dask chunking for more details.\n    overwrite_encoded_chunks : bool, optional\n        Whether to drop the zarr chunks encoded for each variable when a\n        dataset is loaded with specified chunk sizes (default: False)\n    decode_cf : bool, optional\n        Whether to decode these variables, assuming they were saved according\n        to CF conventions.\n    mask_and_scale : bool, optional\n        If True, replace array values equal to `_FillValue` with NA and scale\n        values according to the formula `original_values * scale_factor +\n        add_offset`, where `_FillValue`, `scale_factor` and `add_offset` are\n        taken from variable attributes (if they exist).  If the `_FillValue` or\n        `missing_value` attribute contains multiple values a warning will be\n        issued and all array values matching one of the multiple values will\n        be replaced by NA.\n    decode_times : bool, optional\n        If True, decode times encoded in the standard NetCDF datetime format\n        into datetime objects. Otherwise, leave them encoded as numbers.\n    concat_characters : bool, optional\n        If True, concatenate along the last dimension of character arrays to\n        form string arrays. Dimensions will only be concatenated over (and\n        removed) if they have no corresponding variable and if they are only\n        used as the last dimension of character arrays.\n    decode_coords : bool or {\"coordinates\", \"all\"}, optional\n        Controls which variables are set as coordinate variables:\n\n        - \"coordinates\" or True: Set variables referred to in the\n          ``'coordinates'`` attribute of the datasets or individual variables\n          as coordinate variables.\n        - \"all\": Set variables referred to in  ``'grid_mapping'``, ``'bounds'`` and\n          other attributes as coordinate variables.\n\n        Only existing variables can be set as coordinates. Missing variables\n        will be silently ignored.\n    drop_variables : str or iterable, optional\n        A variable or list of variables to exclude from being parsed from the\n        dataset. This may be useful to drop variables with problems or\n        inconsistent values.\n    consolidated : bool, optional\n        Whether to open the store using zarr's consolidated metadata\n        capability. Only works for stores that have already been consolidated.\n        By default (`consolidate=None`), attempts to read consolidated metadata,\n        falling back to read non-consolidated metadata if that fails.\n\n        When the experimental ``zarr_version=3``, ``consolidated`` must be\n        either be ``None`` or ``False``.\n    chunk_store : MutableMapping, optional\n        A separate Zarr store only for chunk data.\n    storage_options : dict, optional\n        Any additional parameters for the storage backend (ignored for local\n        paths).\n    decode_timedelta : bool, optional\n        If True, decode variables and coordinates with time units in\n        {'days', 'hours', 'minutes', 'seconds', 'milliseconds', 'microseconds'}\n        into timedelta objects. If False, leave them encoded as numbers.\n        If None (default), assume the same value of decode_time.\n    use_cftime : bool, optional\n        Only relevant if encoded dates come from a standard calendar\n        (e.g. \"gregorian\", \"proleptic_gregorian\", \"standard\", or not\n        specified).  If None (default), attempt to decode times to\n        ``np.datetime64[ns]`` objects; if this is not possible, decode times to\n        ``cftime.datetime`` objects. If True, always decode times to\n        ``cftime.datetime`` objects, regardless of whether or not they can be\n        represented using ``np.datetime64[ns]`` objects.  If False, always\n        decode times to ``np.datetime64[ns]`` objects; if this is not possible\n        raise an error.\n    zarr_version : int or None, optional\n\n        .. deprecated:: 2024.9.1\n           Use ``zarr_format`` instead.\n\n    zarr_format : int or None, optional\n        The desired zarr format to target (currently 2 or 3). The default\n        of None will attempt to determine the zarr version from ``store`` when\n        possible, otherwise defaulting to the default version used by\n        the zarr-python library installed.\n    use_zarr_fill_value_as_mask : bool, optional\n        If True, use the zarr Array ``fill_value`` to mask the data, the same as done\n        for NetCDF data with ``_FillValue`` or ``missing_value`` attributes. If False,\n        the ``fill_value`` is ignored and the data are not masked. If None, this defaults\n        to True for ``zarr_version=2`` and False for ``zarr_version=3``.\n    chunked_array_type: str, optional\n        Which chunked array type to coerce this datasets' arrays to.\n        Defaults to 'dask' if installed, else whatever is registered via the `ChunkManagerEntryPoint` system.\n        Experimental API that should not be relied upon.\n    from_array_kwargs: dict, optional\n        Additional keyword arguments passed on to the ``ChunkManagerEntrypoint.from_array`` method used to create\n        chunked arrays, via whichever chunk manager is specified through the ``chunked_array_type`` kwarg.\n        Defaults to ``{'manager': 'dask'}``, meaning additional kwargs will be passed eventually to\n        :py:func:`dask.array.from_array`. Experimental API that should not be relied upon.\n    create_default_indexes : bool, default: True\n        If True, create pandas indexes for :term:`dimension coordinates <dimension coordinate>`,\n        which loads the coordinate data into memory. Set it to False if you want to avoid loading\n        data into memory.\n\n        Note that backends can still choose to create other indexes. If you want to control that,\n        please refer to the backend's documentation.\n\n    Returns\n    -------\n    dataset : Dataset\n        The newly created dataset.\n\n    See Also\n    --------\n    open_dataset\n    open_mfdataset\n\n    References\n    ----------\n    https://zarr.readthedocs.io/",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_34",
    "repo": "xarray",
    "question": "Why does `convert_calendar` use both vectorized NumPy operations and element-wise Python loops, and what other performance trade-offs are made throughout the calendar conversion code?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `convert_calendar` function in `xarray/coding/calendar_ops.py` contains several performance-related design decisions:\n\n1. **Early return optimization**: The function returns immediately (without copying data) if the source calendar already matches the target calendar AND the dtype representation matches (numpy vs cftime). The condition `is_np_datetime_like(time.dtype) ^ use_cftime` checks via XOR that conversion is truly needed, avoiding unnecessary data copying.\n\n2. **Vectorized computation in `_interpolate_day_of_year` vs. element-wise loop**: The `_interpolate_day_of_year` function uses fully vectorized NumPy operations (`np.round`, division, multiplication on arrays) to compute new day-of-year values in O(n) time without Python loops. However, `_convert_to_new_calendar_with_new_day_of_year` is called in a Python list comprehension loop, where each date is individually constructed through `cftime.num2date` and date type constructors.\n\n3. **`convert_times` is called with `raise_on_invalid=False`**: In the `align_on == \"date\"` path, `convert_times` is called with `raise_on_invalid=False`, and results are filtered with `out.sel({dim: out[dim].notnull()})` to handle invalid dates by replacing them with null values.\n\n4. **`_is_leap_year` uses `np.vectorize`**: This wraps `cftime.is_leap_year` in `np.vectorize`, providing array broadcasting semantics. The `_days_in_year` function shortcuts entirely for `360_day` calendars using `full_like(years, 360)` to avoid the `np.vectorize` call entirely.\n\n5. **`_random_day_of_year` uses `groupby().map()`**: The random alignment is applied per-year via `time.groupby(f\"{dim}.year\").map(_random_day_of_year, ...)`. Inside `_random_day_of_year`, `np.random.default_rng().integers(0, 72, 5)` selects 5 random indices within five 72-day periods (`72 * np.arange(5)`), and the function handles leap year differences between source and target calendars.\n\n6. **Duplicate removal with `np.unique`**: After the year/random alignment, the code uses `np.unique(out[dim], return_index=True)[1]` via `out.isel(...)` to deduplicate dates post-hoc via sorted unique indices.\n\n7. **`_decimal_year` uses `apply_ufunc` with `dask='parallelized'`**: For the `interp_calendar` function, decimal year computation is made dask-compatible by using `apply_ufunc` with `vectorize=True` and `dask='parallelized'`, enabling lazy/parallel evaluation on chunked arrays.\n\n8. **`_should_cftime_be_used` is called to determine cftime usage**: The function `_should_cftime_be_used(time, calendar, use_cftime)` is called to determine whether cftime should be used. Helper functions `_is_standard_calendar` and `_is_numpy_compatible_time_range` exist in `times.py` and are used to check calendar compatibility and numpy datetime64 range compatibility.",
    "rubric": [
      "Mentions the early return optimization when source calendar matches target calendar and dtype representation matches (using XOR check on is_np_datetime_like and use_cftime)",
      "Explains that `_interpolate_day_of_year` uses vectorized NumPy operations (np.round, array division/multiplication) while `_convert_to_new_calendar_with_new_day_of_year` runs in a Python list comprehension loop constructing dates individually",
      "Notes that `convert_times` is called with `raise_on_invalid=False` in the 'date' alignment path, filtering invalid dates via `notnull()` selection",
      "Identifies that `_is_leap_year` uses `np.vectorize` to wrap `cftime.is_leap_year`, and `_days_in_year` shortcuts for 360_day calendars with `full_like(years, 360)`",
      "Describes that `_random_day_of_year` is applied per-year via `groupby().map()` and uses `np.random.default_rng().integers(0, 72, 5)` to select random indices within five 72-day periods",
      "Mentions duplicate removal using `np.unique(out[dim], return_index=True)[1]` after year/random alignment",
      "Explains that `_decimal_year` uses `apply_ufunc` with `vectorize=True` and `dask='parallelized'` for dask-compatible lazy/parallel evaluation",
      "Notes that `_should_cftime_be_used` determines cftime usage by checking `_is_standard_calendar` and `_is_numpy_compatible_time_range`"
    ],
    "key_files": [
      "xarray/coding/calendar_ops.py",
      "xarray/coding/times.py",
      "xarray/coding/cftime_offsets.py"
    ],
    "source_doc": "[docstring: xarray/coding/calendar_ops.py] xarray.coding.calendar_ops.convert_calendar\nTransform a time-indexed Dataset or DataArray to one that uses another calendar.\n\n    This function only converts the individual timestamps; it does not modify any\n    data except in dropping invalid/surplus dates, or inserting values for missing dates.\n\n    If the source and target calendars are both from a standard type, only the\n    type of the time array is modified. When converting to a calendar with a\n    leap year from to a calendar without a leap year, the 29th of February will\n    be removed from the array. In the other direction the 29th of February will\n    be missing in the output, unless `missing` is specified, in which case that\n    value is inserted. For conversions involving the `360_day` calendar, see Notes.\n\n    This method is safe to use with sub-daily data as it doesn't touch the time\n    part of the timestamps.\n\n    Parameters\n    ----------\n    obj : DataArray or Dataset\n      Input DataArray or Dataset with a time coordinate of a valid dtype\n      (:py:class:`numpy.datetime64`  or :py:class:`cftime.datetime`).\n    calendar : str\n      The target calendar name.\n    dim : str\n      Name of the time coordinate in the input DataArray or Dataset.\n    align_on : {None, 'date', 'year', 'random'}\n      Must be specified when either the source or target is a `\"360_day\"`\n      calendar; ignored otherwise. See Notes.\n    missing : any, optional\n      By default, i.e. if the value is None, this method will simply attempt\n      to convert the dates in the source calendar to the same dates in the\n      target calendar, and drop any of those that are not possible to\n      represent.  If a value is provided, a new time coordinate will be\n      created in the target calendar with the same frequency as the original\n      time coordinate; for any dates that are not present in the source, the\n      data will be filled with this value.  Note that using this mode requires\n      that the source data have an inferable frequency; for more information\n      see :py:func:`xarray.infer_freq`.  For certain frequency, source, and\n      target calendar combinations, this could result in many missing values, see notes.\n    use_cftime : bool, optional\n      Whether to use cftime objects in the output, only used if `calendar` is\n      one of {\"proleptic_gregorian\", \"gregorian\" or \"standard\"}.\n      If True, the new time axis uses cftime objects.\n      If None (default), it uses :py:class:`numpy.datetime64` values if the date\n          range permits it, and :py:class:`cftime.datetime` objects if not.\n      If False, it uses :py:class:`numpy.datetime64`  or fails.\n\n    Returns\n    -------\n      Copy of source with the time coordinate converted to the target calendar.\n      If `missing` was None (default), invalid dates in the new calendar are\n      dropped, but missing dates are not inserted.\n      If `missing` was given, the new data is reindexed to have a time axis\n      with the same frequency as the source, but in the new calendar; any\n      missing datapoints are filled with `missing`.\n\n    Notes\n    -----\n    Passing a value to `missing` is only usable if the source's time coordinate as an\n    inferable frequencies (see :py:func:`~xarray.infer_freq`) and is only appropriate\n    if the target coordinate, generated from this frequency, has dates equivalent to the\n    source. It is usually **not** appropriate to use this mode with:\n\n    - Period-end frequencies: 'A', 'Y', 'Q' or 'M', in opposition to 'AS' 'YS', 'QS' and 'MS'\n    - Sub-monthly frequencies that do not divide a day evenly: 'W', 'nD' where `n != 1`\n      or 'mH' where 24 % m != 0).\n\n    If one of the source or target calendars is `\"360_day\"`, `align_on` must\n    be specified and two options are offered.\n\n    \"year\"\n      The dates are translated according to their relative position in the year,\n      ignoring their original month and day information, meaning that the\n      missing/surplus days are added/removed at regular intervals.\n\n      From a `360_day` to a standard calendar, the output will be missing the\n      following dates (day of year in parentheses):\n        To a leap year:\n          January 31st (31), March 31st (91), June 1st (153), July 31st (213),\n          September 31st (275) and November 30th (335).\n        To a non-leap year:\n          February 6th (36), April 19th (109), July 2nd (183),\n          September 12th (255), November 25th (329).\n\n      From a standard calendar to a `\"360_day\"`, the following dates in the\n      source array will be dropped:\n        From a leap year:\n          January 31st (31), April 1st (92), June 1st (153), August 1st (214),\n          September 31st (275), December 1st (336)\n        From a non-leap year:\n          February 6th (37), April 20th (110), July 2nd (183),\n          September 13th (256), November 25th (329)\n\n      This option is best used on daily and subdaily data.\n\n    \"date\"\n      The month/day information is conserved and invalid dates are dropped\n      from the output. This means that when converting from a `\"360_day\"` to a\n      standard calendar, all 31sts (Jan, March, May, July, August, October and\n      December) will be missing as there is no equivalent dates in the\n      `\"360_day\"` calendar and the 29th (on non-leap years) and 30th of February\n      will be dropped as there are no equivalent dates in a standard calendar.\n\n      This option is best used with data on a frequency coarser than daily.\n\n    \"random\"\n      Similar to \"year\", each day of year of the source is mapped to another day of year\n      of the target. However, instead of having always the same missing days according\n      the source and target years, here 5 days are chosen randomly, one for each fifth\n      of the year. However, February 29th is always missing when converting to a leap year,\n      or its value is dropped when converting from a leap year. This is similar to the method\n      used in the LOCA dataset (see Pierce, Cayan, and Thrasher (2014). doi:10.1175/JHM-D-14-0082.1).\n\n      This option is best used on daily data.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer references specific line numbers (e.g., line 165, 247, 278, 854, 236, 191, 206, 337, 919) that cannot be verified from the documentation provided - these could be hallucinated.",
      "The answer describes internal implementation details (vectorized operations, Python loops, np.vectorize, groupby().map(), np.unique deduplication, apply_ufunc, etc.) that are not present in the documentation and cannot be verified from it.",
      "The documentation describes 'random' alignment as choosing 5 days randomly within five periods of the year. The answer claims these are '5 random indices within five 72-day periods' using np.random.default_rng().integers(0, 72, 5) - the 72-day period detail is not in the docs and could be hallucinated.",
      "The answer's claim about _interpolate_day_of_year, _convert_to_new_calendar_with_new_day_of_year, convert_times, _is_leap_year, _days_in_year, _decimal_year, _should_cftime_be_used functions cannot be verified from the provided documentation.",
      "The documentation states 'one for each fifth of the year' for the random method, which is consistent with the answer's '5 random indices' claim, but the specific implementation details (72-day periods, np.random.default_rng) are unverifiable."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for design rationale explanations and references to truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_35",
    "repo": "xarray",
    "question": "Where is `_get_day_of_month` defined and called from, and which offset subclasses actually supply it with meaningful day options?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `_get_day_of_month` function is defined at line 237 in `xarray/coding/cftime_offsets.py`. It is called from four distinct locations, all within the same file:\n\n1. **`BaseCFTimeOffset._get_offset_day`** (line 194-197): This is the method on the base class that delegates to `_get_day_of_month(other, self._day_option)`. The base class sets `_day_option: ClassVar[DayOption | None] = None` (line 110), so calling `_get_offset_day` on `BaseCFTimeOffset` directly will raise `NotImplementedError` (since `_get_day_of_month` checks for `None` at line 258).\n\n2. **`roll_qtrday`** (lines 348 and 354): This free function calls `_get_day_of_month(other, day_option)` twice \u2014 once for the `n > 0` branch and once for the `n <= 0` branch \u2014 to compare `other.day` against the offset reference day when deciding whether to adjust the number of periods.\n\n3. **`YearOffset.__apply__`** (line 532): Calls `_get_day_of_month(other, self._day_option)` to compute a `reference_day` which is then passed to `_adjust_n_years`.\n\n4. **`QuarterOffset.onOffset`** (line 447): Indirectly calls it via `self._get_offset_day(date)`, which delegates to `_get_day_of_month`.\n\nThe subclasses that set `_day_option` (and thus provide meaningful values for `_get_day_of_month`) are:\n- `QuarterBegin` (`_day_option = \"start\"`, line 481)\n- `QuarterEnd` (`_day_option = \"end\"`, line 506)\n- `YearBegin` (`_day_option = \"start\"`, line 564)\n- `YearEnd` (`_day_option = \"end\"`, line 589)\n\nThe `DayOption` type alias is defined at line 80 as `Literal[\"start\", \"end\"]`. When `day_option` is `\"end\"`, the function returns `other.daysinmonth` (a property on cftime datetime objects). When `day_option` is `\"start\"`, it simply returns `1`. Notably, `MonthBegin` and `MonthEnd` do NOT use `_get_day_of_month` \u2014 they hardcode their logic directly (e.g., `MonthEnd.__apply__` at line 394 passes `other.daysinmonth` directly to `_adjust_n_months`).\n\nThe function is not imported or called from any other module in xarray \u2014 it is purely internal to `xarray/coding/cftime_offsets.py`.",
    "rubric": [
      "Identifies that `_get_day_of_month` is defined in `xarray/coding/cftime_offsets.py`",
      "Identifies the call in `BaseCFTimeOffset._get_offset_day` which delegates to `_get_day_of_month(other, self._day_option)`",
      "Identifies two calls in `roll_qtrday` (one for n > 0 branch, one for n <= 0 branch)",
      "Identifies the call in `YearOffset.__apply__` to compute a `reference_day`",
      "Identifies the indirect call via `QuarterOffset.onOffset` through `_get_offset_day`",
      "Lists the subclasses that set `_day_option`: QuarterBegin ('start'), QuarterEnd ('end'), YearBegin ('start'), YearEnd ('end')",
      "Notes that the base class sets `_day_option = None`, causing `NotImplementedError` if called directly",
      "Explains that 'start' returns 1 and 'end' returns `other.daysinmonth`",
      "Notes that MonthBegin/MonthEnd do NOT use `_get_day_of_month`",
      "Notes the function is internal to `cftime_offsets.py` and not imported elsewhere"
    ],
    "key_files": [
      "xarray/coding/cftime_offsets.py",
      "xarray/tests/test_cftime_offsets.py"
    ],
    "source_doc": "[docstring: xarray/coding/cftime_offsets.py] xarray.coding.cftime_offsets._get_day_of_month\nFind the day in `other`'s month that satisfies a BaseCFTimeOffset's\n    onOffset policy, as described by the `day_option` argument.\n\n    Parameters\n    ----------\n    other : cftime.datetime\n    day_option : 'start', 'end'\n        'start': returns 1\n        'end': returns last day of the month\n\n    Returns\n    -------\n    day_of_month : int",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims are verifiable from stripped code; only specific line numbers needed removal as they may shift when comments/docstrings are stripped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_36",
    "repo": "xarray",
    "question": "How does the cftime date range generator step through dates when given a frequency string, including how it handles alignment and direction?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `_generate_linear_date_range_with_freq` function in `xarray/coding/cftime_offsets.py` generates cftime.datetime objects using an iterative stepping algorithm. Here's how it works:\n\n1. **Frequency conversion**: The `freq` string parameter is first converted to a `BaseCFTimeOffset` subclass instance via `to_offset(freq)`. The `to_offset` function uses a regex pattern `_PATTERN` (matching `^((?P<multiple>[+-]?\\d+)|())(?P<freq>(...))$`) to parse the frequency string into a multiplier and frequency type, then looks up the appropriate offset class from the `_FREQUENCIES` dictionary.\n\n2. **Start date alignment**: If `start` is provided, the function rolls it to the nearest valid date boundary. For positive offsets (`offset.n >= 0`), it uses `offset.rollforward(start)`; for negative offsets, it uses `offset.rollback(start)`.\n\n3. **Edge case for empty ranges**: If `periods` is None and `end < start` with a non-negative offset, it sets `end = None` and `periods = 0`, resulting in an empty generator.\n\n4. **Computing missing bounds**: If `end` is None, it's computed as `start + (periods - 1) * offset`. If `start` is None, it's computed as `end - (periods - 1) * offset`.\n\n5. **Iteration with safety checks**: For positive offsets, the function yields dates while `current <= end`, advancing by `current + offset`. It raises a `ValueError` if the next date isn't strictly greater than current (\"Offset {offset} did not increment date\"). For negative offsets, it iterates while `current >= end` and checks that each step strictly decreases.\n\n6. **Integration with `_cftime_range`**: The generator output is materialized into a numpy array via `np.array(list(...))` in `_cftime_range`, which also handles the `inclusive` parameter by trimming start/end dates as needed, and wraps the result in a `CFTimeIndex`.\n\n7. **`rollforward`/`rollback` implementations**: The base `BaseCFTimeOffset.rollforward` checks `self.onOffset(date)` and if not on offset, advances by `type(self)()` (a length-1 offset). Specialized subclasses like `YearBegin`, `YearEnd`, `QuarterBegin`, and `QuarterEnd` override these methods to use their specific month parameter. The `Day` class has `onOffset` always returning `True`, meaning rollforward/rollback is effectively a no-op for daily offsets.\n\n8. **`__add__` delegates to `__apply__`**: The `BaseCFTimeOffset.__add__` method calls `self.__apply__(other)`. For `Day`, this adds a `timedelta(days=self.n)`. For `Tick` subclasses (`Hour`, `Minute`, `Second`, `Millisecond`, `Microsecond`), it calls `other + self.as_timedelta()`. For month/year offsets, it uses `_shift_month` with adjusted month counts.",
    "rubric": [
      "Explains that the frequency string is parsed by `to_offset` using a regex pattern to extract a multiplier and frequency type, then mapped to a `BaseCFTimeOffset` subclass instance via a frequencies dictionary",
      "Describes that the start date is aligned using `rollforward` for positive offsets or `rollback` for negative offsets",
      "Mentions the edge case where `periods` is None and `end < start` with non-negative offset results in an empty range (periods set to 0)",
      "Explains how missing bounds (start or end) are computed using the other bound plus/minus `(periods - 1) * offset`",
      "Describes the iteration loop: for positive offsets yields while `current <= end`, for negative offsets yields while `current >= end`, advancing by adding the offset each step",
      "Mentions the safety check that raises ValueError if the offset doesn't strictly increment (or decrement for negative) the date",
      "Explains that `__add__` delegates to `__apply__`, and describes how different offset subclasses implement `__apply__` (e.g., Day uses timedelta, Tick subclasses use as_timedelta, month/year offsets use _shift_month)",
      "Describes that `rollforward`/`rollback` in the base class checks `onOffset` and advances/retreats by a length-1 offset, while specialized subclasses override with their specific month parameter"
    ],
    "key_files": [
      "xarray/coding/cftime_offsets.py",
      "xarray/compat/pdcompat.py"
    ],
    "source_doc": "[docstring: xarray/coding/cftime_offsets.py] xarray.coding.cftime_offsets._generate_linear_date_range_with_freq\nGenerate a regular range of cftime.datetime objects with a\n    given frequency.\n\n    Adapted from pandas.tseries.offsets.generate_range (now at\n    pandas.core.arrays.datetimes._generate_range).\n\n    Parameters\n    ----------\n    start : cftime.datetime, or None\n        Start of range\n    end : cftime.datetime, or None\n        End of range\n    periods : int, or None\n        Number of elements in the sequence\n    freq: str\n        Step size between cftime.datetime objects. Not None.\n\n    Returns\n    -------\n    A generator object of cftime.datetime objects",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Only one claim about pandas GH issues was documentation-dependent; all other claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_37",
    "repo": "xarray",
    "question": "What is the architecture of `lazy_elemwise_func` and `_ElementwiseFunctionArray`, including their class hierarchy, callers, and how they handle chunked vs non-chunked arrays?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "`lazy_elemwise_func` is a function in `xarray/coding/common.py` that branches based on whether the input array is chunked (e.g., dask) or not.\n\n**Implementation details:**\n- For chunked arrays (detected via `is_chunked_array` from `xarray.namedarray.pycompat`), it calls `chunkmanager.map_blocks(func, array, dtype=dtype)` where the chunk manager is obtained via `get_chunked_array_type(array)`. The dask implementation (`DaskManager.map_blocks` in `xarray/namedarray/daskmanager.py`) delegates directly to `dask.array.map_blocks`.\n- For non-chunked arrays, it returns an `_ElementwiseFunctionArray` instance that wraps the array and defers computation.\n\n**`_ElementwiseFunctionArray` class hierarchy:**\n- Inherits from `ExplicitlyIndexedNDArrayMixin` (in `xarray/core/indexing.py`), which itself inherits from both `NDArrayMixin` (in `xarray/core/utils.py`) and `ExplicitlyIndexed`.\n- `NDArrayMixin` inherits from `NdimSizeLenMixin`, which provides `ndim`, `size`, and `__len__` properties derived from `shape`.\n- `NDArrayMixin` provides default `dtype` and `shape` properties that delegate to `self.array`, but `_ElementwiseFunctionArray` overrides `dtype` to return `np.dtype(self._dtype)` (using its own stored dtype, not the underlying array's dtype).\n- The `__init__` method has an `assert not is_chunked_array(array)` guard, and wraps the input array via `indexing.as_indexable(array)`.\n- The function is only actually applied when `get_duck_array()` is called: `self.func(self.array.get_duck_array())`. There is also an `async_get_duck_array` variant.\n- All indexing methods (`__getitem__`, `_oindex_get`, `_vindex_get`, `transpose`) return new `_ElementwiseFunctionArray` instances wrapping the indexed/transposed underlying array, preserving laziness.\n\n**Callers in the codebase:**\n- `CFMaskCoder.decode()` in `xarray/coding/variables.py` uses it with `_apply_mask` transform\n- `CFScaleOffsetCoder.decode()` in `xarray/coding/variables.py` uses it with `_scale_offset_decoding` transform\n- `_convert_unsigned_fill_value()` in `xarray/coding/variables.py` uses it with `partial(np.asarray, dtype=...)` for unsigned/signed integer conversion\n- `CFDatetimeCoder.decode()` in `xarray/coding/times.py` uses it with `decode_cf_datetime` transform\n- `CFTimedeltaCoder.decode()` in `xarray/coding/times.py` uses it with `decode_cf_timedelta` transform\n- `EncodedStringCoder.decode()` in `xarray/coding/strings.py` uses it with `decode_bytes_array` transform\n\n**Chunked array detection:**\n`is_chunked_array(x)` returns `True` if `x` is a dask duck array OR if it's a duck array with a `chunks` attribute. `get_chunked_array_type` iterates registered chunk managers to find one that recognizes the array type, raising `TypeError` if mixed chunked array types are detected or no manager is found.",
    "rubric": [
      "Explains that lazy_elemwise_func branches on is_chunked_array: chunked arrays use chunkmanager.map_blocks, non-chunked arrays get wrapped in _ElementwiseFunctionArray",
      "Describes _ElementwiseFunctionArray's class hierarchy: inherits ExplicitlyIndexedNDArrayMixin -> NDArrayMixin (which inherits NdimSizeLenMixin) + ExplicitlyIndexed",
      "Notes that NDArrayMixin provides default dtype/shape delegating to self.array, but _ElementwiseFunctionArray overrides dtype to return np.dtype(self._dtype)",
      "Explains that __init__ asserts the array is not chunked and wraps it via indexing.as_indexable",
      "Explains that the function is lazily applied only when get_duck_array() is called (self.func(self.array.get_duck_array()))",
      "Notes that indexing methods (__getitem__, _oindex_get, _vindex_get, transpose) return new _ElementwiseFunctionArray instances, preserving laziness",
      "Identifies at least 3 callers: CFMaskCoder.decode, CFScaleOffsetCoder.decode, CFDatetimeCoder.decode, CFTimedeltaCoder.decode, EncodedStringCoder.decode, or _convert_unsigned_fill_value",
      "For chunked arrays, describes that get_chunked_array_type finds the appropriate chunk manager and map_blocks delegates to e.g. dask.array.map_blocks"
    ],
    "key_files": [
      "xarray/coding/common.py",
      "xarray/core/indexing.py",
      "xarray/core/utils.py",
      "xarray/namedarray/parallelcompat.py",
      "xarray/namedarray/pycompat.py",
      "xarray/namedarray/daskmanager.py",
      "xarray/coding/variables.py",
      "xarray/coding/times.py",
      "xarray/coding/strings.py"
    ],
    "source_doc": "[docstring: xarray/coding/common.py] xarray.coding.common.lazy_elemwise_func\nLazily apply an element-wise function to an array.\n    Parameters\n    ----------\n    array : any valid value of Variable._data\n    func : callable\n        Function to apply to indexed slices of an array. For use with dask,\n        this should be a pickle-able object.\n    dtype : coercible to np.dtype\n        Dtype for the result of this function.\n\n    Returns\n    -------\n    Either a dask.array.Array or _ElementwiseFunctionArray.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; minor corrections needed for class name (DaskManager vs DaskManagerEntrypoint) and caller attribution (EncodedStringCoder not CharacterArrayCoder).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_38",
    "repo": "xarray",
    "question": "Why does `StackedBytesArray` exist as a lazy wrapper instead of eagerly converting character arrays to bytes in `char_to_bytes`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `StackedBytesArray` class exists as a **lazy wrapper** to defer the actual byte-stacking computation until data is accessed. Here's why this design was chosen:\n\n1. **Lazy evaluation for non-chunked arrays**: In `char_to_bytes()`, when the input is a non-chunked (numpy) array, instead of eagerly calling `_numpy_char_to_bytes()` to perform the view-based reshape, it returns a `StackedBytesArray` wrapper. This defers the actual conversion until the data is accessed (via indexing or `get_duck_array()`). For dask arrays, the conversion is handled differently via `chunkmanager.map_blocks()` which is already lazy by nature.\n\n2. **Integration with xarray's explicit indexing system**: `StackedBytesArray` inherits from `ExplicitlyIndexedNDArrayMixin` (which inherits from both `NDArrayMixin` and `ExplicitlyIndexed`). This makes the wrapper compatible with xarray's indexing protocols (`BasicIndexer`, `OuterIndexer`, `VectorizedIndexer`). Being `ExplicitlyIndexed` means `as_indexable()` will pass it through unchanged rather than wrapping it again.\n\n3. **Dimension reduction via the wrapper pattern**: The class presents a reduced-dimensionality view \u2014 `shape` returns `self.array.shape[:-1]` and `dtype` returns `\"S\" + str(self.array.shape[-1])` \u2014 effectively hiding the last character dimension. This allows `CharacterArrayCoder.decode()` to strip the last dimension name from `dims` and replace the data with this wrapper, making the Variable appear to have one fewer dimension.\n\n4. **Enforced last-dimension slicing constraint**: In `__getitem__`, the indexer is expanded to match `self.array.ndim` (not `self.ndim`), and then it checks that the last element of the key tuple is `slice(None)`. This enforces that the last (character) dimension is always fully selected. If any other indexing is attempted on that dimension, it raises `IndexError(\"too many indices\")`.\n\n5. **Actual conversion only in `get_duck_array()`**: The real byte-stacking work happens only when `get_duck_array()` is called, which delegates to `_numpy_char_to_bytes()`. This function uses a numpy `view()` trick (creating a C-contiguous array, then viewing it with an `\"S{n}\"` dtype and reshaping to drop the last dimension) rather than concatenation, making it very efficient.\n\n6. **`__init__` defensive check**: The constructor validates that the input dtype is exactly `\"S1\"` and wraps the array with `indexing.as_indexable()`, ensuring the underlying array is wrapped for xarray's explicit indexing system.",
    "rubric": [
      "Explains that StackedBytesArray defers the actual byte-stacking computation until data is accessed (lazy evaluation), unlike dask which uses map_blocks",
      "Mentions integration with xarray's explicit indexing system via inheriting from ExplicitlyIndexedNDArrayMixin (which combines NDArrayMixin and ExplicitlyIndexed)",
      "Explains the dimension-reduction behavior: shape hides the last dimension and dtype synthesizes an 'S{n}' type, making the Variable appear to have one fewer dimension",
      "Notes that __getitem__ enforces that the last (character) dimension must be fully sliced (slice(None)), raising IndexError otherwise",
      "Explains that actual conversion only happens in get_duck_array() which calls _numpy_char_to_bytes using numpy view/reshape trick",
      "Mentions that __init__ validates dtype=='S1' and wraps the array with indexing.as_indexable()"
    ],
    "key_files": [
      "xarray/coding/strings.py",
      "xarray/core/indexing.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/coding/strings.py] xarray.coding.strings.StackedBytesArray\nxarray.coding.strings.StackedBytesArray:\n  Wrapper around array-like objects to create a new indexable object where\n    values, when accessed, are automatically stacked along the last dimension.\n\n    >>> indexer = indexing.BasicIndexer((slice(None),))\n    >>> np.array(StackedBytesArray(np.array([\"a\", \"b\", \"c\"], dtype=\"S1\"))[indexer])\n    array(b'abc', dtype='|S3')\n\nxarray.coding.strings.StackedBytesArray.__init__:\n  Parameters\n        ----------\n        array : array-like\n            Original array of values to wrap.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The vast majority of claims in the answer (lazy evaluation details, inheritance from ExplicitlyIndexedNDArrayMixin, shape/dtype behavior, __getitem__ enforcement, get_duck_array() internals, __init__ defensive checks, line number references) go far beyond what the provided documentation states. The documentation only describes StackedBytesArray as a wrapper that stacks values along the last dimension, shows a basic example, and documents __init__ taking an array-like parameter.",
      "None of the detailed claims directly contradict the documentation, but they cannot be verified from the provided documentation alone \u2014 they appear to be derived from reading the actual source code.",
      "The basic description of StackedBytesArray as a wrapper that stacks along the last dimension is consistent with the documentation's docstring."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only one docstring quote and minor design rationale statements needed removal/rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_39",
    "repo": "xarray",
    "question": "Where does data flow through `build_output_coords_and_indexes`, from how it gathers input coordinates to how its results get consumed downstream?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `build_output_coords_and_indexes` function in `xarray/computation/apply_ufunc.py` serves as a key data-flow node that is called from two places: `apply_dataarray_vfunc` and `apply_dataset_vfunc`. The data flows as follows:\n\n1. **Input gathering**: `_get_coords_list(args)` extracts coordinates from any argument that has a `.coords` attribute (catching `AttributeError` for scalars/Variables).\n\n2. **Fast path**: When there is exactly one coords object and no `exclude_dims`, the function skips the merge by directly reading `unpacked_coords.variables` and `unpacked_coords.xindexes`. If `combine_attrs == \"drop\"` in this path, it creates new variables using `var._replace(attrs={})` to produce variables with empty attrs.\n\n3. **Full merge path**: Otherwise, it delegates to `merge_coordinates_without_align()` in `xarray/structure/merge.py`, which first calls `collect_from_coordinates()` to group variables/indexes into `dict[Hashable, list[MergeElement]]` (where `MergeElement = tuple[Variable, Index | None]`), then filters out variables whose dims intersect with `exclude_dims`, and finally calls `merge_collected()` which resolves conflicts between same-named variables from different objects.\n\n4. **Output filtering per output**: After merging, the function iterates over `signature.output_core_dims` (a tuple of tuples). For each output, it computes `dropped_dims = signature.all_input_core_dims - set(output_dims)`. The `all_input_core_dims` property is lazily computed as a `frozenset` of all dimensions from all input core dims. If `dropped_dims` is non-empty, coordinates whose dims intersect with `dropped_dims` are filtered out, along with their indexes via `filter_indexes_from_coords()`.\n\n5. **Index filtering logic**: `filter_indexes_from_coords()` (in `xarray/core/indexes.py`) is used to ensure index consistency with the filtered coordinate set.\n\n6. **Downstream consumption**: In `apply_dataarray_vfunc`, the output coords/indexes are passed directly to `DataArray(..., fastpath=True)`. In `apply_dataset_vfunc`, they're passed to `_fast_dataset()` which updates the variables dict in-place with coord variables and uses `Dataset._construct_direct()`.\n\nThe actual function signature uses `\"override\"` as the default for `combine_attrs`. In practice, callers always pass `combine_attrs=keep_attrs` explicitly.",
    "rubric": [
      "Mentions `_get_coords_list(args)` extracts coordinates from arguments with a `.coords` attribute, catching `AttributeError` for scalars/Variables",
      "Describes the fast path: when there is exactly one coords object and no `exclude_dims`, it reads `unpacked_coords.variables` and `unpacked_coords.xindexes` directly, skipping merge; and handles `combine_attrs == 'drop'` via `var._replace(attrs={})`",
      "Describes the full merge path: delegates to `merge_coordinates_without_align()`, which calls `collect_from_coordinates()` and then `merge_collected()`, filtering out variables whose dims intersect `exclude_dims`",
      "Explains per-output filtering: iterates over `signature.output_core_dims`, computes `dropped_dims = signature.all_input_core_dims - set(output_dims)`, and filters coordinates whose dims intersect `dropped_dims`",
      "Mentions `filter_indexes_from_coords()` is used to keep indexes consistent with the filtered coordinate set",
      "Describes downstream consumption: in `apply_dataarray_vfunc`, coords/indexes are passed to `DataArray(..., fastpath=True)`; in `apply_dataset_vfunc`, they're passed to `_fast_dataset()` which uses `Dataset._construct_direct()`"
    ],
    "key_files": [
      "xarray/computation/apply_ufunc.py",
      "xarray/structure/merge.py",
      "xarray/core/indexes.py",
      "xarray/core/variable.py"
    ],
    "source_doc": "[docstring: xarray/computation/apply_ufunc.py] xarray.computation.apply_ufunc.build_output_coords_and_indexes\nBuild output coordinates and indexes for an operation.\n\n    Parameters\n    ----------\n    args : Iterable\n        List of raw operation arguments. Any valid types for xarray operations\n        are OK, e.g., scalars, Variable, DataArray, Dataset.\n    signature : _UfuncSignature\n        Core dimensions signature for the operation.\n    exclude_dims : set, optional\n        Dimensions excluded from the operation. Coordinates along these\n        dimensions are dropped.\n    combine_attrs : {\"drop\", \"identical\", \"no_conflicts\", \"drop_conflicts\",                      \"override\"} or callable, default: \"drop\"\n        A callable or a string indicating how to combine attrs of the objects being\n        merged:\n\n        - \"drop\": empty attrs on returned Dataset.\n        - \"identical\": all attrs must be the same on every object.\n        - \"no_conflicts\": attrs from all objects are combined, any that have\n          the same name must also have the same value.\n        - \"drop_conflicts\": attrs from all objects are combined, any that have\n          the same name but different values are dropped.\n        - \"override\": skip comparing and copy attrs from the first dataset to\n          the result.\n\n        If a callable, it must expect a sequence of ``attrs`` dicts and a context object\n        as its only parameters.\n\n    Returns\n    -------\n    Dictionaries of Variable and Index objects with merged coordinates.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that 'the docstring states the default for combine_attrs is \"drop\" but the actual function signature uses \"override\" as the default' \u2014 the documentation does say the default is \"drop\", so claiming the actual default is \"override\" cannot be verified from the provided documentation alone and could be a hallucination or could be accurate code-level detail.",
      "Most of the answer's claims (line numbers, internal implementation details like _get_coords_list, fast path optimization, merge_coordinates_without_align, filter_indexes_from_coords, downstream consumption details) go far beyond what the documentation provides. These are plausible code-level details but cannot be verified from the given documentation.",
      "The documentation states the return is 'Dictionaries of Variable and Index objects with merged coordinates' \u2014 the answer's description of the return flow is consistent with this but adds significant implementation detail not verifiable from the doc."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for docstring-dependent claims and implementation details of truncated functions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_40",
    "repo": "xarray",
    "question": "How does xarray's curvefit implementation work end-to-end, from DataArray method call through parameter initialization to the apply_ufunc wrapper?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `curvefit` function in `xarray/computation/fit.py` implements curve fitting by wrapping `scipy.optimize.curve_fit` through `apply_ufunc`. Here is how it works in detail:\n\n**Call chain delegation:**\n- `DataArray.curvefit()` first converts itself to a temporary Dataset via `_to_temp_dataset()`, then calls `Dataset.curvefit()`. The temp dataset uses a special sentinel key `_THIS_ARRAY = ReprObject(\"<this-array>\")` as the variable name.\n- `Dataset.curvefit()` delegates to the standalone `curvefit` function imported from `xarray.computation.fit`.\n\n**Parameter extraction:**\n- `_get_func_args()` uses `inspect.signature(func).parameters` to introspect the callable. It skips the first parameter (the x/coordinate argument) by doing `params = list(func_args)[1:]`. If the function has `VAR_POSITIONAL` or `VAR_KEYWORD` parameters, it raises a `ValueError` requiring the user to supply `param_names` explicitly.\n\n**Parameter initialization (`_initialize_curvefit_params`):**\n- Default initial guesses for all parameters are set to `1` via `dict.fromkeys(params, 1)`. Default bounds are `(-np.inf, np.inf)`.\n- Priority order: (1) explicitly passed `p0` values override everything (applied last), (2) function signature defaults (extracted via `func_args[p].default`), (3) the fallback default of `1`.\n- When bounds are specified and the current default guess falls outside those bounds, a feasibility adjustment is applied via `_initialize_feasible()`, which implements: if both bounds finite \u2192 midpoint `0.5*(lb+ub)`, lower finite only \u2192 `lb+1`, upper finite only \u2192 `ub-1`, both infinite \u2192 `0`.\n\n**Dimension handling:**\n- Coordinates are automatically detected as dimensions by checking `coord.equals(obj[c])` for each dimension `c` in the calling object.\n- If no `reduce_dims` are identified (neither from `coords` matching dimensions nor from explicit `reduce_dims`), a `ValueError` is raised about \"fitting on scalar data.\"\n- `preserved_dims` are computed as `set(obj.dims) - set(reduce_dims_)`, and initial guesses/bounds DataArrays are validated to only contain dimensions within `preserved_dims`.\n\n**The inner `_wrapper` function and `apply_ufunc` call:**\n- Coordinates are broadcast with each other via `broadcast(*coords_)` and then broadcast to match the object's shape (excluding preserved dims) via `broadcast_like(obj, exclude=preserved_dims)`.\n- The wrapper receives flattened args: `n_coords` coordinate arrays, then `n_params` initial guesses, then `n_params` lower bounds, then `n_params` upper bounds \u2014 total of `n_coords + 3*n_params` extra arguments.\n- Inside `_wrapper`, coordinates are vstacked and raveled; data Y is raveled. When `skipna=True`, a mask is computed requiring both: any non-NaN coordinate value at each point AND non-NaN y value. If all data is masked out (y.size == 0), NaN arrays are returned.\n- Coordinates are squeezed via `np.squeeze(x)` after masking.\n- Only `RuntimeError` is caught (not all exceptions).\n- `apply_ufunc` is called with `vectorize=True`, `dask=\"parallelized\"`, output core dims of `[\"param\"]` and `[\"cov_i\", \"cov_j\"]`, both with `np.float64` output dtype. The `exclude_dims` is set to the reduce dims, and `dask_gufunc_kwargs` specifies output sizes for the param/covariance dimensions.\n\n**Result construction:**\n- Input core dims are `reduce_dims_` repeated `n_coords + 1` times (for data + each coordinate), plus empty lists `[]` for each of the `3 * n_params` scalar arguments (p0, lb, ub).\n- The result Dataset gets coordinate assignments: `param`, `cov_i`, and `cov_j` are all assigned the parameter name list.\n- Variable naming: when called from a DataArray (detected via `name is _THIS_ARRAY` identity check), the prefix is empty string `\"\"`, resulting in keys like `curvefit_coefficients`. For Dataset variables, the prefix is `f\"{name}_\"`.\n- The result's `attrs` are copied from the input object's `attrs`.\n- The return type is `type(obj)()` \u2014 preserving the Dataset subclass if any.",
    "rubric": [
      "Explains the delegation chain: DataArray.curvefit() converts to temp dataset via _to_temp_dataset(), then Dataset.curvefit() delegates to the standalone curvefit function in xarray/computation/fit.py",
      "Describes how _get_func_args uses inspect.signature to extract parameters, skips the first arg (coordinate), and raises ValueError for VAR_POSITIONAL/VAR_KEYWORD parameters",
      "Explains _initialize_curvefit_params: default guess is 1, default bounds are (-inf, inf), priority order is p0 > function signature defaults > fallback of 1",
      "Describes _initialize_feasible logic: both bounds finite \u2192 midpoint, lower finite only \u2192 lb+1, upper finite only \u2192 ub-1, both infinite \u2192 0",
      "Explains how reduce_dims are identified by checking coord.equals(obj[c]) for each dimension, and ValueError if no reduce_dims found",
      "Describes the inner _wrapper function: vstacks and ravels coordinates, ravels Y, applies NaN mask when skipna=True, catches only RuntimeError",
      "Explains how apply_ufunc is called with vectorize=True, dask='parallelized', output_core_dims for param and covariance dimensions",
      "Describes input_core_dims structure: reduce_dims repeated n_coords+1 times, then empty lists for 3*n_params scalar args (p0, lb, ub)",
      "Explains result naming: _THIS_ARRAY identity check gives empty prefix for DataArray calls, otherwise f'{name}_' prefix for Dataset variables"
    ],
    "key_files": [
      "xarray/computation/fit.py",
      "xarray/core/dataarray.py",
      "xarray/core/dataset.py"
    ],
    "source_doc": "[docstring: xarray/computation/fit.py] xarray.computation.fit.curvefit\nCurve fitting optimization for arbitrary functions.\n\n    Wraps `scipy.optimize.curve_fit` with `apply_ufunc`.\n\n    Parameters\n    ----------\n    obj : Dataset or DataArray\n        Object to perform the curvefit on\n    coords : hashable, DataArray, or sequence of hashable or DataArray\n        Independent coordinate(s) over which to perform the curve fitting. Must share\n        at least one dimension with the calling object. When fitting multi-dimensional\n        functions, supply `coords` as a sequence in the same order as arguments in\n        `func`. To fit along existing dimensions of the calling object, `coords` can\n        also be specified as a str or sequence of strs.\n    func : callable\n        User specified function in the form `f(x, *params)` which returns a numpy\n        array of length `len(x)`. `params` are the fittable parameters which are optimized\n        by scipy curve_fit. `x` can also be specified as a sequence containing multiple\n        coordinates, e.g. `f((x0, x1), *params)`.\n    reduce_dims : str, Iterable of Hashable or None, optional\n        Additional dimension(s) over which to aggregate while fitting. For example,\n        calling `ds.curvefit(coords='time', reduce_dims=['lat', 'lon'], ...)` will\n        aggregate all lat and lon points and fit the specified function along the\n        time dimension.\n    skipna : bool, default: True\n        Whether to skip missing values when fitting. Default is True.\n    p0 : dict-like, optional\n        Optional dictionary of parameter names to initial guesses passed to the\n        `curve_fit` `p0` arg. If the values are DataArrays, they will be appropriately\n        broadcast to the coordinates of the array. If none or only some parameters are\n        passed, the rest will be assigned initial values following the default scipy\n        behavior.\n    bounds : dict-like, optional\n        Optional dictionary of parameter names to tuples of bounding values passed to the\n        `curve_fit` `bounds` arg. If any of the bounds are DataArrays, they will be\n        appropriately broadcast to the coordinates of the array. If none or only some\n        parameters are passed, the rest will be unbounded following the default scipy\n        behavior.\n    param_names : sequence of hashable, optional\n        Sequence of names for the fittable parameters of `func`. If not supplied,\n        this will be automatically determined by arguments of `func`. `param_names`\n        should be manually supplied when fitting a function that takes a variable\n        number of parameters.\n    errors : {\"raise\", \"ignore\"}, default: \"raise\"\n        If 'raise', any errors from the `scipy.optimize_curve_fit` optimization will\n        raise an exception. If 'ignore', the coefficients and covariances for the\n        coordinates where the fitting failed will be NaN.\n    kwargs : optional\n        Additional keyword arguments to passed to scipy curve_fit.\n\n    Returns\n    -------\n    Dataset\n        A single dataset which contains:\n\n        [var]_curvefit_coefficients\n            The coefficients of the best fit.\n        [var]_curvefit_covariance\n            The covariance matrix of the coefficient estimates.\n\n    See Also\n    --------\n    Dataset.polyfit\n    scipy.optimize.curve_fit",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for line numbers and documentation-level explanations about scipy internals.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_41",
    "repo": "xarray",
    "question": "What is the architecture of `where_method` across xarray's layers, and how does the `dtypes.NA` sentinel flow through both implementations?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `where_method` function in `xarray/computation/ops.py` takes `self`, `cond`, and `other` (defaulting to `dtypes.NA`) and dispatches to the low-level implementation via `apply_ufunc`. It has two distinct implementations that work together:\n\n1. **High-level `where_method` in `xarray/computation/ops.py`**: This is the version called by DataArray, Dataset, Variable, and GroupBy objects. It uses `apply_ufunc` to dispatch to the low-level implementation. It determines the `join` strategy based on whether `other` is the sentinel `dtypes.NA` value: if `other` is `NA`, it uses `\"inner\"` join; otherwise, it uses `\"exact\"` join. Both `join` and `dataset_join` are set to the same value. It also passes `dask=\"allowed\"` and `keep_attrs=True`.\n\n2. **Low-level `where_method` in `xarray/core/duck_array_ops.py`**: This operates on array data. When `other` is `dtypes.NA`, it calls `dtypes.get_fill_value(data.dtype)` to determine the appropriate missing value for the data's dtype. It then calls the `where(cond, data, other)` function which promotes both `x` and `y` to a shared dtype using `as_shared_dtype([x, y], xp=xp)` before calling `xp.where()`.\n\nThe `dtypes.NA` sentinel is a `utils.ReprObject(\"<NA>\")` instance defined in `xarray/core/dtypes.py`. The code uses `is` identity checks against this sentinel (e.g., `other is dtypes.NA`).\n\nThe `get_fill_value` function delegates to `maybe_promote(dtype)` which returns different fill values based on dtype: `np.nan` for real floating and integer types (integers get promoted to float32 or float64), `np.nan + np.nan * 1j` for complex floating, `NaT` for datetime64 and timedelta64, and `np.nan` (as object) for other types.\n\nThe function is exposed as the `.where()` method on multiple xarray classes including `DataArray`, `Dataset`, `Variable`, and `GroupBy`.\n\nThe low-level `where_method` from `duck_array_ops` is also referenced internally in other parts of the codebase such as `nanops.py`.",
    "rubric": [
      "Identifies that there are two `where_method` implementations: a high-level one in computation/ops.py and a low-level one in core/duck_array_ops.py",
      "Explains that the high-level version uses `apply_ufunc` to dispatch to the low-level duck_array_ops.where_method",
      "Explains that join strategy is 'inner' when other is dtypes.NA and 'exact' otherwise, using `is` identity check",
      "Mentions that dask='allowed' and keep_attrs=True are passed to apply_ufunc",
      "Explains that the low-level version resolves dtypes.NA by calling dtypes.get_fill_value(data.dtype)",
      "Describes that dtypes.NA is a ReprObject('<NA>') sentinel instance",
      "Explains that get_fill_value delegates to maybe_promote which returns np.nan for floats/integers, NaT for datetime/timedelta, and complex nan for complex types",
      "Mentions that the low-level where function promotes both x and y to a shared dtype before calling xp.where()"
    ],
    "key_files": [
      "xarray/computation/ops.py",
      "xarray/core/duck_array_ops.py",
      "xarray/core/dtypes.py",
      "xarray/core/common.py"
    ],
    "source_doc": "[docstring: xarray/computation/ops.py] xarray.computation.ops.where_method\nReturn elements from `self` or `other` depending on `cond`.\n\n    Parameters\n    ----------\n    cond : DataArray or Dataset with boolean dtype\n        Locations at which to preserve this objects values.\n    other : scalar, DataArray or Dataset, optional\n        Value to use for locations in this object where ``cond`` is False.\n        By default, inserts missing values.\n\n    Returns\n    -------\n    Same type as caller.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed a comment-based rationale claim and softened claims about files not shown in the provided stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_42",
    "repo": "xarray",
    "question": "Why is `decode_cf_variable` designed with that specific ordering of decoding steps, and what guards does it use to avoid incorrect or redundant processing?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `decode_cf_variable` function exists as the core single-variable decoding pipeline in xarray's CF conventions system. Its design reveals several important architectural decisions:\n\n**1. Early exit for already-decoded variables:**\nThe very first thing the function does is check `_contains_datetime_like_objects(var)` and returns immediately if true. This is a defensive guard to ensure datetime-like Variables already containing `np.datetime64`, `np.timedelta64`, or `cftime.datetime` objects are not double-decoded. This check delegates to `is_np_datetime_like(var.dtype)` or `contains_cftime_datetimes(var)` in `xarray/core/common.py`.\n\n**2. Deliberate ordering of decoding steps:**\nThe function applies coders in a specific sequence:\n- First: string decoding (`CharacterArrayCoder`, `EncodedStringCoder`) \u2014 operates on raw byte data\n- Second: `ObjectVLenStringCoder` and `Numpy2StringDTypeCoder` \u2014 handle special string dtypes\n- Third: mask and scale (`CFMaskCoder` then `CFScaleOffsetCoder`) \u2014 applied before time decoding\n- Fourth: timedelta decoding, then datetime decoding\n- Fifth: endianness correction and boolean decoding\n- Last: wrapping in `LazilyIndexedArray` if needed\n\nThis order is the reverse of encoding order in `encode_cf_variable`, where the sequence is: `CFDatetimeCoder` \u2192 `CFTimedeltaCoder` \u2192 `CFScaleOffsetCoder` \u2192 `CFMaskCoder` \u2192 `NativeEnumCoder` \u2192 `NonStringCoder` \u2192 `DefaultFillvalueCoder` \u2192 `BooleanCoder`.\n\n**3. Time-aware masking to properly handle NaT:**\n`CFMaskCoder` and `CFScaleOffsetCoder` both accept `decode_times` and `decode_timedelta` parameters. When masking time-like integer data, the fill value is set to `np.iinfo(np.int64).min` (rather than `np.nan`) when `decode_times` or `decode_timedelta` is enabled and `data.dtype.kind in \"iu\"`. The `_is_time_like` helper distinguishes between \"datetime\" (units contain \"since\") and \"timedelta\" (units are plain time strings like \"hours\") to apply the correct masking strategy.\n\n**4. Lazy evaluation design:**\nAll transformations (masking, scaling, time decoding) use `lazy_elemwise_func`, which wraps data in `_ElementwiseFunctionArray` (for numpy) or uses `chunkmanager.map_blocks` (for dask). The function is computed only upon indexing or `.get_duck_array()`. At the end, if the data is not a duck array and not already an `ExplicitlyIndexedNDArrayMixin`, it gets wrapped in `indexing.LazilyIndexedArray` to support lazy indexing.\n\n**5. `decode_timedelta` defaults derive from `decode_times`:**\nWhen `decode_timedelta` is None, the function infers it from `decode_times`: if `decode_times` is a `CFDatetimeCoder` instance, a `CFTimedeltaCoder` is created with the same `time_unit`; if `decode_times` is `True`, a default `CFTimedeltaCoder()` is created; if `decode_times` is `False`, timedelta decoding is disabled. When `decode_timedelta` is a boolean `True`, `CFTimedeltaCoder` is created with `decode_via_units=True` and `decode_via_dtype=True`.\n\n**6. Deprecation transition pattern:**\nThe function has a two-branch pattern for `use_cftime`: if `decode_times` is not yet a `CFDatetimeCoder`, passing `use_cftime` emits a `FutureWarning` and wraps it; if `decode_times` is already a `CFDatetimeCoder` and `use_cftime` is also provided, a `TypeError` is raised to prevent conflicting configuration.\n\n**7. Original dtype preservation:**\nThe `original_dtype` is captured before any transformations and stored in `encoding` via `encoding.setdefault('dtype', original_dtype)`.",
    "rubric": [
      "Explains the early exit guard: `_contains_datetime_like_objects(var)` check at the top prevents double-decoding of variables already containing datetime-like objects (np.datetime64, np.timedelta64, cftime.datetime)",
      "Describes the deliberate ordering: strings first, then mask/scale, then timedelta, then datetime, then endianness and boolean \u2014 which is the reverse of the encoding order in `encode_cf_variable`",
      "Explains that CFMaskCoder and CFScaleOffsetCoder receive decode_times/decode_timedelta parameters so they can use NaT-compatible fill values (np.iinfo(np.int64).min) instead of NaN for time-like integer data",
      "Describes how decode_timedelta defaults are inferred from decode_times when None (creates CFTimedeltaCoder with matching time_unit from CFDatetimeCoder, or default CFTimedeltaCoder if True, or False if decode_times is False)",
      "Mentions the deprecation/transition handling for use_cftime: FutureWarning when decode_times is a bool, TypeError when decode_times is already a CFDatetimeCoder",
      "Notes the lazy evaluation design using lazy_elemwise_func and wrapping in LazilyIndexedArray at the end for non-duck-array data",
      "Mentions that original_dtype is captured before transformations and preserved in encoding via encoding.setdefault('dtype', original_dtype)"
    ],
    "key_files": [
      "xarray/conventions.py",
      "xarray/coding/variables.py",
      "xarray/coding/times.py",
      "xarray/coding/common.py"
    ],
    "source_doc": "[docstring: xarray/conventions.py] xarray.conventions.decode_cf_variable\nDecodes a variable which may hold CF encoded information.\n\n    This includes variables that have been masked and scaled, which\n    hold CF style time variables (this is almost always the case if\n    the dataset has been serialized) and which have strings encoded\n    as character arrays.\n\n    Parameters\n    ----------\n    name : str\n        Name of the variable. Used for better error messages.\n    var : Variable\n        A variable holding potentially CF encoded information.\n    concat_characters : bool\n        Should character arrays be concatenated to strings, for\n        example: [\"h\", \"e\", \"l\", \"l\", \"o\"] -> \"hello\"\n    mask_and_scale : bool\n        Lazily scale (using scale_factor and add_offset) and mask\n        (using _FillValue). If the _Unsigned attribute is present\n        treat integer arrays as unsigned.\n    decode_times : bool or CFDatetimeCoder\n        Decode cf times (\"hours since 2000-01-01\") to np.datetime64.\n    decode_endianness : bool\n        Decode arrays from non-native to native endianness.\n    stack_char_dim : bool\n        Whether to stack characters into bytes along the last dimension of this\n        array. Passed as an argument because we need to look at the full\n        dataset to figure out if this is appropriate.\n    use_cftime : bool, optional\n        Only relevant if encoded dates come from a standard calendar\n        (e.g. \"gregorian\", \"proleptic_gregorian\", \"standard\", or not\n        specified).  If None (default), attempt to decode times to\n        ``np.datetime64[ns]`` objects; if this is not possible, decode times to\n        ``cftime.datetime`` objects. If True, always decode times to\n        ``cftime.datetime`` objects, regardless of whether or not they can be\n        represented using ``np.datetime64[ns]`` objects.  If False, always\n        decode times to ``np.datetime64[ns]`` objects; if this is not possible\n        raise an error.\n\n        .. deprecated:: 2025.01.1\n           Please pass a :py:class:`coders.CFDatetimeCoder` instance initialized with ``use_cftime`` to the ``decode_times`` kwarg instead.\n    decode_timedelta : None, bool, or CFTimedeltaCoder\n        Decode cf timedeltas (\"hours\") to np.timedelta64.\n\n    Returns\n    -------\n    out : Variable\n        A variable holding the decoded equivalent of var.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (ordering of coders, lazy evaluation mechanisms, GH-6453 guard, original_dtype preservation, etc.) that are not present in the provided documentation. These are plausible code-level details but cannot be verified from the docstring alone.",
      "The claim about `decode_timedelta` defaults deriving from `decode_times` with specific behavior (decode_via_units=False, decode_via_dtype=True) is not documented and cannot be verified from the provided text.",
      "The claim about CFTimedeltaCoder's __init__ defaults (decode_via_units=False, decode_via_dtype=True) is highly specific implementation detail not verifiable from the docstring.",
      "The description of the deprecation pattern (FutureWarning vs TypeError) is plausible given the deprecation notice in the docstring but adds implementation specifics not documented.",
      "The claim about _is_time_like helper distinguishing datetime vs timedelta based on 'since' keyword is not documented but is plausible.",
      "The answer's claim about mask_and_scale coders accepting decode_times and decode_timedelta parameters is not verifiable from the docstring."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few rationale/purpose explanations and the GH issue reference rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_43",
    "repo": "xarray",
    "question": "Where does `decode_cf` fit in xarray's data loading pipeline, and what decoding steps does it orchestrate?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `decode_cf` function in `xarray/conventions.py` serves as a high-level entry point for decoding data. Here's where it fits in the system:\n\n**Call chain - Who calls `decode_cf`:**\n1. It is called directly from `xarray/convert.py` inside the `from_iris()` function, which converts an Iris cube to a DataArray by first creating a temporary Dataset and then calling `decode_cf` on it.\n2. It is exposed publicly as `xr.decode_cf()`.\n3. Importantly, `xarray.backends.api.open_dataset()` does **NOT** call `decode_cf()` directly. Instead, it delegates to backend-specific `open_dataset` methods, which in turn call `StoreBackendEntrypoint.open_dataset` (in `backends/store.py`), which calls `conventions.decode_cf_variables()` directly \u2014 bypassing `decode_cf` itself.\n\n**Call chain - What `decode_cf` calls:**\n1. `decode_cf` distinguishes between `Dataset` and `AbstractDataStore` inputs. For a Dataset, it reads `_variables`, `attrs`, `coords`, `_close`, and `encoding`. For an AbstractDataStore, it calls `obj.load()` and `obj.get_encoding()`.\n2. It calls `decode_cf_variables()`, which iterates over all variables and calls `decode_cf_variable()` for each one.\n3. `decode_cf_variable()` applies a specific ordered pipeline of coders:\n   - First, if the variable already contains datetime-like objects, it returns immediately.\n   - Character concatenation: `CharacterArrayCoder().decode()` then `EncodedStringCoder().decode()`\n   - Object VLen strings: `ObjectVLenStringCoder().decode()`\n   - Numpy2 StringDType: `Numpy2StringDTypeCoder().decode()`\n   - Masking and scaling: `CFMaskCoder().decode()` then `CFScaleOffsetCoder().decode()` (both initialized with `decode_times` and `decode_timedelta` args)\n   - Timedelta decoding: `CFTimedeltaCoder().decode()`\n   - Datetime decoding: `CFDatetimeCoder().decode()`\n   - Endian conversion: `EndianCoder().decode()`\n   - Boolean decoding: `BooleanCoder().decode()`\n   - Finally wraps raw data in `LazilyIndexedArray` if needed, and sets `encoding['dtype']` to the original dtype.\n\n**Key details from code:**\n- The `_item_or_default` helper enables per-variable decoding options \u2014 if the argument is a `Mapping`, it looks up the variable name as a key; otherwise it uses the scalar value for all variables.\n- When `decode_times` is truthy, `_update_bounds_attributes()` is called first, which copies `units` and `calendar` attributes from time variables to their associated bounds variables (referenced via the \"bounds\" attribute).\n- `decode_coords=\"all\"` triggers parsing of `CF_RELATED_DATA` attributes, which is a 10-element tuple: \"bounds\", \"grid_mapping\", \"climatology\", \"geometry\", \"node_coordinates\", \"node_count\", \"part_node_count\", \"interior_ring\", \"cell_measures\", \"formula_terms\". Of these, only \"grid_mapping\", \"cell_measures\", and \"formula_terms\" require special colon-based parsing (`CF_RELATED_DATA_NEEDS_PARSING`).\n- When `decode_timedelta` is None and `decode_times` is a `CFDatetimeCoder`, the timedelta decoder inherits the `time_unit` from the datetime coder.\n- `TIME_UNITS` includes \"nanoseconds\" along with \"days\", \"hours\", \"minutes\", \"seconds\", \"milliseconds\", and \"microseconds\".\n- The `_resolve_decoders_kwargs` function in `backends/api.py` sets all decoder kwargs to `False` when `decode_cf=False`, and removes any kwargs that are `None` \u2014 this is how `open_dataset`'s `decode_cf` flag controls decoding.\n- After decoding, `decode_cf` constructs a new Dataset, sets coordinate names as the union of decoded coord_names and any extra coordinates from the original object, copies the close method, and preserves encoding.",
    "rubric": [
      "Mentions that `decode_cf` is called from `from_iris()` in `convert.py`",
      "Explains that `open_dataset` does NOT call `decode_cf` directly but instead calls `decode_cf_variables` via `StoreBackendEntrypoint.open_dataset`",
      "Describes that `decode_cf` handles both Dataset and AbstractDataStore inputs differently (Dataset reads _variables/attrs/coords vs DataStore calls load()/get_encoding())",
      "Identifies that `decode_cf` calls `decode_cf_variables()` which iterates over variables calling `decode_cf_variable()` for each",
      "Lists the ordered decoding pipeline: character/string coders, masking/scaling, timedelta, datetime, endian, boolean",
      "Mentions `_update_bounds_attributes` copies units/calendar from time variables to bounds variables when decode_times is truthy",
      "Explains that decode_coords='all' triggers parsing of CF_RELATED_DATA attributes",
      "Notes that `_resolve_decoders_kwargs` sets all decoder kwargs to False when decode_cf=False",
      "Mentions per-variable decoding support via Mapping-type arguments checked by a helper function"
    ],
    "key_files": [
      "xarray/conventions.py",
      "xarray/coding/times.py",
      "xarray/backends/store.py",
      "xarray/backends/api.py",
      "xarray/convert.py",
      "xarray/coders.py"
    ],
    "source_doc": "[docstring: xarray/conventions.py] xarray.conventions.decode_cf\nDecode the given Dataset or Datastore according to CF conventions into\n    a new Dataset.\n\n    Parameters\n    ----------\n    obj : Dataset or DataStore\n        Object to decode.\n    concat_characters : bool, optional\n        Should character arrays be concatenated to strings, for\n        example: [\"h\", \"e\", \"l\", \"l\", \"o\"] -> \"hello\"\n    mask_and_scale : bool, optional\n        Lazily scale (using scale_factor and add_offset) and mask\n        (using _FillValue).\n    decode_times : bool | CFDatetimeCoder | Mapping[str, bool | CFDatetimeCoder], optional\n        Decode cf times (e.g., integers since \"hours since 2000-01-01\") to\n        np.datetime64.\n    decode_coords : bool or {\"coordinates\", \"all\"}, optional\n        Controls which variables are set as coordinate variables:\n\n        - \"coordinates\" or True: Set variables referred to in the\n          ``'coordinates'`` attribute of the datasets or individual variables\n          as coordinate variables.\n        - \"all\": Set variables referred to in  ``'grid_mapping'``, ``'bounds'`` and\n          other attributes as coordinate variables.\n    drop_variables : str or iterable, optional\n        A variable or list of variables to exclude from being parsed from the\n        dataset. This may be useful to drop variables with problems or\n        inconsistent values.\n    use_cftime : bool, optional\n        Only relevant if encoded dates come from a standard calendar\n        (e.g. \"gregorian\", \"proleptic_gregorian\", \"standard\", or not\n        specified).  If None (default), attempt to decode times to\n        ``np.datetime64[ns]`` objects; if this is not possible, decode times to\n        ``cftime.datetime`` objects. If True, always decode times to\n        ``cftime.datetime`` objects, regardless of whether or not they can be\n        represented using ``np.datetime64[ns]`` objects.  If False, always\n        decode times to ``np.datetime64[ns]`` objects; if this is not possible\n        raise an error.\n\n        .. deprecated:: 2025.01.1\n           Please pass a :py:class:`coders.CFDatetimeCoder` instance initialized with ``use_cftime`` to the ``decode_times`` kwarg instead.\n\n    decode_timedelta : bool | CFTimedeltaCoder | Mapping[str, bool | CFTimedeltaCoder], optional\n        If True or :py:class:`CFTimedeltaCoder`, decode variables and\n        coordinates with time units in\n        {\"days\", \"hours\", \"minutes\", \"seconds\", \"milliseconds\", \"microseconds\"}\n        into timedelta objects. If False, leave them encoded as numbers.\n        If None (default), assume the same behavior as decode_times. The\n        resolution of the decoded timedeltas can be configured with the\n        ``time_unit`` argument in the :py:class:`CFTimedeltaCoder` passed.\n\n    Returns\n    -------\n    decoded : Dataset",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation states decode_timedelta time units are {\"days\", \"hours\", \"minutes\", \"seconds\", \"milliseconds\", \"microseconds\"}, while the answer claims TIME_UNITS includes \"nanoseconds\" in addition to these six. This is a claim about code internals that cannot be verified from the documentation alone, but it doesn't directly contradict the docs (docs describe the parameter behavior, not necessarily internal constants).",
      "The answer mentions specific line numbers and internal implementation details (e.g., line 498, line 571-583, etc.) that cannot be verified from the provided documentation. These are plausible code-level details but are beyond the scope of verification.",
      "The answer claims CFTimedeltaCoder defaults to decode_via_units=False and decode_via_dtype=True - this cannot be verified from the documentation provided.",
      "The answer's description of decode_cf_variable's pipeline order and specific coders (ObjectVLenStringCoder, Numpy2StringDTypeCoder, EndianCoder, BooleanCoder, etc.) cannot be verified from the documentation but represents plausible code-level details."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc leakage in line numbers, GH issue references, and one claim about CFTimedeltaCoder defaults from truncated code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_44",
    "repo": "xarray",
    "question": "How do the `floor`, `ceil`, and `round` methods on the datetime accessor handle both numpy datetime and cftime arrays, including dask support?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `floor`, `ceil`, and `round` methods on `TimeAccessor` work through a shared implementation pipeline with two distinct paths depending on the data type (numpy datetime vs cftime).\n\n**Algorithm flow:**\n\n1. **Entry point**: All three methods (`floor`, `ceil`, `round`) delegate to `_tslib_round_accessor(name, freq)`, passing the method name as a string.\n\n2. **Data extraction**: `_tslib_round_accessor` first calls `_index_or_data(self._obj)` to extract the underlying data. If the DataArray's variable is an `IndexVariable`, it calls `obj.to_index()` to get a pandas Index; otherwise it gets `obj.data` directly.\n\n3. **Dispatch via `_round_field`**: This function checks if the values are a dask array using `is_duck_dask_array`. For dask arrays, it uses `dask.array.map_blocks` with a dtype of `np.datetime64` for numpy datetime-like types or `np.dtype(\"O\")` for cftime types. For non-dask arrays, it calls `_round_through_series_or_index` directly.\n\n4. **`_round_through_series_or_index`**: This is where the two paths diverge:\n   - **numpy datetime path**: Values are wrapped in a `pd.Series`, and the rounding is performed via `pd.Series.dt.floor/ceil/round(freq=freq)`, delegating entirely to pandas.\n   - **cftime path**: Values are wrapped in a `CFTimeIndex` (which extends `pd.Index`), and the rounding method is called on the CFTimeIndex.\n\n5. **CFTimeIndex rounding implementation** (in `cftimeindex.py`, not fully visible in truncated code):\n   - The CFTimeIndex has floor/ceil/round methods that are called with `freq` parameter.\n   - The `exact_cftime_datetime_difference` function (visible in `resample_cftime.py`) splits computation into seconds and microseconds parts: it replaces microseconds with 0, computes seconds difference, then adds back the microsecond difference separately, returning a `timedelta(seconds=seconds, microseconds=microseconds)`.\n\n6. **Result wrapping**: The rounded values are reshaped to the original array shape, wrapped in a new `Variable` (preserving dims, attrs, and encoding from the original), and returned via `self._obj._replace(newvar, name=name)`.\n\n**Registration**: The `CombinedDatetimelikeAccessor.__new__` method checks if the underlying data contains datetime-like objects and returns either a `DatetimeAccessor` or `TimedeltaAccessor` instance. Both inherit from `TimeAccessor`, which defines the rounding methods.",
    "rubric": [
      "Mentions that floor/ceil/round delegate to _tslib_round_accessor with the method name as a string",
      "Explains that _index_or_data extracts either a pandas Index (for IndexVariable) or raw .data from the DataArray",
      "Describes that _round_field checks for dask arrays using is_duck_dask_array and uses dask.array.map_blocks if needed",
      "Explains the numpy datetime path: values are wrapped in a pd.Series and rounding is done via pd.Series.dt.floor/ceil/round",
      "Explains the cftime path: values are wrapped in a CFTimeIndex and the rounding method is called on it",
      "Mentions that results are reshaped to the original array shape and wrapped in a new Variable preserving dims, attrs, and encoding",
      "Mentions that CombinedDatetimelikeAccessor.__new__ dispatches to DatetimeAccessor or TimedeltaAccessor based on dtype"
    ],
    "key_files": [
      "xarray/core/accessor_dt.py",
      "xarray/coding/cftimeindex.py",
      "xarray/core/resample_cftime.py"
    ],
    "source_doc": "[docstring: xarray/core/accessor_dt.py] xarray.core.accessor_dt.TimeAccessor\nxarray.core.accessor_dt.TimeAccessor.floor:\n  Round timestamps downward to specified frequency resolution.\n\n        Parameters\n        ----------\n        freq : str\n            a freq string indicating the rounding resolution e.g. \"D\" for daily resolution\n\n        Returns\n        -------\n        floor-ed timestamps : same type as values\n            Array-like of datetime fields accessed for each element in values\n\nxarray.core.accessor_dt.TimeAccessor.ceil:\n  Round timestamps upward to specified frequency resolution.\n\n        Parameters\n        ----------\n        freq : str\n            a freq string indicating the rounding resolution e.g. \"D\" for daily resolution\n\n        Returns\n        -------\n        ceil-ed timestamps : same type as values\n            Array-like of datetime fields accessed for each element in values\n\nxarray.core.accessor_dt.TimeAccessor.round:\n  Round timestamps to specified frequency resolution.\n\n        Parameters\n        ----------\n        freq : str\n            a freq string indicating the rounding resolution e.g. \"D\" for daily resolution\n\n        Returns\n        -------\n        rounded timestamps : same type as values\n            Array-like of datetime fields accessed for each element in values",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Several claims about CFTimeIndex internal rounding implementation (integer rounding functions, asi8, _total_microseconds) come from truncated code not shown in the stripped version; the rationale for exact_cftime_datetime_difference is from comments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_45",
    "repo": "xarray",
    "question": "What is the full architecture of the CoordinateTransform system, including its class hierarchy, integration with indexing, and concrete implementations?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `CoordinateTransform` class in `xarray/core/coordinate_transform.py` is an effectively abstract class (its `forward`, `reverse`, and `equals` methods raise `NotImplementedError`). Here's what the code reveals:\n\n**Class Attributes and Initialization:**\n- The class has four instance attributes: `coord_names` (tuple of Hashable), `dims` (tuple of str), `dim_size` (dict of str\u2192int), and `dtype` (Any).\n- The `__init__` method converts `coord_names` to a tuple, derives `dims` from the keys of `dim_size` (via `tuple(dim_size)`), creates a dict copy of `dim_size`, and defaults `dtype` to `np.dtype(np.float64)` if None is passed.\n\n**The `generate_coords` method implementation:**\n- Uses `np.meshgrid` with `indexing=\"ij\"` to create position arrays for all dimensions.\n- Creates positions from `np.arange(self.dim_size[d])` for each dimension.\n- Delegates to `self.forward(dim_positions)` to compute the actual coordinate labels.\n- Accepts an optional `dims` parameter; if None, it uses `self.dims`.\n\n**Integration with xarray's indexing system:**\n- `CoordinateTransformIndex` (in `xarray/core/indexes.py`) wraps a `CoordinateTransform` and inherits from `Index`.\n- `CoordinateTransformIndex.isel()` returns `None`.\n- `CoordinateTransformIndex.sel()` requires `method='nearest'` and requires all coordinate names to have labels provided. It uses `self.transform.reverse()` to convert world coordinates back to grid positions, then rounds positions to integers using `np.round(pos).astype(\"int\")`.\n- `CoordinateTransformIndex.rename()` performs a `copy.deepcopy` of the transform and updates `coord_names`, `dims`, and `dim_size` on the copy.\n\n**Lazy array adapter:**\n- `CoordinateTransformIndexingAdapter` in `xarray/core/indexing.py` is imported and used with `CoordinateTransformIndex`.\n- Its `_in_memory` property always returns `False`.\n- `get_duck_array()` calls `self._transform.generate_coords(dims=self._dims)` and returns the result for the specific coordinate name.\n- `transpose()` creates a new adapter with reordered dims (does not compute any values).\n- All setter methods (`__setitem__`, `_oindex_set`, `_vindex_set`) raise `TypeError` with the message \"setting values is not supported on coordinate transform arrays.\"\n- The `shape` property is derived from `self._transform.dim_size.values()`.\n\n**Concrete subclass `RangeCoordinateTransform`:**\n- Implements `forward` as `start + positions * step` and `reverse` as `(labels - start) / step`.\n- The `step` property is computed as `(stop - start) / size` (or 1.0 for empty arrays).\n- Its `equals` method uses `np.isclose` by default for floating-point comparison, with an optional `exact=True` parameter for strict equality.\n- Has a `slice()` method that creates a new transform representing a subset of the original range.\n- Uses `__slots__ = (\"_step\", \"start\", \"stop\")`.\n\n**Public API:**\n- Both `CoordinateTransform` and `CoordinateTransformIndex` are exported in `xarray/indexes/__init__.py` alongside `RangeIndex`, `PandasIndex`, `PandasMultiIndex`, `Index`, `NDPointIndex`, and `TreeAdapter`.\n- In `Variable._in_memory`, `CoordinateTransformIndexingAdapter` is explicitly checked\u2014if the variable's data is an instance of it, the adapter's `_in_memory` property (always False) is used.",
    "rubric": [
      "CoordinateTransform.__init__ converts coord_names to tuple, derives dims from dim_size keys, copies dim_size to dict, and defaults dtype to np.float64",
      "CoordinateTransform's forward, reverse, and equals methods raise NotImplementedError (effectively abstract)",
      "generate_coords uses np.meshgrid with indexing='ij' on np.arange arrays for each dimension, then calls self.forward",
      "CoordinateTransformIndex wraps a CoordinateTransform and inherits from Index",
      "CoordinateTransformIndex.sel requires method='nearest', uses self.transform.reverse() and rounds positions with np.round().astype('int')",
      "CoordinateTransformIndex.isel returns None",
      "CoordinateTransformIndex.rename does a deepcopy of the transform and updates coord_names, dims, and dim_size",
      "CoordinateTransformIndexingAdapter._in_memory always returns False",
      "CoordinateTransformIndexingAdapter.get_duck_array calls generate_coords and returns the result for a specific coordinate name",
      "CoordinateTransformIndexingAdapter setter methods raise TypeError with message about coordinate transform arrays",
      "RangeCoordinateTransform implements forward as start + positions * step and reverse as (labels - start) / step",
      "RangeCoordinateTransform.equals uses np.isclose by default with an optional exact parameter for strict equality",
      "Both CoordinateTransform and CoordinateTransformIndex are exported in xarray's public indexes API"
    ],
    "key_files": [
      "xarray/core/coordinate_transform.py",
      "xarray/core/indexes.py",
      "xarray/core/indexing.py",
      "xarray/indexes/range_index.py",
      "xarray/indexes/__init__.py"
    ],
    "source_doc": "[docstring: xarray/core/coordinate_transform.py] xarray.core.coordinate_transform.CoordinateTransform\nxarray.core.coordinate_transform.CoordinateTransform:\n  Abstract coordinate transform with dimension & coordinate names.\n\n    .. caution::\n        This API is experimental and subject to change. Please report any bugs or surprising\n        behaviour you encounter.\n\nxarray.core.coordinate_transform.CoordinateTransform.forward:\n  Perform grid -> world coordinate transformation.\n\n        Parameters\n        ----------\n        dim_positions : dict\n            Grid location(s) along each dimension (axis).\n\n        Returns\n        -------\n        coord_labels : dict\n            World coordinate labels.\n\nxarray.core.coordinate_transform.CoordinateTransform.reverse:\n  Perform world -> grid coordinate reverse transformation.\n\n        Parameters\n        ----------\n        labels : dict\n            World coordinate labels.\n\n        Returns\n        -------\n        dim_positions : dict\n            Grid relative location(s) along each dimension (axis).\n\nxarray.core.coordinate_transform.CoordinateTransform.equals:\n  Check equality with another CoordinateTransform of the same kind.\n\n        Parameters\n        ----------\n        other : CoordinateTransform\n            The other CoordinateTransform object to compare with this object.\n        exclude : frozenset of hashable, optional\n            Dimensions excluded from checking. It is None by default, (i.e.,\n            when this method is not called in the context of alignment). For a\n            n-dimensional transform this option allows a CoordinateTransform to\n            optionally ignore any dimension in ``exclude`` when comparing\n            ``self`` with ``other``. For a 1-dimensional transform this kwarg\n            can be safely ignored, as this method is not called when all of the\n            transform's dimensions are also excluded from alignment.\n\nxarray.core.coordinate_transform.CoordinateTransform.generate_coords:\n  Compute all coordinate labels at once.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in purpose/rationale descriptions ('lazy coordinate computation', 'memory optimization', 'never materialized until accessed'), but nearly all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_46",
    "repo": "xarray",
    "question": "Why does `to_index` in `AbstractCoordinates` use integer codes and levels instead of directly computing the cartesian product of actual index values?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `to_index` method in `AbstractCoordinates` uses a codes-based cartesian product algorithm when dealing with multi-dimensional coordinates. Instead of materializing the full outer-product of values (which would be expensive for large dimension sizes), it operates on integer codes and level arrays \u2014 a compressed representation. This approach avoids duplicating actual index values in memory by using `np.tile` and `np.repeat` only on small integer code arrays, then constructing a `pd.MultiIndex` from levels and codes.\n\nKey performance design decisions:\n\n1. **Fast path for single-dimension case**: When there's only one dimension, `to_index` skips the entire cartesian product computation and directly delegates to `self._data.get_index(dim)`, which calls `Index.to_pandas_index()` \u2014 for `PandasIndex`, this simply returns the already-stored `self.index` attribute.\n\n2. **Codes-based representation avoids value duplication**: The algorithm uses `pd.factorize()` to convert non-MultiIndex indexes into integer codes and unique levels. The actual cartesian product is computed only over the small integer code arrays using `np.tile`/`np.repeat`, not the actual values. The final `pd.MultiIndex` stores levels separately and references them via codes, so the level values appear only once in memory.\n\n3. **`np.fromiter` with `dtype=np.intp`**: Uses a generator expression with `np.fromiter` to collect index lengths into a numpy array with platform-native integer pointer type (`np.intp`), enabling vectorized cumulative product computation via `np.cumprod` to calculate repeat and tile counts in one pass.\n\n4. **Empty-product early-out**: If `cumprod_lengths[-1] == 0` (any dimension has size 0), it immediately sets `repeat_counts` to zeros via `np.zeros_like`, avoiding division-by-zero and unnecessary computation.\n\n5. **Lazy caching in `Indexes` class**: The `Indexes` class uses lazy-initialized properties (`_coord_name_id`, `_id_index`, `_id_coord_names`, `_dims`) that are computed on first access and cached (initialized to `None` in `__init__`).\n\n6. **Identity-based deduplication in `get_unique()`**: Uses `id()` (object identity) rather than value equality to identify unique indexes, using a `set` of `int` ids rather than performing equality comparisons between index objects.",
    "rubric": [
      "Explains that operating on integer codes avoids duplicating actual index values in memory \u2014 the levels are stored only once and referenced via codes",
      "Mentions that `pd.factorize()` is used to convert non-MultiIndex indexes into integer codes and unique levels",
      "Describes that `np.tile` and `np.repeat` are applied to the small integer code arrays rather than the full value arrays",
      "Notes the fast-path optimization for single-dimension case that skips the cartesian product entirely and delegates to `get_index(dim)`",
      "Mentions the empty-product early-out: when `cumprod_lengths[-1] == 0`, repeat counts are set to zeros to avoid division-by-zero",
      "Notes that the final result is a `pd.MultiIndex` constructed from levels and codes, which inherently stores level values only once"
    ],
    "key_files": [
      "xarray/core/coordinates.py",
      "xarray/core/indexes.py",
      "xarray/core/common.py"
    ],
    "source_doc": "[docstring: xarray/core/coordinates.py] xarray.core.coordinates.AbstractCoordinates\nxarray.core.coordinates.AbstractCoordinates.indexes:\n  Mapping of pandas.Index objects used for label based indexing.\n\n        Raises an error if this Coordinates object has indexes that cannot\n        be coerced to pandas.Index objects.\n\n        See Also\n        --------\n        Coordinates.xindexes\n\nxarray.core.coordinates.AbstractCoordinates.xindexes:\n  Mapping of :py:class:`~xarray.indexes.Index` objects\n        used for label based indexing.\n\nxarray.core.coordinates.AbstractCoordinates.to_index:\n  Convert all index coordinates into a :py:class:`pandas.Index`.\n\n        Parameters\n        ----------\n        ordered_dims : sequence of hashable, optional\n            Possibly reordered version of this object's dimensions indicating\n            the order in which dimensions should appear on the result.\n\n        Returns\n        -------\n        pandas.Index\n            Index subclass corresponding to the outer-product of all dimension\n            coordinates. This will be a MultiIndex if this object is has more\n            than more dimension.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides high-level docstrings for `indexes`, `xindexes`, and `to_index` methods. The generated answer contains extremely detailed implementation claims (specific line numbers, algorithm details, use of np.tile/np.repeat, pd.factorize, np.fromiter, etc.) that cannot be verified from the provided documentation.",
      "The documentation states to_index returns 'a MultiIndex if this object is has more than more dimension' - the answer's claim about codes-based cartesian product is plausible but entirely unsupported by the provided doc.",
      "Claims about Indexes class lazy caching, identity-based deduplication in get_unique(), and specific line numbers are all beyond the scope of the provided documentation and cannot be verified."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor leakage from a comment attributing the algorithm to pandas.core.reshape.util; nearly all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_47",
    "repo": "xarray",
    "question": "Where is `map_over_datasets` implemented, and where does the actual tree traversal logic live?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `map_over_datasets` function is defined in `xarray/core/datatree_mapping.py` and is also imported in the `DataTree` class in `xarray/core/datatree.py`. The `DataTree` class imports `map_over_datasets` from `datatree_mapping` and likely delegates its `map_over_datasets` method to the module-level function. The module-level function is imported from `xarray.core.datatree_mapping`.\n\nKey implementation details:\n\n1. **Location of traversal logic**: The actual tree traversal is performed by `group_subtrees()`, which is imported from `xarray/core/treenode.py` and called in `map_over_datasets`. The `TreeNode` class in treenode.py uses BFS with `collections.deque` in its `subtree` and `subtree_with_keys` properties, starting with `NodePath()` and using the `/` operator on `NodePath` (which extends `PurePosixPath`) to build child paths.\n\n2. **Non-DataTree argument handling**: In `map_over_datasets`, non-DataTree arguments are passed through to each node's function call unchanged. The function extracts DataTree args with `tree_args = [arg for arg in args if isinstance(arg, DataTree)]`, iterates via `group_subtrees(*tree_args)`, and for each node inserts non-DataTree args back into their original position using `node_dataset_args.insert(i, arg)`.\n\n3. **Name propagation**: The result tree's name is determined by `result_name()` imported from `xarray/core/utils.py`, called as `result_name(tree_args)`.\n\n4. **Error context**: The `add_path_context_to_errors` context manager adds a note to any exception using `e.add_note()` indicating which tree path caused the error.\n\n5. **Binary operations on DataTree**: `DataTree` inherits from `DataTreeOpsMixin`, suggesting binary operations are supported. The import of `map_over_datasets` in datatree.py indicates it is used for tree-wide operations, where non-DataTree values would be passed to every node unchanged per the argument handling logic.",
    "rubric": [
      "Identifies that `map_over_datasets` is defined in `xarray/core/datatree_mapping.py`",
      "Mentions that tree traversal is performed by `group_subtrees()` which is defined in `xarray/core/treenode.py`",
      "Notes that `group_subtrees` uses BFS with `collections.deque` and `NodePath` (which extends `PurePosixPath`) to iterate through tree nodes",
      "Explains that non-DataTree arguments are extracted, and for each node, non-DataTree args are inserted back into their original positions via `node_dataset_args.insert(i, arg)`",
      "Mentions that the `DataTree` class also has a `map_over_datasets` method that delegates to the module-level function from `datatree_mapping`",
      "Notes that result tree naming uses `result_name()` from `xarray/core/utils.py`"
    ],
    "key_files": [
      "xarray/core/datatree_mapping.py",
      "xarray/core/treenode.py",
      "xarray/core/datatree.py",
      "xarray/core/utils.py"
    ],
    "source_doc": "[docstring: xarray/core/datatree_mapping.py] xarray.core.datatree_mapping.map_over_datasets\nApplies a function to every dataset in one or more DataTree objects with\n    the same structure (ie.., that are isomorphic), returning new trees which\n    store the results.\n\n    The function will be applied to any dataset stored in any of the nodes in\n    the trees. The returned trees will have the same structure as the supplied\n    trees.\n\n    ``func`` needs to return a Dataset, tuple of Dataset objects or None in order\n    to be able to rebuild the subtrees after mapping, as each result will be\n    assigned to its respective node of a new tree via `DataTree.from_dict`. Any\n    returned value that is one of these types will be stacked into a separate\n    tree before returning all of them.\n\n    ``map_over_datasets`` is essentially syntactic sugar for the combination of\n    ``group_subtrees`` and ``DataTree.from_dict``. For example, in the case of\n    a two argument function that return one result, it is equivalent to::\n\n        results = {}\n        for path, (left, right) in group_subtrees(left_tree, right_tree):\n            results[path] = func(left.dataset, right.dataset)\n        return DataTree.from_dict(results)\n\n    Parameters\n    ----------\n    func : callable\n        Function to apply to datasets with signature:\n\n        `func(*args: Dataset, **kwargs) -> Union[Dataset, tuple[Dataset, ...]]`.\n\n        (i.e. func must accept at least one Dataset and return at least one Dataset.)\n    *args : tuple, optional\n        Positional arguments passed on to `func`. Any DataTree arguments will be\n        converted to Dataset objects via `.dataset`.\n    kwargs : dict, optional\n        Optional keyword arguments passed directly to ``func``.\n\n    Returns\n    -------\n    Result of applying `func` to each node in the provided trees, packed back\n    into DataTree objects via `DataTree.from_dict`.\n\n    See Also\n    --------\n    DataTree.map_over_datasets\n    group_subtrees\n    DataTree.from_dict",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few claims about unseen functions (result_name heuristic, group_subtrees internals, TreeIsomorphismError) rely on documentation or code not shown.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "xarray_gen_48",
    "repo": "xarray",
    "question": "How does `RenderDataTree` handle tree traversal, child truncation via `maxchildren`, and the rendering of indentation/branch characters?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `RenderDataTree` class in `xarray/core/datatree_render.py` implements tree rendering for `DataTree` objects using an iterator-based approach. Here is how it works in detail:\n\n**Core Algorithm (`__next` method):**\nThe private `__next` method is a recursive generator that traverses the tree depth-first. It maintains a `continues` tuple of booleans tracking whether each ancestor level still has more siblings to render. For each node, it yields a `Row` (a `NamedTuple` with `pre`, `fill`, and `node` fields). Children are accessed via `node.children.values()`, which returns values from a `Frozen` wrapper (an immutable `Mapping` wrapper from `xarray.core.utils`) around the internal `_children` dict.\n\n**`maxchildren` truncation logic:**\nWhen `maxchildren` is set and `nchildren > maxchildren`, children are filtered using `math.ceil`: the first `ceil(maxchildren/2)` children and the last `ceil(nchildren - maxchildren/2)` children are rendered, with an ellipsis (literal string `\"...\"`) node yielded between them via `RenderDataTree.__item(\"...\", continues, self.style)`. The `by_attr` method handles this string node specially with `isinstance(node, str)` check, yielding `f\"{fill}{node}\"` instead of accessing attributes.\n\n**`__item` static method:**\nThis constructs `Row` objects. For the root node (empty `continues` tuple), it returns `Row(\"\", \"\", node)`. For other nodes, it builds `pre` (indentation + branch character) and `fill` (indentation for continuation lines) from the `continues` tuple. Each True in `continues` maps to `style.vertical`, each False maps to `style.empty` (spaces of equal length). The last element determines whether `style.cont` (\u251c\u2500\u2500) or `style.end` (\u2514\u2500\u2500) is used as the branch character.\n\n**Style system:**\n`ContStyle` is the only concrete style provided, initializing `AbstractStyle` with Unicode characters: vertical=`\u2502   ` (U+2502 + 3 spaces), cont=`\u251c\u2500\u2500 ` (U+251C + U+2500 + U+2500 + space), end=`\u2514\u2500\u2500 ` (U+2514 + U+2500 + U+2500 + space). The `AbstractStyle.empty` property generates spaces of equal length to these strings. An assertion ensures all three strings have equal length.\n\n**Integration with DataTree repr:**\n`RenderDataTree` is imported in `xarray/core/formatting.py`, and `OPTIONS[\"display_max_children\"]` (default: 12, defined in `xarray/core/options.py`) is used as the `maxchildren` parameter. The renderer is iterated over to produce node representations.\n\n**`__str__` delegates to node:**\n`RenderDataTree.__str__` returns `str(self.node)`, meaning `str(RenderDataTree(tree))` delegates to the node's own `__str__` method, not the `by_attr` rendering.\n\n**`by_attr` supports callable attrname:**\nAlthough the type hint for `attrname` is `str`, `by_attr` also accepts a callable \u2014 it checks `callable(attrname)` and if true, calls `attrname(node)` instead of `getattr(node, attrname, \"\")`. If the result is a list or tuple, each element becomes a separate line.\n\n**`_is_last` helper:**\nA standalone generator function that wraps an iterable to yield `(item, is_last_bool)` tuples, enabling the renderer to know when it's processing the last child at each level without requiring random access to the children collection.",
    "rubric": [
      "Explains that `__next` is a recursive generator doing depth-first traversal, maintaining a `continues` tuple of booleans tracking ancestor sibling status",
      "Describes that each iteration yields a `Row` NamedTuple with `pre`, `fill`, and `node` fields",
      "Explains the `maxchildren` truncation logic: renders first `ceil(maxchildren/2)` and last `ceil(nchildren - maxchildren/2)` children with a '...' ellipsis node between them",
      "Describes the `__item` static method: root node gets empty strings; other nodes build `pre` and `fill` from the `continues` tuple using `style.vertical`/`style.empty` for indentation and `style.cont`/`style.end` for the branch character",
      "Mentions that `ContStyle` provides Unicode box-drawing characters (\u2502, \u251c\u2500\u2500, \u2514\u2500\u2500) and `AbstractStyle.empty` generates spaces of equal length",
      "Explains that `by_attr` handles string nodes (the '...' ellipsis) specially via `isinstance(node, str)` check",
      "Mentions the `_is_last` helper generator that yields `(item, is_last_bool)` tuples to detect the last child without random access"
    ],
    "key_files": [
      "xarray/core/datatree_render.py",
      "xarray/core/formatting.py",
      "xarray/core/treenode.py",
      "xarray/core/options.py"
    ],
    "source_doc": "[docstring: xarray/core/datatree_render.py] xarray.core.datatree_render.RenderDataTree\nxarray.core.datatree_render.RenderDataTree.__init__:\n  Render tree starting at `node`.\n        Keyword Args:\n            style (AbstractStyle): Render Style.\n            childiter: Child iterator. Note, due to the use of node.children.values(),\n                Iterables that change the order of children  cannot be used\n                (e.g., `reversed`).\n            maxlevel: Limit rendering to this depth.\n            maxchildren: Limit number of children at each node.\n        :any:`RenderDataTree` is an iterator, returning a tuple with 3 items:\n        `pre`\n            tree prefix.\n        `fill`\n            filling for multiline entries.\n        `node`\n            :any:`NodeMixin` object.\n        It is up to the user to assemble these parts to a whole.\n\n        Examples\n        --------\n\n        >>> from xarray import Dataset\n        >>> from xarray.core.datatree import DataTree\n        >>> from xarray.core.datatree_render import RenderDataTree\n        >>> root = DataTree.from_dict(\n        ...     {\n        ...         \"/\": Dataset({\"a\": 0, \"b\": 1}),\n        ...         \"/sub0\": Dataset({\"c\": 2, \"d\": 3}),\n        ...         \"/sub0/sub0B\": Dataset({\"e\": 4}),\n        ...         \"/sub0/sub0A\": Dataset({\"f\": 5, \"g\": 6}),\n        ...         \"/sub1\": Dataset({\"h\": 7}),\n        ...     },\n        ...     name=\"root\",\n        ... )\n\n        # Simple one line:\n\n        >>> for pre, _, node in RenderDataTree(root):\n        ...     print(f\"{pre}{node.name}\")\n        ...\n        root\n        \u251c\u2500\u2500 sub0\n        \u2502   \u251c\u2500\u2500 sub0B\n        \u2502   \u2514\u2500\u2500 sub0A\n        \u2514\u2500\u2500 sub1\n\n        # Multiline:\n\n        >>> for pre, fill, node in RenderDataTree(root):\n        ...     print(f\"{pre}{node.name}\")\n        ...     for variable in node.variables:\n        ...         print(f\"{fill}{variable}\")\n        ...\n        root\n        a\n        b\n        \u251c\u2500\u2500 sub0\n        \u2502   c\n        \u2502   d\n        \u2502   \u251c\u2500\u2500 sub0B\n        \u2502   \u2502   e\n        \u2502   \u2514\u2500\u2500 sub0A\n        \u2502       f\n        \u2502       g\n        \u2514\u2500\u2500 sub1\n            h\n\n        :any:`by_attr` simplifies attribute rendering and supports multiline:\n        >>> print(RenderDataTree(root).by_attr())\n        root\n        \u251c\u2500\u2500 sub0\n        \u2502   \u251c\u2500\u2500 sub0B\n        \u2502   \u2514\u2500\u2500 sub0A\n        \u2514\u2500\u2500 sub1\n\n        # `maxlevel` limits the depth of the tree:\n\n        >>> print(RenderDataTree(root, maxlevel=2).by_attr(\"name\"))\n        root\n        \u251c\u2500\u2500 sub0\n        \u2514\u2500\u2500 sub1\n\n        # `maxchildren` limits the number of children per node\n\n        >>> print(RenderDataTree(root, maxchildren=1).by_attr(\"name\"))\n        root\n        \u251c\u2500\u2500 sub0\n        \u2502   \u251c\u2500\u2500 sub0B\n        \u2502   ...\n        ...\n\nxarray.core.datatree_render.RenderDataTree.by_attr:\n  Return rendered tree with node attribute `attrname`.\n\n        Examples\n        --------\n\n        >>> from xarray import Dataset\n        >>> from xarray.core.datatree import DataTree\n        >>> from xarray.core.datatree_render import RenderDataTree\n        >>> root = DataTree.from_dict(\n        ...     {\n        ...         \"/sub0/sub0B\": Dataset({\"foo\": 4, \"bar\": 109}),\n        ...         \"/sub0/sub0A\": None,\n        ...         \"/sub1/sub1A\": None,\n        ...         \"/sub1/sub1B\": Dataset({\"bar\": 8}),\n        ...         \"/sub1/sub1C/sub1Ca\": None,\n        ...     },\n        ...     name=\"root\",\n        ... )\n        >>> print(RenderDataTree(root).by_attr(\"name\"))\n        root\n        \u251c\u2500\u2500 sub0\n        \u2502   \u251c\u2500\u2500 sub0B\n        \u2502   \u2514\u2500\u2500 sub0A\n        \u2514\u2500\u2500 sub1\n            \u251c\u2500\u2500 sub1A\n            \u251c\u2500\u2500 sub1B\n            \u2514\u2500\u2500 sub1C\n                \u2514\u2500\u2500 sub1Ca",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim about `__str__` delegating to `str(self.node)` which then calls `datatree_repr(self)` is not mentioned in the documentation and could be a plausible implementation detail or a hallucination \u2014 it cannot be verified from the provided docs.",
      "The claim about `by_attr` supporting callable `attrname` is not documented and is an extrapolation from likely source code knowledge, not the provided documentation.",
      "The claim about `_is_last` helper function is not mentioned in the documentation \u2014 plausible implementation detail but unverifiable.",
      "The detailed description of `__next` method internals, `__item` static method, `Row` NamedTuple, `continues` tuple, etc. are implementation details not present in the provided documentation \u2014 they are plausible but unverifiable from the docs alone.",
      "The claim about `math.ceil` logic for `maxchildren` truncation is an implementation detail not in the docs.",
      "The claim about `ContStyle` Unicode characters is plausible given the rendered output in the examples but not explicitly documented.",
      "The claim about `OPTIONS['display_max_children']` defaulting to 12 and integration with `datatree_repr` is not in the provided documentation.",
      "The description of `Frozen` wrapper from `xarray.core.utils` is not in the provided documentation.",
      "The documentation shows `by_attr()` called without arguments defaults to rendering `name`, and the answer's description is consistent with this behavior."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: 'adapted from anytree' attribution, some claims about DataTree repr details from truncated code, and reference to documentation for type hints.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_04",
    "repo": "pylint",
    "question": "How does pylint's docparams extension detect Google-style docstrings and validate that all function parameters are documented?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `test_existing_func_params_in_google_docstring` function demonstrates how pylint's docparams extension validates Google-style docstrings. Here's how the system design works:\n\n**Detection Flow:**\n1. `DocstringParameterChecker.visit_functiondef()` (in `pylint/extensions/docparams.py`) is triggered for each function definition.\n2. It calls `utils.docstringify()` which auto-detects the docstring format by instantiating each docstring type (Sphinx, Epytext, Google, Numpy) and calling `matching_sections()`. The type with the most matching sections wins.\n3. For Google-style, `GoogleDocstring.matching_sections()` counts matches across 5 regex patterns: `re_param_section`, `re_raise_section`, `re_returns_section`, `re_yields_section`, and `re_property_returns_line`. For this test function, it would return 2 (Args + Returns sections match).\n\n**Parameter Extraction via `match_param_docs()`:**\n4. `GoogleDocstring.match_param_docs()` calls `_parse_section(self.re_param_section)` which parses the \"Args:\" section. The `_parse_section` method determines minimum indentation from the section header's leading spaces plus 1 (`min_section_indent` returns `len(section_match.group(1)) + 1`).\n5. Blank lines between parameters (like between `yarg` and `zarg`) are skipped entirely (`if not line.strip(): continue`).\n6. Multi-line parameter descriptions (like `yarg`'s \"bla\\n    bla yarg\") are grouped into a single entry because continuation lines have greater indentation than `min_indentation`.\n7. Each entry is matched against `re_param_line`, which captures: group(1) = param name, group(2) = optional type in parentheses, group(3) = description. Escaped backslashes in param names are removed via `param_name.replace('\\\\', '')`.\n\n**Type Pattern Matching:**\n8. The `re_type` pattern (inherited from `SphinxDocstring`) is `[~!.]?\\w(?:\\w|\\.\\[^\\.])*`, which allows dots between word characters, enabling qualified names like `my.qualified.type`.\n9. The `re_multiple_type` pattern allows combinations of types separated by `or`, `,`, or `|`, and also supports container types with parenthesized/bracketed contents.\n\n**Validation in `check_arguments_in_docstring()`:**\n10. Expected argument names are collected from the AST node's args. For the test function: `{xarg, yarg, zarg, warg}`.\n11. `params_with_doc` and `params_with_type` are compared against expected. Since all four params have both type declarations `(int)` / `(my.qualified.type)` and descriptions, no warnings are emitted.\n12. The method also checks for \"differing\" args (documented but not in signature) using symmetric difference.\n\n**Configuration:**\nThe `.rc` file sets `accept-no-param-doc=no` (which based on the code logic means functions without parameter documentation will trigger warnings), `docstring-min-length: -1` (based on `_is_shorter_than_min_length` logic, this means no functions are skipped for being too short), and `no-docstring-rgx=^$` (only matches empty string, so no functions are skipped by name).",
    "rubric": [
      "Explains that visit_functiondef calls utils.docstringify() which auto-detects docstring format by instantiating each type (Sphinx, Epytext, Google, Numpy) and selecting the one with the highest matching_sections() count",
      "Describes that GoogleDocstring.matching_sections() counts matches across regex patterns: re_param_section, re_raise_section, re_returns_section, re_yields_section, and re_property_returns_line",
      "Explains that match_param_docs() calls _parse_section(re_param_section) to extract parameter entries from the Args/Arguments/Parameters section",
      "Describes how _parse_section uses indentation-based parsing: min_section_indent returns len(section_match.group(1)) + 1, blank lines are skipped, and continuation lines with greater indentation are grouped with the preceding parameter entry",
      "Explains that each entry is matched against re_param_line which captures param name (group 1), optional type in parentheses (group 2), and description (group 3), with backslashes removed from param names",
      "Describes how check_arguments_in_docstring collects expected argument names from the AST node's args, then compares against params_with_doc and params_with_type sets to find missing or differing parameters",
      "Mentions that the re_type pattern (from SphinxDocstring) allows qualified names like my.qualified.type via the pattern [~!.]?\\w(?:\\w|\\.[^\\.])*"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Google.test_existing_func_params_in_google_docstring\nExample of a function with correctly documented parameters and\n    return values (Google style)\n\n    Args:\n        xarg (int): bla xarg\n        yarg (my.qualified.type): bla\n            bla yarg\n\n        zarg (int): bla zarg\n        warg (my.qualified.type): bla warg\n\n    Returns:\n        float: sum",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (regex patterns, method names, parsing logic, configuration settings) that are not present in the provided documentation chunk. These are plausible extrapolations from knowledge of the pylint codebase but cannot be verified from the given doc.",
      "The documentation only shows a docstring describing a function with Google-style parameter documentation (xarg, yarg, zarg, warg) and a Returns section. The answer's claims about internal implementation are neither supported nor contradicted by this limited documentation.",
      "The answer correctly identifies the four parameters (xarg, yarg, zarg, warg) and their types (int, my.qualified.type) as shown in the documentation.",
      "The answer correctly notes that yarg has a multi-line description ('bla\\n    bla yarg') based on the indentation shown in the doc.",
      "The answer correctly identifies that the Returns section documents 'float: sum'."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor interpretive descriptions of config semantics needed slight adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_03",
    "repo": "pylint",
    "question": "Where in the codebase are `@typing.overload` decorated functions excluded from docstring parameter checks, and how does that exclusion flow work?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `@typing.overload` exclusion from docparam checks flows through the following path: When the AST walker encounters a `FunctionDef` node, it invokes `DocstringParameterChecker.visit_functiondef()` in `pylint/extensions/docparams.py`. The very first check in this method calls `checker_utils.is_overload_stub(node)`, which is defined in `pylint/checkers/utils.py`. If the node has decorators, `is_overload_stub` calls `decorated_with(node, [\"typing.overload\", \"overload\"])` to check both fully-qualified and short decorator names. The `decorated_with` function (in `pylint/checkers/utils.py`) iterates over decorator nodes, infers each one, and checks if any inferred result's `.name` or `.qname()` matches the provided list. If `is_overload_stub` returns True, `visit_functiondef` returns immediately without calling `check_functiondef_params`, `check_functiondef_returns`, or `check_functiondef_yields`. This means the two `@overload` stub methods of `Word.starts_with` are silently skipped, while the non-overloaded implementation proceeds through normal checking, where its type annotations satisfy the type documentation requirement via `args_with_annotation()` in `_check_docs_utils.py`.",
    "rubric": [
      "Mentions that `visit_functiondef` in the docparams extension checks `is_overload_stub(node)` as its first condition and returns early if true",
      "Identifies that `is_overload_stub` is defined in `pylint/checkers/utils.py` and calls `decorated_with(node, ['typing.overload', 'overload'])`",
      "Explains that `decorated_with` iterates over decorator nodes, infers each one, and checks if `.name` or `.qname()` matches the provided qualified names",
      "Notes that the early return prevents `check_functiondef_params`, `check_functiondef_returns`, and `check_functiondef_yields` from being called on overload stubs",
      "Mentions that non-overloaded implementations proceed through normal checking where type annotations can satisfy documentation requirements"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/checkers/utils.py",
      "tests/functional/ext/docparams/missing_param_doc.py",
      "tests/functional/ext/docparams/missing_param_doc.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/missing_param_doc.py] tests.functional.ext.docparams.missing_param_doc.Word\ntests.functional.ext.docparams.missing_param_doc.Word:\n  Methods decorated with `typing.overload` are excluded\n    from the docparam checks. For example: `missing-param-doc` and\n    `missing-type-doc`.\n\ntests.functional.ext.docparams.missing_param_doc.Word.starts_with:\n  Returns:\n            True if `self.word` begins with `letter`\n\n        Args:\n            letter: str",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, function names, file paths) that are not present in the documentation and cannot be verified from it. These are plausible extrapolations based on knowledge of pylint's codebase but are not supported by the provided doc chunk.",
      "The documentation only states that methods decorated with `typing.overload` are excluded from docparam checks (specifically `missing-param-doc` and `missing-type-doc`). The answer's detailed flow description is consistent with this high-level statement but adds unverifiable specifics.",
      "The claim about 'two @overload stub methods of Word.starts_with (lines 226 and 229)' and 'the non-overloaded implementation (line 231)' cannot be verified from the documentation provided.",
      "The core claim \u2014 that @typing.overload decorated methods are excluded from docparam checks \u2014 is directly supported by the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor line number references are doc-dependent; all substantive code-flow claims are verifiable from stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_01",
    "repo": "pylint",
    "question": "What is the overall architecture of the ClassChecker, including its inheritance, helper classes, module-level functions, and how it gets registered?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `ClassChecker` class in `pylint/checkers/classes/class_checker.py` is a comprehensive checker that inherits from `BaseChecker`, which itself inherits from `_ArgumentsProvider`. It is registered along with `SpecialMethodsChecker` in the `pylint/checkers/classes/__init__.py` module's `register()` function.\n\n**Architecture:**\n- `ClassChecker` inherits from `BaseChecker` (in `pylint/checkers/base_checker.py`), which inherits from `_ArgumentsProvider` (in `pylint/config/arguments_provider.py`), and `BaseChecker` is decorated with `@functools.total_ordering`.\n- Both `ClassChecker` and `SpecialMethodsChecker` are registered together via `linter.register_checker()` in `pylint/checkers/classes/__init__.py`.\n\n**Key internal classes and structures:**\n- `ScopeAccessMap`: Helper class that uses `node_frame_class()` (imported from `pylint/checkers/utils.py`) to determine which class a node belongs to.\n- `_CallSignature` and `_ParameterSignature`: Two `NamedTuple` classes. `_CallSignature` has fields `args`, `kws`, `starred_args`, `starred_kws`. `_ParameterSignature` has fields `args`, `kwonlyargs`, `varargs`, `kwargs`. They are used in functions like `_signature_from_call`, `_signature_from_arguments`, and `_definition_equivalent_to_call` for override detection.\n- `_AccessNodes`: A `TypeAlias` for `nodes.Attribute | nodes.AssignAttr`.\n\n**Notable constants and possible bugs:**\n- `ALLOWED_PROPERTIES = {\"bultins.property\", \"functools.cached_property\"}` \u2014 contains a typo: `\"bultins.property\"` is missing the 'i' in 'builtins'. The correct spelling `\"builtins.property\"` appears in `BUILTIN_DECORATORS` on the next line.\n- `INVALID_BASE_CLASSES = {\"bool\", \"range\", \"slice\", \"memoryview\"}` \u2014 used in `_is_invalid_base_class()` which checks if a class is in this set and is a builtin object.\n- `ASTROID_TYPE_COMPARATORS` maps AST node types to lambda comparison functions for comparing default values.\n\n**Configuration options:**\nThe checker defines configuration options (details in the full source), and uses `dummy_variables_rgx` (as seen in the `_has_different_parameters` function signature accepting `dummy_parameter_regex`).\n\n**The MSGS dictionary contains multiple message definitions**, including fatal (F0202), error (E0202, E0203, E0211, E0213), warning (W0201, W0212, W0213), and convention (C0202) categories.\n\n**Visitor methods:**\n- The checker likely aliases `visit_functiondef` to `visit_asyncfunctiondef` and `leave_functiondef` to `leave_asyncfunctiondef` (common pylint pattern).\n- The `cached_property` decorator is imported, suggesting its use for lazy properties.\n\n**Module-level functions visible in the code:**\n- `_signature_from_call`: Extracts a `_CallSignature` from a `nodes.Call`\n- `_signature_from_arguments`: Extracts a `_ParameterSignature` from `nodes.Arguments`\n- `_definition_equivalent_to_call`: Compares a definition signature to a call signature\n- `_is_trivial_super_delegation`: Checks if a function is a trivial delegation to super\n- `_positional_parameters`: Returns positional parameters of a method\n- `_has_different_parameters_default_value`: Checks if default values differ between original and overridden\n- `_has_different_parameters`: Checks parameter differences with dummy regex support\n- `_has_different_keyword_only_parameters`: Checks keyword-only parameter differences\n- `_different_parameters`: Comprehensive parameter difference checking\n- `_is_invalid_base_class`: Uses `INVALID_BASE_CLASSES` and `is_builtin_object`\n- `_has_data_descriptor`: Checks if a class has a data descriptor for an attribute\n- `_called_in_methods`: Checks if a function is called in specified methods\n- `_is_attribute_property`: Checks if a name is a property in a class\n- `_has_same_layout_slots`: Compares slot layouts",
    "rubric": [
      "Mentions ClassChecker inherits from BaseChecker, which inherits from _ArgumentsProvider and is decorated with @functools.total_ordering",
      "Explains that ClassChecker and SpecialMethodsChecker are both registered together via linter.register_checker() in the classes __init__.py register() function",
      "Identifies ScopeAccessMap as a helper class that uses node_frame_class to track accessed variables per class scope",
      "Describes the _CallSignature and _ParameterSignature NamedTuples and their fields (args/kws/starred_args/starred_kws and args/kwonlyargs/varargs/kwargs respectively)",
      "Mentions the ALLOWED_PROPERTIES typo: 'bultins.property' is missing the 'i' compared to the correct 'builtins.property' in BUILTIN_DECORATORS",
      "Lists key module-level functions like _signature_from_call, _definition_equivalent_to_call, _is_trivial_super_delegation, _different_parameters, _is_invalid_base_class, _has_data_descriptor, _called_in_methods",
      "Notes that visit_asyncfunctiondef is aliased to visit_functiondef and leave_asyncfunctiondef is aliased to leave_functiondef",
      "Mentions INVALID_BASE_CLASSES constant containing 'bool', 'range', 'slice', 'memoryview'"
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/classes/__init__.py",
      "pylint/checkers/base_checker.py",
      "pylint/checkers/utils.py",
      "pylint/config/arguments_provider.py"
    ],
    "source_doc": "[docstring: pylint/checkers/classes/class_checker.py] pylint.checkers.classes.class_checker.ClassChecker\npylint.checkers.classes.class_checker.ClassChecker:\n  Checker for class nodes.\n\n    Checks for :\n    * methods without self as first argument\n    * overridden methods signature\n    * access only to existent members via self\n    * attributes not defined in the __init__ method\n    * unreachable code\n\npylint.checkers.classes.class_checker.ClassChecker.visit_classdef:\n  Init visit variable _accessed.\n\npylint.checkers.classes.class_checker.ClassChecker._check_consistent_mro:\n  Detect that a class has a consistent mro or duplicate bases.\n\npylint.checkers.classes.class_checker.ClassChecker._check_proper_bases:\n  Detect that a class inherits something which is not\n        a class or a type.\n\npylint.checkers.classes.class_checker.ClassChecker._check_typing_final:\n  Detect that a class does not subclass a class decorated with\n        `typing.final`.\n\npylint.checkers.classes.class_checker.ClassChecker.leave_classdef:\n  Checker for Class nodes.\n\n        check that instance attributes are defined in __init__ and check\n        access to existent members\n\npylint.checkers.classes.class_checker.ClassChecker._check_unused_private_variables:\n  Check if private variables are never used within a class.\n\npylint.checkers.classes.class_checker.ClassChecker.visit_functiondef:\n  Check method arguments, overriding.\n\npylint.checkers.classes.class_checker.ClassChecker._check_useless_super_delegation:\n  Check if the given function node is an useless method override.\n\n        We consider it *useless* if it uses the super() builtin, but having\n        nothing additional whatsoever than not implementing the method at all.\n        If the method uses super() to delegate an operation to the rest of the MRO,\n        and if the method called is the same as the current one, the arguments\n        passed to super() are the same as the parameters that were passed to\n        this method, then the method could be removed altogether, by letting\n        other implementation to take precedence.\n\npylint.checkers.classes.class_checker.ClassChecker._check_redefined_slots:\n  Check if `node` redefines a slot which is defined in an ancestor class.\n\npylint.checkers.classes.class_checker.ClassChecker.leave_functiondef:\n  On method node, check if this method couldn't be a function.\n\n        ignore class, static and abstract methods, initializer,\n        methods overridden from a parent class.\n\npylint.checkers.classes.class_checker.ClassChecker.visit_attribute:\n  Check if the getattr is an access to a class member\n        if so, register it.\n\n        Also check for access to protected\n        class member from outside its class (but ignore __special__\n        methods)\n\npylint.checkers.classes.class_checker.ClassChecker._check_super_without_brackets:\n  Check if there is a function call on a super call without brackets.\n\npylint.checkers.classes.class_checker.ClassChecker._check_in_slots:\n  Check that the given AssignAttr node\n        is defined in the class slots.\n\npylint.checkers.classes.class_checker.ClassChecker._check_classmethod_declaration:\n  Checks for uses of classmethod() or staticmethod().\n\n        When a @classmethod or @staticmethod decorator should be used instead.\n        A message will be emitted only if the assignment is at a class scope\n        and only if the classmethod's argument belongs to the class where it\n        is defined.\n        `node` is an assign node.\n\npylint.checkers.classes.class_checker.ClassChecker._check_protected_attribute_access:\n  Given an attribute access node (set or get), check if attribute\n        access is legitimate.\n\n        Call _check_first_attr with node before calling\n        this method. Valid cases are:\n        * self._attr in a method or cls._attr in a classmethod. Checked by\n        _check_first_attr.\n        * Klass._attr inside \"Klass\" class.\n        * Klass2._attr inside \"Klass\" class when Klass2 is a base class of\n            Klass.\n\npylint.checkers.classes.class_checker.ClassChecker._is_called_inside_special_method:\n  Returns true if the node is located inside a special (aka dunder) method.\n\npylint.checkers.classes.class_checker.ClassChecker._is_classmethod:\n  Check if the given *func* node is a class method.\n\npylint.checkers.classes.class_checker.ClassChecker._is_inferred_instance:\n  Check if the inferred value of the given *expr* is an instance of\n        *klass*.\n\npylint.checkers.classes.class_checker.ClassChecker._is_class_or_instance_attribute:\n  Check if the given attribute *name* is a class or instance member of the\n        given *klass*.\n\n        Returns ``True`` if the name is a property in the given klass,\n        ``False`` otherwise.\n\npylint.checkers.classes.class_checker.ClassChecker._check_accessed_members:\n  Check that accessed members are defined.\n\npylint.checkers.classes.class_checker.ClassChecker._check_first_arg_for_type:\n  Check the name of first argument, expect:.\n\n        * 'self' for a regular method\n        * 'cls' for a class method or a metaclass regular method (actually\n          valid-classmethod-first-arg value)\n        * 'mcs' for a metaclass class method (actually\n          valid-metaclass-classmethod-first-arg)\n        * not one of the above for a static method\n\npylint.checkers.classes.class_checker.ClassChecker._check_bases_classes:\n  Check that the given class node implements abstract methods from\n        base classes.\n\npylint.checkers.classes.class_checker.ClassChecker._check_init:\n  Check that the __init__ method call super or ancestors'__init__\n        method (unless it is used for type hinting with `typing.overload`).\n\npylint.checkers.classes.class_checker.ClassChecker._check_signature:\n  Check that the signature of the two given methods match.\n\npylint.checkers.classes.class_checker.ClassChecker._uses_mandatory_method_param:\n  Check that attribute lookup name use first attribute variable name.\n\n        Name is `self` for method, `cls` for classmethod and `mcs` for metaclass.\n\npylint.checkers.classes.class_checker.ClassChecker._is_mandatory_method_param:\n  Check if nodes.Name corresponds to first attribute variable name.\n\n        Name is `self` for method, `cls` for classmethod and `mcs` for metaclass.\n        Static methods return False.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not mention line counts (2410 lines), inheritance from BaseChecker or _ArgumentsProvider, the @functools.total_ordering decorator, or the SpecialMethodsChecker registration - these are plausible code-level details but not verifiable from the provided documentation.",
      "The documentation does not mention ScopeAccessMap, _CallSignature, _ParameterSignature, _AccessNodes, or their specific line numbers - these are plausible internal implementation details but unverifiable.",
      "The typo 'bultins.property' in ALLOWED_PROPERTIES is not mentioned in the documentation - this could be a real code observation or a hallucination.",
      "INVALID_BASE_CLASSES, ASTROID_TYPE_COMPARATORS, configuration options with specific defaults, MSGS dictionary with 36 messages, line number aliases, open() method details, _ancestors_to_call function at line 2393 - none of these are verifiable from the provided documentation.",
      "The claim about _is_classmethod checking 'type == classmethod OR name == __class_getitem__' goes beyond the doc which only says 'Check if the given *func* node is a class method.'",
      "The documentation does describe the general purpose of ClassChecker and its methods consistently with what the answer states about their functionality."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Many claims about specific line numbers, exact option defaults, and details from truncated portions of the code cannot be verified from the stripped source shown, but core architectural claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_02",
    "repo": "pylint",
    "question": "Why does pylint's keyword argument matching skip positional-only parameter names early and emit a warning instead of continuing to check `parameter_name_to_index`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `name1` function in `tests/functional/a/arguments.py` tests a critical behavior in pylint's argument-checking logic: **positional-only parameters that share a name with a keyword argument should not trigger `redundant-keyword-arg`** when the function also has a `**kwargs` parameter.\n\nWhen a function is defined as `def name1(apple, /, **kwargs)`, calling it as `name1(\"Red apple\", apple=\"Green apple\")` is valid Python \u2014 the keyword `apple` gets captured into `**kwargs` rather than conflicting with the positional parameter.\n\nThe implementation in `pylint/checkers/typecheck.py` handles this during keyword argument matching. The code checks whether:\n1. The called function has a `**kwargs` parameter (`called.args.kwarg`), AND\n2. The keyword matches a positional-only parameter name (`keyword in [arg.name for arg in called.args.posonlyargs]`)\n\nIf both conditions are true, it emits the **warning-level** `kwarg-superseded-by-positional-arg` (W1117) \u2014 defined in the `MSGS` dictionary \u2014 instead of proceeding to check against `parameter_name_to_index`, which would otherwise lead to a false-positive `redundant-keyword-arg` (E1124). The code skips all subsequent checks for that keyword, which is the key mechanism preventing the false positive.\n\nThe test specifically validates that **neither** `redundant-keyword-arg` **nor** any other error is emitted for `name1(\"Red apple\", apple=\"Green apple\")` \u2014 as confirmed by the absence of any error for that line in `arguments.txt`.\n\nA related consequence of this design is visible in `name3(param1=43)` (line 318), which emits `no-value-for-parameter` as shown in `arguments.txt`: because `param1` is positional-only and handled by the early check, it's never marked as assigned in the parameters list. Since it has no default value, the final validation loop correctly detects it as unassigned.\n\nThe `parameter_name_to_index` dictionary includes both `posonlyargs` and regular `args` (the code imports `itertools` and uses chaining), so without the early guard for positional-only parameters, the keyword `apple` *would* match and trigger `redundant-keyword-arg` \u2014 which would be a false positive.",
    "rubric": [
      "Explains that without the early guard, positional-only parameter names would match in parameter_name_to_index (since it includes both posonlyargs and regular args) and trigger a false-positive redundant-keyword-arg (E1124)",
      "Explains that the check requires both a **kwargs parameter to exist AND the keyword to match a positional-only parameter name",
      "Explains that in valid Python (e.g., `name1(\"Red apple\", apple=\"Green apple\")`), the keyword goes into **kwargs rather than conflicting with the positional parameter",
      "Mentions that the code emits the warning-level kwarg-superseded-by-positional-arg (W1117) message and then uses `continue` to skip further processing of that keyword",
      "Notes that the `parameter_name_to_index` dict is built from chaining posonlyargs and args together, which is why the positional-only names would otherwise be found there"
    ],
    "key_files": [
      "pylint/checkers/typecheck.py",
      "tests/functional/a/arguments.py",
      "tests/functional/a/arguments.txt",
      "tests/functional/k/kwarg_superseded_by_positional_arg.py"
    ],
    "source_doc": "[docstring: tests/functional/a/arguments.py] tests.functional.a.arguments.name1\nPositional-only parameter with `**kwargs`.\n    Calling this function with the `apple` keyword should not emit\n    `redundant-keyword-arg` since it is added to `**kwargs`.\n\n    >>> name1(\"Red apple\", apple=\"Green apple\")\n    \"Red apple\"\n    {\"apple\": \"Green apple\"}",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (specific line numbers, variable names, code paths in pylint/checkers/typecheck.py) that are not present in the documentation and cannot be verified from it alone \u2014 these could be accurate but are extrapolations beyond the provided doc.",
      "The answer claims specific line numbers (e.g., line 297, line 318, line 3) in the test file that cannot be verified from the documentation.",
      "The answer references `name3(param1=43)` behavior which is not mentioned in the documentation at all.",
      "The core claim about the function's purpose \u2014 that calling with `apple` keyword should not emit `redundant-keyword-arg` \u2014 is correctly supported by the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims verifiable from code structure, MSGS dict, and test output; removed design rationale and comment-dependent claims",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_05",
    "repo": "pylint",
    "question": "What is the full mechanism by which pylint's docstring parameter checker validates Numpy-style parameter documentation, from format detection through parameter matching?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `test_existing_func_params_in_numpy_docstring` function in the test file is a test case for pylint's `DocstringParameterChecker` extension, which validates that function parameters are properly documented in docstrings. The function has four parameters (`xarg`, `yarg`, `zarg`, `warg`).\n\nThe checking system works as follows:\n\n1. **Docstring type detection**: The `docstringify()` function in `_check_docs_utils.py` auto-detects the docstring format by instantiating all four docstring types (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) and selecting the one with the highest `matching_sections()` count. For Numpy-style docs, the section headers (like `Parameters\\n----------`) with underlines are the distinguishing feature.\n\n2. **Class hierarchy**: `NumpyDocstring` inherits from `GoogleDocstring` (which inherits from `Docstring`). Numpy differs from Google in its `_re_section_template` which requires a `[-=]+` underline beneath section headers, its `re_param_line` regex which uses named groups (`param_name`, `param_type`, `param_desc`), and its overridden `_is_section_header()` method that detects lines matching `\\s*-+$`.\n\n3. **Parameter matching**: `NumpyDocstring.match_param_docs()` overrides the Google version. It parses each entry from `_parse_section()` and handles a special case where a parameter has description-only (no type) by checking with `re_only_desc = re.match(r\"\\s*(\\*{0,2}\\w+)\\s*:?\\n\\s*\\w*$\", entry)`. It also handles single-line type descriptions (`arg : a number type.`) that the multi-line regex pattern can't match by checking `re.match(r\"\\s*(\\*{0,2}\\w+)\\s*:.+$\", entry)`.\n\n4. **Indentation handling**: `NumpyDocstring.min_section_indent()` returns `len(section_match.group(1))` (the exact indentation of the header), whereas `GoogleDocstring.min_section_indent()` returns `len(section_match.group(1)) + 1` (one more than the header).\n\n5. **Configuration**: The test's `.rc` file sets `accept-no-param-doc=no`, `docstring-min-length: -1`, and `no-docstring-rgx=^$`. The `accept-no-param-doc=no` setting means functions with parameters but no parameter documentation will be flagged (via the `check_arguments_in_docstring` method in `DocstringParameterChecker`).\n\n6. **The checker** is registered as a pylint plugin via `pylint.extensions.docparams`, and the `DocstringParameterChecker` class has `name = \"parameter_documentation\"` and defines 13 message types (W9005 through W9021).",
    "rubric": [
      "Explains that docstringify() auto-detects docstring format by instantiating all docstring types and selecting the one with the highest matching_sections() count",
      "Describes the class hierarchy: NumpyDocstring inherits from GoogleDocstring which inherits from Docstring",
      "Explains how NumpyDocstring's _re_section_template requires a [-=]+ underline beneath section headers, distinguishing it from GoogleDocstring's colon-based header format",
      "Describes NumpyDocstring.match_param_docs() handling: parses entries from _parse_section(), uses re_param_line with named groups (param_name, param_type, param_desc), and handles special cases (description-only and single-line type descriptions)",
      "Notes the difference in min_section_indent between Numpy (exact header indentation) and Google (header indentation + 1)",
      "Mentions NumpyDocstring._is_section_header() detects lines matching \\s*-+$ while GoogleDocstring's version always returns False",
      "Explains the role of accept-no-param-doc configuration option in determining whether functions without parameter docs are flagged"
    ],
    "key_files": [
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Numpy.test_existing_func_params_in_numpy_docstring\nExample of a function with correctly documented parameters and\n    return values (Numpy style)\n\n    Parameters\n    ----------\n    xarg: int\n        bla xarg\n    yarg: my.qualified.type\n        bla yarg\n\n    zarg: int\n        bla zarg\n    warg: my.qualified.type\n        bla warg\n\n    Returns\n    -------\n    float\n        sum",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage: design rationale for indentation difference and docstring content claims that can't be verified from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_06",
    "repo": "pylint",
    "question": "Why does the docstring parameter checker skip return documentation validation for abstract methods?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The functional test file for `missing_return_doc_Google` contains a test case with an `@abc.abstractmethod` decorated method that verifies pylint's `DocstringParameterChecker` **skips return documentation validation for abstract methods**.\n\nThe core mechanism is in `check_functiondef_returns()` in `pylint/extensions/docparams.py`, which has an early-return guard:\n```python\nif (not node_doc.supports_yields and node.is_generator()) or node.is_abstract():\n    return\n```\n\nWhen `node.is_abstract()` is True (i.e., the method is decorated with `@abc.abstractmethod`), the entire return documentation check is bypassed. This means:\n1. No `redundant-returns-doc` warning is emitted even if the abstract method documents returns but has `return 10` (which would normally trigger the redundancy check if the method returned None).\n2. No `missing-return-doc` or `missing-return-type-doc` warnings are emitted.\n\nThis same `is_abstract()` guard pattern is applied consistently across multiple check methods:\n- `check_functiondef_returns()` \u2014 skips return doc validation\n- `check_functiondef_yields()` \u2014 skips yields doc validation\n- `_add_raise_message()` \u2014 handles abstract method exemption for raises differently, removing `NotImplementedError` from the set of missing exception docs rather than skipping entirely\n\nNotably, the abstract method exemption for raises is handled differently \u2014 it doesn't skip all raise checking but only forgives the undocumented `NotImplementedError`. This contrasts with returns/yields where ALL documentation checking is skipped.\n\nThe test file also includes a related case where a non-`@abc.abstractmethod` method that raises `NotImplementedError()` without return docs also passes \u2014 but for different reasons: in that case, the `visit_return` method is never triggered because the function never reaches a `return` statement.\n\nThe `GoogleDocstring` class (inheriting from `Docstring`) has `supports_yields = True`, which means the first part of the guard condition (`not node_doc.supports_yields and node.is_generator()`) is always False for Google-style docstrings. The early return in `check_functiondef_returns` for Google docs relies solely on the `node.is_abstract()` check.\n\nThe expected test output (`missing_return_doc_Google.txt`) confirms that the abstract method test produces **no warnings** \u2014 the only warnings from the file are `redundant-returns-doc` on lines 43, 52, 61 and `unreachable` on lines 95, 112, plus `useless-param-doc`/`useless-type-doc` on line 167.",
    "rubric": [
      "Explains that `check_functiondef_returns()` has an early-return guard checking `node.is_abstract()` that bypasses all return doc validation",
      "Notes that abstract methods may document returns to specify what implementations should return, even though the abstract method itself may not meaningfully return (or may return a placeholder value like `return 10`)",
      "Mentions that without this guard, abstract methods would get `redundant-returns-doc` warnings when they document returns but don't actually return a meaningful value",
      "Notes the same `is_abstract()` pattern is applied to `check_functiondef_yields()` for yield documentation",
      "Explains that the raises check (`_add_raise_message`) handles abstract methods differently \u2014 only forgiving undocumented `NotImplementedError` rather than skipping all raise checking"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_Google.py",
      "tests/functional/ext/docparams/return/missing_return_doc_Google.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_Google.py] tests.functional.ext.docparams.return.missing_return_doc_Google.Foo\ntests.functional.ext.docparams.return.missing_return_doc_Google.Foo:\n  test_ignores_return_in_abstract_method_google\n    Example of an abstract method documenting the return type that an\n    implementation should return.\n\ntests.functional.ext.docparams.return.missing_return_doc_Google.Foo.foo_method:\n  docstring ...\n\n        Returns:\n            int: Ten",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation is extremely minimal - it only shows a class Foo with a test method `test_ignores_return_in_abstract_method_google` and a `foo_method` with a Google-style Returns section documenting 'int: Ten'. The generated answer makes extensive claims about implementation details (specific line numbers, code snippets, guard conditions, other test methods, expected output files) that cannot be verified from this documentation alone.",
      "The answer references specific line numbers (253-263, 256, 268, 666, 131, 115-128, 43, 52, 61, 95, 112, 167) and file paths that are not present in the documentation chunk provided.",
      "The answer mentions `test_ignores_return_in_abstract_method_google_2` at line 131, which is not mentioned in the documentation.",
      "The answer describes specific warning outputs from `missing_return_doc_Google.txt` that are not in the documentation.",
      "However, the core claim that the test verifies abstract methods can document return types without triggering warnings is consistent with the docstring's description: 'Example of an abstract method documenting the return type that an implementation should return.'"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in design rationale claims; core technical claims are all verifiable from stripped code and test output.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_08",
    "repo": "pylint",
    "question": "How does the pylint docparams extension detect and handle redundant return documentation in Sphinx-style docstrings?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The pylint docparams extension checks for missing/redundant return documentation in Sphinx-style docstrings through a multi-step algorithm:\n\n**1. Docstring Type Detection (`docstringify` function in `_check_docs_utils.py`):**\nThe `docstringify()` function auto-detects the docstring format by instantiating all four docstring types (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) and calling `matching_sections()` on each. The one with the highest count wins. For SphinxDocstring, `matching_sections()` counts up to 5 possible regex matches: `re_param_in_docstring`, `re_raise_in_docstring`, `re_rtype_in_docstring`, `re_returns_in_docstring`, and `re_property_type_in_docstring`.\n\n**2. Return Documentation Checking (`check_functiondef_returns` in `docparams.py`):**\nThis method is called from `visit_functiondef`. It first performs two early-exit checks:\n- If `supports_yields` is False (which it is for SphinxDocstring) AND the node is a generator \u2192 skip.\n- If the node `is_abstract()` \u2192 skip entirely. This is why the `Foo.foo` abstract method decorated with `@abc.abstractmethod` in the test file does NOT trigger any warnings despite having `:returns:` and `:rtype:` documentation.\n\nIf neither early exit applies, it checks for **redundant** return docs: if the docstring has `:returns:` or `:rtype:` (detected via `has_returns()` and `has_rtype()`), but none of the function's return nodes actually return a non-None value, it emits `redundant-returns-doc` (W9008).\n\n**3. Sphinx-specific regex patterns:**\n- `re_returns_in_docstring = re.compile(r\":returns?:\")` \u2014 matches both `:return:` and `:returns:`\n- `re_rtype_in_docstring = re.compile(r\":rtype:\")` \u2014 matches `:rtype:`\n- `has_returns()` searches the docstring for the `:returns?:` pattern\n- `has_rtype()` searches for `:rtype:`\n\n**4. The `returns_something` utility:**\nDetermines if a Return node returns a meaningful value. It returns False if the return value is `None` (either bare `return` or explicit `return None`). This is used to decide whether return documentation is redundant.\n\n**5. Missing return doc checking (`visit_return`):**\nThis is a separate visitor triggered per return statement. It only fires when `accept-no-return-doc` is False (default is True). When active, it checks if the docstring has `:returns:` (or for properties, a non-directive first line) and `:rtype:`. It also skips the rtype check if the function has a PEP 484 return type annotation (`func_node.returns` or `func_node.type_comment_returns`).\n\n**6. SphinxDocstring.supports_yields is False:**\nWhen a generator is encountered, the checker skips return checking entirely (in `check_functiondef_returns`). In `visit_yield`, if `supports_yields` is False, it falls back to checking `has_returns()` and `has_rtype()` instead of yield-specific methods.\n\n**Key test outcomes for the test file:**\nThe expected output (`.txt` file) shows only two warnings: `redundant-returns-doc` at lines 44 and 52 \u2014 these are functions that document returns/rtype but only return `None`. The abstract method `Foo.foo` (line 91) is silently skipped due to the `node.is_abstract()` check, even though it has both `:returns:` and `:rtype:` documentation.",
    "rubric": [
      "Explains the docstringify() function auto-detects format by instantiating all four docstring types and selecting the one with the highest matching_sections() count",
      "Describes SphinxDocstring.matching_sections() counting up to 5 regex matches (param, raise, rtype, returns, property_type)",
      "Explains check_functiondef_returns early-exit conditions: skips generators when supports_yields is False, and skips abstract methods via is_abstract()",
      "Describes the redundant return doc check: if has_returns() or has_rtype() is True but no return node returns a non-None value, emits redundant-returns-doc",
      "Identifies the Sphinx regex patterns: re_returns_in_docstring matches :return(s): and re_rtype_in_docstring matches :rtype:",
      "Explains how returns_something() determines if a return is meaningful (returns False for bare return or explicit return None)",
      "Describes visit_return for missing-return-doc checking, including the accept-no-return-doc config gate and the PEP 484 annotation skip for rtype",
      "Notes that SphinxDocstring.supports_yields is False, affecting how generators are handled in both check_functiondef_returns and visit_yield"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_Sphinx.py",
      "tests/functional/ext/docparams/return/missing_return_doc_Sphinx.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_Sphinx.py] tests.functional.ext.docparams.return.missing_return_doc_Sphinx.Foo\ntests.functional.ext.docparams.return.missing_return_doc_Sphinx.Foo:\n  test_ignores_return_in_abstract_method_sphinx\n    Example of an abstract method documenting the return type that an\n    implementation should return.\n\ntests.functional.ext.docparams.return.missing_return_doc_Sphinx.Foo.foo:\n  docstring ...\n\n        :returns: Ten\n        :rtype: int",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation chunk is extremely minimal \u2014 it only shows a class Foo with an abstract method test and a method foo with :returns: Ten and :rtype: int. The vast majority of the answer's claims (regex patterns, line numbers, algorithm details, specific warning codes, supports_yields behavior, etc.) are not verifiable from this documentation alone.",
      "The answer claims Foo.foo is an abstract method decorated with @abc.abstractmethod, but the documentation only shows 'test_ignores_return_in_abstract_method_sphinx' as a method of Foo that is described as abstract \u2014 Foo.foo's docstring just shows :returns: Ten and :rtype: int with no indication it is abstract.",
      "The answer references specific line numbers (253, 335, 408, 73, 44, 52, 91) and file paths that cannot be verified from the provided documentation.",
      "Claims about expected test output (.txt file) showing warnings at lines 44 and 52 are not supported by the documentation provided."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Minor doc leakage from line number references and the supports_yields docstring explanation; nearly all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_07",
    "repo": "pylint",
    "question": "Where in the codebase does pylint skip return documentation checks for abstract methods, and how does this relate to the Numpy docstring functional tests?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The test file `tests/functional/ext/docparams/return/missing_return_doc_Numpy.py` is located in the functional tests for the `pylint.extensions.docparams` plugin. The `Foo` class with `@abc.abstractmethod` decorator tests that abstract methods are excluded from return documentation checks.\n\nThe check logic resides in `pylint/extensions/docparams.py` in the `DocstringParameterChecker.check_functiondef_returns()` method (line 253-263). Specifically, at line 256, the condition `node.is_abstract()` causes the method to return early (skip checking) when a function is abstract. This means the `Foo.foo` method decorated with `@abc.abstractmethod` will never trigger `redundant-returns-doc` or any return-related warnings even though it actually returns a value (`return 10`).\n\nThe Numpy docstring parsing is handled by `NumpyDocstring` class (in `pylint/extensions/_check_docs_utils.py`, line 812) which inherits from `GoogleDocstring`. The key difference is:\n- `NumpyDocstring._re_section_template` requires an underline of dashes/equals (`[-=]+`) below the section header\n- `GoogleDocstring._re_section_template` uses a colon at the end of the header\n\nThe `NumpyDocstring._is_section_header()` static method (line 927) checks for lines matching `\\s*-+$` to detect section boundaries when parsing, while `GoogleDocstring._is_section_header()` always returns `False` since it relies on indentation alone.\n\nThe docstring type auto-detection happens in the `docstringify()` function (line 259), which instantiates each docstring type class and calls `matching_sections()` to find the best match. The `NumpyDocstring` inherits `matching_sections()` from `GoogleDocstring` (line 617-628) which counts matching sections for params, raises, returns, yields, and property returns.\n\nThe `.rc` file configures this test to load `pylint.extensions.docparams` as a plugin. The expected output (`.txt` file) shows 3 `redundant-returns-doc` warnings (lines 62, 73, 98) plus `useless-param-doc` and `useless-type-doc` on line 164\u2014but notably NO warnings for the abstract method at line 109.",
    "rubric": [
      "Identifies the `check_functiondef_returns` method in `pylint/extensions/docparams.py` as the location of the skip logic",
      "Explains that `node.is_abstract()` condition causes an early return, bypassing the redundant-returns-doc check",
      "Mentions the functional test with an `@abc.abstractmethod`-decorated method (Foo.foo) that returns a value but triggers no warning",
      "Notes that Numpy docstring parsing is handled by the `NumpyDocstring` class in `_check_docs_utils.py`, which inherits from `GoogleDocstring`",
      "Explains how Numpy section headers are detected differently from Google style (underline of dashes vs. colon at end of header)"
    ],
    "key_files": [
      "tests/functional/ext/docparams/return/missing_return_doc_Numpy.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_Numpy.py] tests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo\ntests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo:\n  test_ignores_return_in_abstract_method_numpy\n    Example of an abstract method documenting the return type that an\n    implementation should return.\n\ntests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo.foo:\n  docstring ...\n\n        Returns\n        -------\n        int\n            Ten",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, method names, class hierarchies, .rc file configurations, expected output line numbers) that are not present in or supported by the provided documentation chunk. These are plausible extrapolations based on knowledge of the pylint codebase but cannot be verified from the given doc.",
      "The documentation only shows that Foo has a method test_ignores_return_in_abstract_method_numpy described as 'Example of an abstract method documenting the return type that an implementation should return' and Foo.foo has a Returns section with 'int' and 'Ten'. The answer's claims about specific line numbers (253-263, 256, 812, 927, 259, 617-628, 62, 73, 98, 164, 109) are unverifiable from the doc.",
      "The core claim that the test checks abstract methods are excluded from return documentation checks is consistent with the docstring description 'Example of an abstract method documenting the return type that an implementation should return'.",
      "The claim that Foo.foo is decorated with @abc.abstractmethod is not explicitly stated in the documentation but is a reasonable inference from the description mentioning 'abstract method'."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Minor rewrites needed for claims referencing docstring content and a test name that may come from documentation, but the vast majority of claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_09",
    "repo": "pylint",
    "question": "What is the full architecture of how a custom checker gets initialized, registered with the linter, and has its visit/leave methods dispatched during an AST walk?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `BaseChecker` class that custom checkers inherit from is defined in `pylint/checkers/base_checker.py`. It inherits from `_ArgumentsProvider` (from `pylint/config/arguments_provider.py`) and is decorated with `@functools.total_ordering`. Here are the key architectural details:\n\n1. **BaseChecker.__init__ signature**: The actual `BaseChecker.__init__` takes `linter: PyLinter` as a required (non-Optional) parameter with no default value.\n\n2. **Parent class `_ArgumentsProvider`**: When `BaseChecker.__init__` is called, it calls `_ArgumentsProvider.__init__(self, linter)` which stores the linter as `self._arguments_manager` and then calls `self._arguments_manager._register_options_provider(self)`.\n\n3. **ASTWalker dispatches visit/leave methods**: The `ASTWalker` class (in `pylint/utils/ast_walker.py`) uses `dir(checker)` to discover all methods starting with `visit_` or `leave_`. The class ID (`cid`) is derived by stripping the prefix (`member[6:]`). Events are stored in `defaultdict(list)` structures (`visit_events` and `leave_events`). During `walk()`, it gets the class name via `astroid.__class__.__name__.lower()` and calls all registered callbacks for that node type, recursing on children between visit and leave callbacks.\n\n4. **Method enabling optimization**: `ASTWalker._is_method_enabled` checks if a method has a `checks_msgs` attribute. If all messages in `checks_msgs` are disabled, the method won't be registered.\n\n5. **`add_message` delegation**: `BaseChecker.add_message` delegates to `self.linter.add_message`.\n\n6. **Checker registration**: `PyLinter.register_checker` adds the checker to `self._checkers[checker.name]` (a `defaultdict(list)`), adds it to `self._registered_checkers` (a set of tuples `(name, checker, id(checker))`).\n\n7. **Scope determination**: In `create_message_definition_from_tuple`, if the checker is a `BaseTokenChecker` or `BaseRawFileChecker`, the default scope is `WarningScope.LINE`; otherwise it's `WarningScope.NODE`.\n\n8. **Confidence defaults**: When `add_message` is called without a confidence parameter, the default value in the signature is `None`.\n\n9. **BaseChecker has `enabled` class attribute**: Set to `True` by default.\n\n10. **Walker walks depth-first**: `ASTWalker.walk()` calls visit callbacks, then recurses on `astroid.get_children()`, then calls leave callbacks. It also increments `self.nbstatements` for each statement node (`astroid.is_statement`).",
    "rubric": [
      "BaseChecker inherits from _ArgumentsProvider and is decorated with @functools.total_ordering",
      "BaseChecker.__init__ takes a required `linter: PyLinter` parameter (no default) and calls _ArgumentsProvider.__init__(self, linter)",
      "_ArgumentsProvider.__init__ stores the linter as self._arguments_manager and calls self._arguments_manager._register_options_provider(self)",
      "ASTWalker.add_checker uses dir(checker) to discover methods starting with 'visit_' or 'leave_' and strips the prefix to get the node class ID (cid)",
      "ASTWalker._is_method_enabled checks for a checks_msgs attribute and skips methods where all messages are disabled",
      "Visit and leave events are stored in defaultdict(list) structures keyed by lowercase class name",
      "ASTWalker.walk() determines the node type via astroid.__class__.__name__.lower(), calls visit callbacks, recurses on get_children(), then calls leave callbacks (depth-first)",
      "ASTWalker.walk() increments self.nbstatements for each statement node (astroid.is_statement)",
      "BaseChecker.add_message delegates to self.linter.add_message",
      "PyLinter.register_checker adds the checker to self._checkers[checker.name] and self._registered_checkers (a set of tuples)",
      "BaseChecker has an `enabled` class attribute defaulting to True",
      "create_message_definition_from_tuple sets default scope to WarningScope.LINE for BaseTokenChecker/BaseRawFileChecker, otherwise WarningScope.NODE"
    ],
    "key_files": [
      "pylint/checkers/base_checker.py",
      "pylint/utils/ast_walker.py",
      "pylint/config/arguments_provider.py",
      "pylint/lint/pylinter.py"
    ],
    "source_doc": "[doc_file: doc/development_guide/how_tos/custom_checkers.rst] Writing an AST Checker\nLet's implement a checker to make sure that all ``return`` nodes in a function\nreturn a unique constant.\nFirstly we will need to fill in some required boilerplate:\n\n.. code-block:: python\n\n  import astroid\n  from astroid import nodes\n  from typing import TYPE_CHECKING, Optional\n\n  from pylint.checkers import BaseChecker\n\n  if TYPE_CHECKING:\n      from pylint.lint import PyLinter\n\n\n  class UniqueReturnChecker(BaseChecker):\n\n      name = \"unique-returns\"\n      msgs = {\n          \"W0001\": (\n              \"Returns a non-unique constant.\",\n              \"non-unique-returns\",\n              \"All constants returned in a function should be unique.\",\n          ),\n      }\n      options = (\n          (\n              \"ignore-ints\",\n              {\n                  \"default\": False,\n                  \"type\": \"yn\",\n                  \"metavar\": \"<y or n>\",\n                  \"help\": \"Allow returning non-unique integers\",\n              },\n          ),\n      )\n\n\nSo far we have defined the following required components of our checker:\n\n* A name. The name is used to generate a special configuration\n   section for the checker, when options have been provided.\n\n* A message dictionary. Each checker is being used for finding problems\n   in your code, the problems being displayed to the user through **messages**.\n   The message dictionary should specify what messages the checker is\n   going to emit. See `Defining a Message`_ for the details about defining a new message.\n\nWe have also defined an optional component of the checker.\nThe options list defines any user configurable options.\nIt has the following format::\n\n    options = (\n        (\"option-symbol\", {\"argparse-like-kwarg\": \"value\"}),\n    )\n\n\n* The ``option-symbol`` is a unique name for the option.\n  This is used on the command line and in config files.\n  The hyphen is replaced by an underscore when used in the checker,\n  similarly to how you would use  ``argparse.Namespace``:\n\n  .. code-block:: python\n\n    if not self.linter.config.ignore_ints:\n        ...\n\nNext we'll track when we enter and leave a function.\n\n.. code-block:: python\n\n  def __init__(self, linter: Optional[\"PyLinter\"] = None) -> None:\n      super().__init__(linter)\n      self._function_stack = []\n\n  def visit_functiondef(self, node: nodes.FunctionDef) -> None:\n      self._function_stack.append([])\n\n  def leave_functiondef(self, node: nodes.FunctionDef) -> None:\n      self._function_stack.pop()\n\nIn the constructor we initialise a stack to keep a list of return nodes\nfor each function.\nAn AST checker is a visitor, and should implement\n``visit_<lowered class name>`` or ``leave_<lowered class name>``\nmethods for the nodes it's interested in.\nIn this case we have implemented ``visit_functiondef`` and ``leave_functiondef``\nto add a new list of return nodes for this function,\nand to remove the list of return nodes when we leave the function.\n\nFinally we'll implement the check.\nWe will define a ``visit_return`` function,\nwhich is called with an ``.astroid.nodes.Return`` node.\n\n.. _astroid_extract_node:\n.. TODO We can shorten/remove this bit once astroid has API docs.\n\nWe'll need to be able to figure out what attributes an\n``.astroid.nodes.Return`` node has available.\nWe can use ``astroid.extract_node`` for this::\n\n  >>> node = astroid.extract_node(\"return 5\")\n  >>> node\n  <Return l.1 at 0x7efe62196390>\n  >>> help(node)\n  >>> node.value\n  <Const.int l.1 at 0x7efe62196ef0>\n\nWe could also construct a more complete example::\n\n  >>> node_a, node_b = astroid.extract_node(\"\"\"\n  ... def test():\n  ...     if True:\n  ...         return 5 #@\n  ...     return 5 #@\n  ... \"\"\")\n  >>> node_a.value\n  <Const.int l.4 at 0x7efe621a74e0>\n  >>> node_a.value.value\n  5\n  >>> node_a.value.value == node_b.value.value\n  True\n\nFor ``astroid.extract_node``, you can use ``#@`` at the end of a line to choose which statements will be extracted into nodes.\n\nFor more information on ``astroid.extract_node``,\nsee the `astroid documentation <https://pylint.readthedocs.io/projects/astroid/en/latest/>`_.\n\nNow we know how to use the astroid node, we can implement our check.\n\n.. code-block:: python\n\n  def visit_return(self, node: nodes.Return) -> None:\n      if not isinstance(node.value, nodes.Const):\n          return\n      for other_return in self._function_stack[-1]:\n          if node.value.value == other_return.value.value and not (\n              self.linter.config.ignore_ints and node.value.pytype() == int\n          ):\n              self.add_message(\"non-unique-returns\", node=node)\n\n      self._function_stack[-1].append(node)\n\nOnce we have established that the source code has failed our check,\nwe use ``~.BaseChecker.add_message`` to emit our failure message.\n\nFinally, we need to register the checker with pylint.\nAdd the ``register`` function to the top level of the file.\n\n.. code-block:: python\n\n  def register(linter: \"PyLinter\") -> None:\n      \"\"\"This required method auto registers the checker during initialization.\n      :param linter: The linter to register the checker to.\n      \"\"\"\n      linter.register_checker(UniqueReturnChecker(linter))\n\nWe are now ready to debug and test our checker!",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 1 states that BaseChecker.__init__ takes `linter: PyLinter` as a required (non-Optional) parameter with no default value, and says the documentation example shows `Optional[\"PyLinter\"] = None` which 'doesn't match the actual base class signature.' The documentation does show `def __init__(self, linter: Optional[\"PyLinter\"] = None) -> None:` but this is the __init__ of UniqueReturnChecker (the custom checker subclass), not necessarily BaseChecker itself. The answer's claim about the actual BaseChecker signature is plausible but not verifiable from the provided documentation alone.",
      "Most claims (2-10) about internal implementation details like _ArgumentsProvider, ASTWalker, add_message delegation, scope determination, confidence defaults, etc. are not mentioned in the documentation at all. These are plausible code-level details but cannot be verified from the provided doc chunk.",
      "The documentation does mention that visit/leave methods should be named `visit_<lowered class name>` or `leave_<lowered class name>`, which is consistent with claim 3 about ASTWalker discovering methods by prefix, but the specific implementation details go far beyond what's documented.",
      "The documentation does show `self.add_message(\"non-unique-returns\", node=node)` which is consistent with claim 5 about add_message, but the delegation chain details are not in the doc.",
      "The documentation does show `register` function calling `linter.register_checker(UniqueReturnChecker(linter))` which is consistent with claim 6, but the internal details of register_checker are not documented here."
    ],
    "strip_verify_leakage": 0.24,
    "strip_verify_summary": "Several claims about truncated code internals, documentation examples, and string values from constants were removed or rewritten; most architectural claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_10",
    "repo": "pylint",
    "question": "Why does the unpacking checker skip validation when inference produces more than one result?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `TestBase` class exists in the test file as part of a test case for pylint's unpacking-non-sequence check, testing method resolution in class hierarchies. The test structure shows that the unpacking check is only performed when inference produces exactly one result (`len(inferred) == 1`).\n\nThe `flow_control_false_positive` function in the test file uses `platform.uname()` and demonstrates a case where multiple inference results would exist. The function name suggests it tests against false positives in flow control scenarios.\n\nFor the `TestBase`/`Test` hierarchy specifically: `TestBase.test(data)` returns `data` (which would be `None` when called as `self.test(None)`), but `Test` overrides `test` to return `(1, 2, 3)`. In `Test.__init__`, `self.test(None)` is called and the result is unpacked into `(self.aaa, self.bbb, self.ccc)`. Since `Test` overrides the `test` method to return a 3-tuple, astroid's inference engine would resolve `self.test(None)` to `Test.test`, producing a tuple `(1, 2, 3)` that matches the unpacking target.\n\nThe `_check_unpacking` method has multiple early-return guards: it skips if the node is inside an abstract class (`is_inside_abstract_class`), inside a comprehension, if the inferred value is `Uninferable`, or if the value comes from a variable-length argument (`*args`).",
    "rubric": [
      "Explains that the `len(inferred) == 1` guard prevents false positives when multiple inference results exist (ambiguous inference)",
      "References the `flow_control_false_positive` test case or `platform.uname()` as an example where multiple inference results would cause incorrect warnings",
      "Mentions that the check in `visit_assign` only calls `_check_unpacking` when exactly one inferred value is present",
      "Explains or implies that multiple inference results mean the checker cannot confidently determine whether the unpacking is valid",
      "Mentions the TestBase/Test hierarchy or similar case where method resolution could produce multiple candidates but inference resolves correctly to one result"
    ],
    "key_files": [
      "pylint/checkers/variables.py",
      "tests/functional/u/unpacking/unpacking_non_sequence.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/u/unpacking/unpacking_non_sequence.py] tests.functional.u.unpacking.unpacking_non_sequence.TestBase\ntests.functional.u.unpacking.unpacking_non_sequence.TestBase:\n  base class with `test` method implementation\n\ntests.functional.u.unpacking.unpacking_non_sequence.TestBase.test:\n  default implementation",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that TestBase is a 'base class with `test` method implementation' and that TestBase.test is a 'default implementation'. The answer makes extensive claims about specific line numbers, method behaviors, return values, and implementation details (e.g., `visit_assign` at line 2167, `len(inferred) == 1` guard, `TestBase.test(data)` returning `data`, `Test.test` returning `(1, 2, 3)`) that are not present in the provided documentation.",
      "The claims about `_check_unpacking` having specific early-return guards (abstract class, comprehension, Uninferable, *args) are not supported by the documentation.",
      "The claim about `flow_control_false_positive` function at lines 124-132 and `platform.uname()` is not in the documentation.",
      "While these claims may be plausible extrapolations from the actual source code, they cannot be verified against the provided documentation chunk alone."
    ],
    "strip_verify_leakage": 0.35,
    "strip_verify_summary": "Several claims about design rationale (performance optimization, historical false positive explanation) rely on documentation/comments not present in stripped code, while code structure claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_11",
    "repo": "pylint",
    "question": "Where is the docstring parameter checker defined and how does it integrate with pylint's linter system?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `DocstringParameterChecker` class is defined in `pylint/extensions/docparams.py` and its docstring parsing utilities live in `pylint/extensions/_check_docs_utils.py`. The checker is registered with pylint's linter via the `register()` function at line 696-697 of `docparams.py`, which calls `linter.register_checker(DocstringParameterChecker(linter))`. The checker's `name` attribute is `\"parameter_documentation\"` (line 45), and it inherits from `BaseChecker` in `pylint/checkers/base_checker.py`.\n\nThe `docstringify()` function in `_check_docs_utils.py` (line 259-274) determines the docstring type by instantiating all four docstring classes (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) and selecting the one with the highest `matching_sections()` count. If none matches (all return 0), it falls back to the `default_type` parameter.\n\nThe `DOCSTRING_TYPES` dictionary (line 931-937) maps five string keys \u2014 `\"sphinx\"`, `\"epytext\"`, `\"google\"`, `\"numpy\"`, and `\"default\"` \u2014 to their respective classes. Notably, `EpytextDocstring` (line 481) inherits from `SphinxDocstring` and reuses its regex patterns by replacing the leading `:` with `@` (e.g., `SphinxDocstring.re_param_raw.replace(\":\", \"@\", 1)`). The `NumpyDocstring` class (line 812) inherits from `GoogleDocstring` and does NOT redefine `re_keyword_param_section`, so it inherits Google's colon-based keyword parameter section regex.\n\nKey implementation details:\n- `SphinxDocstring.supports_yields` is `False` (line 408)\n- `GoogleDocstring.supports_yields` and `NumpyDocstring.supports_yields` are both `True` (lines 615, 878)\n- The `Docstring.__init__` method (line 292-294) calls `expandtabs()` on the docstring text\n- `GoogleDocstring.min_section_indent` returns `len(section_match.group(1)) + 1` (line 764) while `NumpyDocstring.min_section_indent` returns `len(section_match.group(1))` (line 924)\n- `NumpyDocstring._is_section_header` detects lines matching `r\"\\s*-+$\"` (line 928), while `GoogleDocstring._is_section_header` always returns `False` (line 770)\n- The checker skips functions decorated with `typing.overload` via `checker_utils.is_overload_stub(node)` (line 198)\n- Constructor names are `{\"__init__\", \"__new__\"}` (line 189)\n- The `_add_raise_message` method automatically removes `NotImplementedError` from missing exceptions for abstract methods (line 667-669)\n- `visit_asyncfunctiondef` is set equal to `visit_functiondef` (line 218), and `visit_yieldfrom` equals `visit_yield` (line 398)",
    "rubric": [
      "DocstringParameterChecker is defined in pylint/extensions/docparams.py",
      "It inherits from BaseChecker",
      "It is registered via the register() function which calls linter.register_checker(DocstringParameterChecker(linter))",
      "Its name attribute is 'parameter_documentation'",
      "Docstring parsing utilities (docstringify, docstring classes) live in pylint/extensions/_check_docs_utils.py",
      "docstringify() determines docstring type by instantiating all four docstring classes and selecting the one with highest matching_sections() count",
      "DOCSTRING_TYPES maps 'sphinx', 'epytext', 'google', 'numpy', and 'default' to their respective classes",
      "EpytextDocstring inherits from SphinxDocstring and replaces leading ':' with '@' in regex patterns",
      "NumpyDocstring inherits from GoogleDocstring",
      "visit_asyncfunctiondef is set equal to visit_functiondef"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/checkers/base_checker.py"
    ],
    "source_doc": "[doc_file: doc/user_guide/checkers/extensions.rst] Parameter Documentation checker Documentation\nIf you document the parameters of your functions, methods and constructors and\ntheir types systematically in your code this optional component might\nbe useful for you. Sphinx style, Google style, and Numpy style are supported.\n(For some examples, see https://pypi.org/project/sphinxcontrib-napoleon/ .)\n\nYou can activate this checker by adding the line::\n\n    load-plugins=pylint.extensions.docparams\n\nto the ``MAIN`` section of your ``.pylintrc``.\n\nThis checker verifies that all function, method, and constructor docstrings\ninclude documentation of the\n\n* parameters and their types\n* return value and its type\n* exceptions raised\n\nand can handle docstrings in\n\n* Sphinx style (``param``, ``type``, ``return``, ``rtype``,\n  ``raise`` / ``except``)::\n\n   def function_foo(x, y, z):\n       '''function foo ...\n\n       :param x: bla x\n       :type x: int\n\n       :param y: bla y\n       :type y: float\n\n       :param int z: bla z\n\n       :return: sum\n       :rtype: float\n\n       :raises OSError: bla\n       '''\n       return x + y + z\n\n* or the Google style (``Args:``, ``Returns:``, ``Raises:``)::\n\n   def function_foo(x, y, z):\n       '''function foo ...\n\n       Args:\n           x (int): bla x\n           y (float): bla y\n\n           z (int): bla z\n\n       Returns:\n           float: sum\n\n       Raises:\n           OSError: bla\n       '''\n       return x + y + z\n\n* or the Numpy style (``Parameters``, ``Returns``, ``Raises``)::\n\n   def function_foo(x, y, z):\n       '''function foo ...\n\n       Parameters\n       ----------\n       x: int\n           bla x\n       y: float\n           bla y\n\n       z: int\n           bla z\n\n       Returns\n       -------\n       float\n           sum\n\n       Raises\n       ------\n       OSError\n           bla\n       '''\n       return x + y + z\n\n\nYou'll be notified of **missing parameter documentation** but also of\n**naming inconsistencies** between the signature and the documentation which\noften arise when parameters are renamed automatically in the code, but not in\nthe documentation.\n**Note:** by default docstrings of private and magic methods are not checked.\nTo change this behaviour (for example, to also check ``__init__``) add\n``no-docstring-rgx=^(?!__init__$)_`` to the ``BASIC`` section of your ``.pylintrc``.\n\nConstructor parameters can be documented in either the class docstring or\nthe ``__init__`` docstring, but not both::\n\n    class ClassFoo(object):\n        '''Sphinx style docstring foo\n\n        :param float x: bla x\n\n        :param y: bla y\n        :type y: int\n        '''\n        def __init__(self, x, y):\n            pass\n\n    class ClassBar(object):\n        def __init__(self, x, y):\n            '''Google style docstring bar\n\n            Args:\n                x (float): bla x\n                y (int): bla y\n            '''\n            pass\n\nIn some cases, having to document all parameters is a nuisance, for instance if\nmany of your functions or methods just follow a **common interface**. To remove\nthis burden, the checker accepts missing parameter documentation if one of the\nfollowing phrases is found in the docstring:\n\n* For the other parameters, see\n* For the parameters, see\n\n(with arbitrary whitespace between the words). Please add a link to the\ndocstring defining the interface, e.g. a superclass method, after \"see\"::\n\n   def callback(x, y, z):\n       '''Sphinx style docstring for callback ...\n\n       :param x: bla x\n       :type x: int\n\n       For the other parameters, see\n       :class:`MyFrameworkUsingAndDefiningCallback`\n       '''\n       return x + y + z\n\n   def callback(x, y, z):\n       '''Google style docstring for callback ...\n\n       Args:\n           x (int): bla x\n\n       For the other parameters, see\n       :class:`MyFrameworkUsingAndDefiningCallback`\n       '''\n       return x + y + z\n\nNaming inconsistencies in existing parameter and their type documentations are\nstill detected.\n\nSee also :ref:`parameter_documentation checker's options' documentation <parameter_documentation-options>`",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains numerous specific implementation details (line numbers, method names, class hierarchies, regex patterns) that cannot be verified from the provided documentation chunk alone. These appear to be code-level details that go beyond what the documentation describes.",
      "The answer mentions EpytextDocstring support, which is not mentioned in the documentation (the doc only mentions Sphinx, Google, and Numpy styles). While this might exist in the code, the documentation explicitly states 'Sphinx style, Google style, and Numpy style are supported.'",
      "The claim about DOCSTRING_TYPES having five keys including 'epytext' and 'default' cannot be verified from the documentation.",
      "The claim that SphinxDocstring.supports_yields is False and that 'Sphinx-style docstrings use the returns section to document generators' is not addressed in the documentation.",
      "None of the specific line numbers, method implementations, or class details can be verified against the provided documentation."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the docstring-based explanation of what supports_yields=False means was documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_12",
    "repo": "pylint",
    "question": "How does the docstring parameter checker auto-detect the docstring style and validate function parameters across different formats?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The Parameter Documentation checker (`DocstringParameterChecker`) in `pylint/extensions/docparams.py` supports checking Sphinx, Epytext, Google, and Numpy style docstrings through a polymorphic class hierarchy. Here's how it works:\n\n**Architecture:**\n- `DocstringParameterChecker` extends `BaseChecker` (from `pylint/checkers/base_checker.py`), which itself extends `_ArgumentsProvider`.\n- The checker's `name` is `\"parameter_documentation\"` and it registers via a `register()` function that calls `linter.register_checker()`.\n- It uses the `_check_docs_utils` module which contains the docstring parsing hierarchy: base `Docstring` \u2192 `SphinxDocstring` \u2192 `EpytextDocstring`, and separately `Docstring` \u2192 `GoogleDocstring` \u2192 `NumpyDocstring`.\n\n**Docstring Style Detection (Auto-detection):**\nThe `docstringify()` function in `_check_docs_utils.py` selects the best docstring parser by instantiating all four parsers (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`), calling `matching_sections()` on each, and returning the one with the highest count. If none match (all return 0), it falls back to the `default_type` configured via the `default-docstring-type` option (choices: `sphinx`, `epytext`, `google`, `numpy`, `default`).\n\n**Parameter Checking Algorithm:**\n1. `visit_functiondef` is called for function/method/async function definitions (via `visit_asyncfunctiondef = visit_functiondef`).\n2. It first skips overload stubs (`@typing.overload`), functions matching `no-docstring-rgx`, and functions shorter than `docstring-min-length`.\n3. `check_arguments_in_docstring()` collects expected argument names from `args`, `posonlyargs`, and `kwonlyargs`, prepends `*` to vararg and `**` to kwarg names.\n4. Arguments matching the `ignored_argument_names` regex config are placed in `expected_but_ignored_argument_names` and excluded from missing-param checks but trigger `useless-param-doc`/`useless-type-doc` if documented.\n5. `params_with_type` is augmented with `args_with_annotation()` results - so PEP 484 type annotations count as type documentation.\n6. Variadic args (`*args`, `**kwargs`) don't need type documentation by default (added to `not_needed_type_in_docstring`).\n7. If ALL parameters are missing both doc and type, `missing-any-param-doc` (W9021) is emitted instead of individual `missing-param-doc` + `missing-type-doc`.\n\n**Variadic Arg Handling:**\nIn `_compare_missing_args`, if a parameter like `*args` appears as expected, the checker also accepts just `args` (without asterisks) in the found names via `name.replace(\"*\", \"\")` check. Similarly, `_compare_different_args` normalizes expected names by stripping asterisks when the stripped version is found in the docstring.\n\n**Constructor Handling:**\nFor `__init__` and `__new__` methods, the checker looks at both the class docstring and the method docstring. If both have parameter documentation, it emits `multiple-constructor-doc` (W9005). If either has parameters, the other is allowed to have none (`node_allow_no_param` / `class_allow_no_param` cross-referencing).\n\n**Raise Checking:**\nThe `visit_raise` method uses `possible_exc_types()` which handles: direct `raise ExcType`, re-raising in `except` handlers (bare `raise`), and inferring return types from factory functions. It supports ancestor matching (documenting a parent exception type satisfies the check) and has a special case for Python 3.13's `re.error`/`re.PatternError` aliasing. The `possible_exc_types()` function filters exceptions using `utils.node_ignores_exception()`.\n\n**Docstring Tolerance:**\nThe `params_documented_elsewhere()` method checks for text matching `\"For the (other)? parameters, see\"` in the docstring, which suppresses all missing parameter warnings.",
    "rubric": [
      "Explains that docstringify() instantiates all four parsers (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) and picks the one with the highest matching_sections() count",
      "Mentions the fallback to default_type when no parser matches (all return 0), with choices being sphinx/epytext/google/numpy/default",
      "Describes the class hierarchy: Docstring base class, SphinxDocstring extends it, EpytextDocstring extends SphinxDocstring, GoogleDocstring extends Docstring, NumpyDocstring extends GoogleDocstring",
      "Explains that check_arguments_in_docstring collects expected args from args/posonlyargs/kwonlyargs and prepends */'' to vararg/kwarg names",
      "Mentions that PEP 484 type annotations (via args_with_annotation) count as type documentation and augment params_with_type",
      "Explains the missing-any-param-doc logic: emitted when ALL parameters are missing both doc and type, instead of individual missing-param-doc/missing-type-doc messages",
      "Describes how variadic args are handled: _compare_missing_args accepts names without asterisks via name.replace('*', '') check",
      "Explains constructor handling: for __init__/__new__, both class and method docstrings are checked, and multiple-constructor-doc is emitted if both have params"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/checkers/base_checker.py"
    ],
    "source_doc": "[doc_file: doc/user_guide/checkers/extensions.rst] Parameter Documentation checker Messages\n:multiple-constructor-doc (W9005): *\"%s\" has constructor parameters documented in class and __init__*\n  Please remove parameter declarations in the class or constructor.\n:missing-raises-doc (W9006): *\"%s\" not documented as being raised*\n  Please document exceptions for all raised exception types.\n:redundant-returns-doc (W9008): *Redundant returns documentation*\n  Please remove the return/rtype documentation from this method.\n:redundant-yields-doc (W9010): *Redundant yields documentation*\n  Please remove the yields documentation from this method.\n:missing-return-doc (W9011): *Missing return documentation*\n  Please add documentation about what this method returns.\n:missing-return-type-doc (W9012): *Missing return type documentation*\n  Please document the type returned by this method.\n:missing-yield-doc (W9013): *Missing yield documentation*\n  Please add documentation about what this generator yields.\n:missing-yield-type-doc (W9014): *Missing yield type documentation*\n  Please document the type yielded by this method.\n:missing-param-doc (W9015): *\"%s\" missing in parameter documentation*\n  Please add parameter declarations for all parameters.\n:missing-type-doc (W9016): *\"%s\" missing in parameter type documentation*\n  Please add parameter type declarations for all parameters.\n:differing-param-doc (W9017): *\"%s\" differing in parameter documentation*\n  Please check parameter names in declarations.\n:differing-type-doc (W9018): *\"%s\" differing in parameter type documentation*\n  Please check parameter names in type declarations.\n:useless-param-doc (W9019): *\"%s\" useless ignored parameter documentation*\n  Please remove the ignored parameter documentation.\n:useless-type-doc (W9020): *\"%s\" useless ignored parameter type documentation*\n  Please remove the ignored parameter type documentation.\n:missing-any-param-doc (W9021): *Missing any documentation in \"%s\"*\n  Please add parameter and/or type documentation.\n\n\n.. _pylint.extensions.redefined_loop_name:",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is only a message listing for the Parameter Documentation checker. The generated answer contains extensive implementation details (architecture, algorithm, variadic arg handling, constructor handling, raise checking, docstring tolerance) that go far beyond what the documentation chunk covers.",
      "However, the message codes and descriptions mentioned in the answer (W9005, W9006, W9008, W9010-W9021) are consistent with the documentation.",
      "The answer's claims about internal implementation details (docstringify() function, matching_sections(), _compare_missing_args, params_documented_elsewhere, etc.) cannot be verified from the provided documentation but appear to be plausible code-level details rather than contradictions.",
      "The answer correctly identifies the checker messages and their purposes as described in the documentation."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rationale statements and one claim about NotImplementedError/abstract methods needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_13",
    "repo": "pylint",
    "question": "What are the five checkers for `consider-iterating-dictionary`, `trailing-newlines`, `invalid-length-returned`, `multiple-constructor-doc`, and `too-complex`, and how is each architecturally implemented?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The source code reveals five checkers/features. Here's how they are architecturally implemented in the codebase:\n\n1. **`consider-iterating-dictionary` (C0201)**: Implemented in `pylint/checkers/refactoring/recommendation_checker.py` within the `RecommendationChecker` class (which extends `checkers.BaseChecker`). The check is triggered in the `visit_call` method via `_check_consider_iterating_dictionary`. It specifically checks if `node.func.attrname == \"keys\"` and uses pattern matching (`match utils.safe_infer(node.func)`) to verify that the inferred bound method is on a `nodes.Dict()` instance. It handles both `for` loop/comprehension contexts and `in`/`not in` comparisons. The message confidence is `INFERENCE`. The `RecommendationChecker` is registered via `pylint/checkers/refactoring/__init__.py`'s `register()` function.\n\n2. **`trailing-newlines` (C0305)**: Implemented in `pylint/checkers/format.py` within the `FormatChecker` class, which inherits from both `BaseTokenChecker` and `BaseRawFileChecker`. The detection logic is in `process_tokens()`: it tracks `last_blank_line_num` (set when a `tokenize.NL` token has an empty stripped line), and after processing all tokens, if `line_num == last_blank_line_num and line_num > 0`, it emits the message. The `line_num > 0` condition means it does not trigger on empty files.\n\n3. **`invalid-length-returned` (E0303)**: Implemented in `pylint/checkers/classes/special_methods_checker.py` within the `SpecialMethodsChecker` class (name=\"classes\"). The `_check_len` method performs two checks: first if the return is not an int (via `_is_int` which checks for both `_is_wrapped_type(node, \"int\")` and `isinstance(node, nodes.Const) and isinstance(node.value, int)`), and second if the value is a negative `Const` (`inferred.value < 0`). The checker uses a `_protocol_map` dictionary that maps special method names (like `\"__len__\"`) to their respective check methods. The `_safe_infer_call_result` helper function is used to safely infer the return value, returning `None` if inference fails, no values are inferred, or there's ambiguity (more than one node inferred).\n\n4. **`multiple-constructor-doc` (W9005)**: Implemented in `pylint/extensions/docparams.py` within the `DocstringParameterChecker` class (name=\"parameter_documentation\"). The `check_single_constructor_params` method is called from `check_functiondef_params` when the function is a constructor and the class node is available, receiving both the class docstring and the init docstring.\n\n5. **McCabe complexity (`too-complex`, R1260)**: Implemented in `pylint/extensions/mccabe.py` as the `McCabeMethodChecker` class (name=\"design\"). It imports `PathGraph` and `PathGraphingAstVisitor` from the external `mccabe` package and subclasses them. The custom `PathGraphingAstVisitor` overrides the dispatch mechanism to work with astroid nodes (visiting by class name like `visitFunctionDef`, `visitMatch`, etc.). The `visit_module` method creates a `PathGraphingAstVisitor`, processes all child nodes, then checks each graph's `complexity()` against `self.linter.config.max_complexity` (default: 10). The extension is registered via a `register()` function that calls `linter.register_checker(McCabeMethodChecker(linter))`.",
    "rubric": [
      "Identifies `consider-iterating-dictionary` (C0201) as implemented in the `RecommendationChecker` class (extending `BaseChecker`) with detection in `_check_consider_iterating_dictionary` triggered from `visit_call`, using pattern matching on `safe_infer` to check for `BoundMethod` on `nodes.Dict`, handling for-loop/comprehension and in/not-in contexts, with INFERENCE confidence.",
      "Identifies `trailing-newlines` (C0305) as implemented in the `FormatChecker` class (inheriting from `BaseTokenChecker` and `BaseRawFileChecker`) with detection in `process_tokens` by tracking `last_blank_line_num` via `tokenize.NL` tokens and checking `line_num == last_blank_line_num and line_num > 0` (skipping empty files).",
      "Identifies `invalid-length-returned` (E0303) as implemented in the `SpecialMethodsChecker` class with detection in `_check_len`, checking both that the return value is an int (via `_is_int`) and that it is not a negative `Const`, using a `_protocol_map` to dispatch checks by special method name.",
      "Identifies `multiple-constructor-doc` (W9005) as implemented in the `DocstringParameterChecker` extension class with detection in `check_single_constructor_params`, which checks if both class and init docstrings have parameters.",
      "Identifies `too-complex` (R1260) as implemented in the `McCabeMethodChecker` extension class (name='design'), which subclasses `PathGraph` and `PathGraphingAstVisitor` from the external `mccabe` package, overrides dispatch to work with astroid node class names, and compares complexity against `max_complexity` (default 10) in `visit_module`."
    ],
    "key_files": [
      "pylint/checkers/refactoring/recommendation_checker.py",
      "pylint/checkers/format.py",
      "pylint/checkers/classes/special_methods_checker.py",
      "pylint/extensions/mccabe.py",
      "pylint/extensions/docparams.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/1/1.6/summary.rst] New checkers\n* We added a new recommendation check, ``consider-iterating-dictionary``,\n  which is emitted when a dictionary is iterated by using ``.keys()``.\n\n  For instance, the following code would trigger this warning, since\n  the dictionary's keys can be iterated without calling the method explicitly.\n\n  .. code-block:: python\n\n      for key in dictionary.keys():\n          ...\n\n      # Can be refactored to:\n      for key in dictionary:\n          ...\n\n* ``trailing-newlines`` check was added, which is emitted when a file has trailing newlines.\n\n\n* ``invalid-length-returned`` check was added, which is emitted when the ``__len__``\n  special method returns something else than a non-negative number. For instance, this\n  example is triggering it::\n\n      class Container(object):\n          def __len__(self):\n              return self._items # Oups, forgot to call len() over it.\n\n\n\n* Add a new check to the *check_docs* extension for looking for duplicate\n  constructor parameters in a class constructor docstring or in a class docstring.\n\n  The check ``multiple-constructor-doc`` is emitted when the parameter is documented\n  in both places.\n\n\n* We added a new extension plugin, ``pylint.extensions.mccabe``, which can be used\n  for warning about the complexity in the code.\n\n  You can enable it as in::\n\n      $ pylint module_or_project --load-plugins=pylint.extensions.mccabe\n\n  See more at :ref:`pylint.extensions.mccabe`",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor documentation-based rationale (empty files as __init__.py markers) and truncated code details needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_14",
    "repo": "pylint",
    "question": "Why does pylint's docstring checker handle generators differently for Sphinx-style vs Google/Numpy-style docs, and what other false-positive prevention mechanisms exist in the checker extensions?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The documentation chunk describes multiple bug fixes and features in pylint 1.7. Here's why these design decisions were made, based on the actual source code:\n\n**1. `redundant-returns-doc` false positive fix for generators:**\nIn `check_functiondef_returns` (docparams.py:256), the guard condition `if (not node_doc.supports_yields and node.is_generator()) or node.is_abstract()` was added. The rationale is that Sphinx-style docstrings (`SphinxDocstring`) set `supports_yields = False` (line 408 of _check_docs_utils.py), meaning they use `:returns:` to document both returns and yields. For generators using Sphinx-style docs, if `supports_yields` is False and the function is a generator, the method returns early to avoid a false positive. GoogleDocstring and NumpyDocstring both set `supports_yields = True` since they have separate `Yields:` sections.\n\n**2. `missing-param-doc`/`missing-type-doc` false positive for 'For the parameters, see' magic string:**\nIn `check_functiondef_params` (docparams.py:234-243), both `node_allow_no_param` and `class_allow_no_param` are computed using an `or` chain: `class_doc.has_params() or class_doc.params_documented_elsewhere() or None`. The `params_documented_elsewhere()` method (base Docstring class, _check_docs_utils.py:330-331) uses regex `re_for_parameters_see = re.compile(r'For\\s+the\\s+(other)?\\s*parameters\\s*,\\s+see', re.X | re.S)`. The design allows either the class docstring or the `__init__` docstring to use this magic string, with the `or None` pattern ensuring that `False` values become `None` (preserving the 3-state: True/None semantics for `accept_no_param_doc`).\n\n**3. Proper exception type inference for `missing-raises-doc`:**\nThe `possible_exc_types` function (_check_docs_utils.py:101-156) handles three distinct cases based on `node.exc`: (a) When `node.exc` is `nodes.Name` (e.g., `raise ex` where `ex = ValueError('foo')`), it uses `utils.safe_infer(node.exc)` to trace the variable back to its inferred type, getting the actual exception class rather than the variable name. (b) When `node.exc is None` (bare re-raise), it walks up to the parent `ExceptHandler` and uses `astroid.unpack_infer` on the handler's type. (c) For direct calls (e.g., `raise ValueError()`), `_get_raise_target` uses pattern matching on `nodes.Call(func=nodes.Name() | nodes.Attribute())` and infers the function. For factory functions returning exceptions (`FunctionDef` case), it examines return values and checks `inherit_from_std_ex` to ensure only standard exceptions are collected. Caught exceptions are filtered out at line 153 using `utils.node_ignores_exception`.\n\n**4. Sphinx keyword synonyms:**\nThe `SphinxDocstring.re_param_raw` regex recognizes six keyword forms: `param|parameter|arg|argument|key|keyword`. The `re_raise_raw` regex recognizes `raises?|except|exception` (four forms). The `re_returns_in_docstring` uses `:returns?:` to match both `:return:` and `:returns:`. These match the full set of synonyms supported by Sphinx's Napoleon extension and core Sphinx domains.\n\n**5. Plugin name sharing (options):**\nIn `_register_options_provider` (arguments_manager.py:91-106), each option's section is determined by `provider.name.capitalize()`. The `_add_arguments_to_parser` method (line 108-122) uses a `try/except KeyError` on `self._argument_groups_dict[section]` \u2014 if the section already exists from the first checker, it reuses that group and just adds new arguments to it. This avoids the crash that occurred when `add_argument_group` was called twice with the same section name.\n\n**6. Singledispatch `unused-argument` and `function-redefined` suppression:**\n`is_registered_in_singledispatch_function` (utils.py:1515-1546) checks if a function has a decorator matching `func.register` where `func` is decorated with `functools.singledispatch` or `singledispatch.singledispatch`. In variables.py:2920-2922, this is used to skip `unused-argument` checks. In basic_error_checker.py:336-341, it's used to skip the `_check_redefinition` call that would emit `function-redefined`.\n\n**7. `allow-global-unused-variables` option:**\nThe option defaults to `True` (variables.py:1309), meaning global unused variables are NOT flagged by default. The `_check_globals` method (variables.py:3271-3290) returns immediately when `True`. When set to `False`, it checks all unconsumed names in the module scope, but exempts `__all__` assignments, `from __future__ import annotations`, names in `TYPE_CHECKING` blocks, and exception bindings used in handlers.\n\n**8. `single-line-class-stmt` option:**\nIn format.py:574-577, when enabled, a match pattern `nodes.NodeNG(parent=nodes.ClassDef(body=[_]))` checks that the class body has exactly one statement (`body=[_]`). This structural pattern match ensures only truly single-statement class bodies are allowed on one line.\n\n**9. Cyclic import local disable:**\nIn imports.py:1122-1127, when building the import graph, the code checks `self.linter.is_message_enabled('cyclic-import', line=node.lineno)` for each import. If disabled at that line or in a `TYPE_CHECKING` block, the edge is added to `_excluded_edges`. During cycle detection in `close()` (line 502-508), `_import_graph_without_ignored_edges` creates a filtered copy of the graph using `difference_update(self._excluded_edges[node])`, effectively removing edges where the disable clause was active.\n\n**10. `logging-format-interpolation` false positive fix:**\nIn logging.py:306-320 (`_check_call_func`), before emitting the warning, the code calls `is_complex_format_str(func.bound)` (line 314). This function (line 378-388) parses the format string using `string.Formatter().parse()` and returns `True` if any field has a non-empty format specification (e.g., `{:>10}`). When complex format specs are present, the message is suppressed because such specs aren't always convertible to `%`-style formatting.",
    "rubric": [
      "Explains that SphinxDocstring sets supports_yields=False because Sphinx uses :returns: for both returns and yields, while Google/Numpy set supports_yields=True since they have separate Yields sections",
      "Describes the early-return guard in check_functiondef_returns that skips the redundant-returns-doc check when supports_yields is False and the function is a generator",
      "Explains the params_documented_elsewhere() method using the 'For the parameters, see' regex pattern to suppress missing-param-doc/missing-type-doc false positives",
      "Describes the or-None pattern in node_allow_no_param/class_allow_no_param that converts False to None for the 3-state accept_no_param_doc semantics",
      "Explains how possible_exc_types handles three cases: Name nodes via safe_infer, bare re-raises via walking to ExceptHandler, and Call nodes via _get_raise_target with FunctionDef return analysis using inherit_from_std_ex",
      "Mentions that caught exceptions are filtered out using node_ignores_exception",
      "Explains how is_complex_format_str prevents logging-format-interpolation false positives by checking for non-empty format specs via string.Formatter().parse()",
      "Describes how cyclic-import local disable works via _excluded_edges and _import_graph_without_ignored_edges filtering",
      "Explains how _argument_groups_dict with try/except KeyError allows plugins sharing the same name to reuse argument groups without crashing",
      "Describes how is_registered_in_singledispatch_function suppresses both unused-argument and function-redefined for singledispatch implementations"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/checkers/variables.py",
      "pylint/checkers/imports.py",
      "pylint/config/arguments_manager.py",
      "pylint/checkers/base/basic_error_checker.py",
      "pylint/checkers/format.py",
      "pylint/checkers/logging.py",
      "pylint/message/message_id_store.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/1/1.7/summary.rst] Bug fixes\n* Fix a false positive of ``redundant-returns-doc``, occurred when the documented\n  function was using *yield* instead of *return*.\n\n* Fix a false positive of ``missing-param-doc`` and ``missing-type-doc``,\n  occurred when a class docstring uses the ``For the parameters, see``\n  magic string but the class ``__init__`` docstring does not, or vice versa.\n\n* Added proper exception type inference for ``missing-raises-doc``. Now:\n\n  .. code-block:: python\n\n      def my_func():\n          \"\"\"\"My function.\"\"\"\n          ex = ValueError('foo')\n          raise ex\n\n  will properly be flagged for missing documentation of\n  ``:raises ValueError:`` instead of ``:raises ex:``, among other scenarios.\n\n* Fix false positives of ``missing-[raises|params|type]-doc`` due to not\n  recognizing valid keyword synonyms supported by Sphinx.\n\n* More thorough validation in ``MessagesStore.register_messages()`` to detect\n  conflicts between a new message and any existing message id, symbol,\n  or ``old_names``.\n\n* We now support having plugins that shares the same name and with each one\n  providing options.\n\n  A plugin can be logically split into multiple classes, each class providing\n  certain capabilities, all of them being tied under the same name. But when\n  two or more such classes are also adding options, then **pylint** crashed,\n  since it already added the first encountered section. Now, these should\n  work as expected.\n\n  .. code-block:: python\n\n     from pylint.checkers import BaseChecker\n\n\n     class DummyPlugin1(BaseChecker):\n         name = 'dummy_plugin'\n         msgs = {'I9061': ('Dummy short desc 01', 'dummy-message-01', 'Dummy long desc')}\n         options = (\n             ('dummy_option_1', {\n                 'type': 'string',\n                 'metavar': '<string>',\n                 'help': 'Dummy option 1',\n             }),\n         )\n\n\n     class DummyPlugin2(BaseChecker):\n         name = 'dummy_plugin'\n         msgs = {'I9060': ('Dummy short desc 02', 'dummy-message-02', 'Dummy long desc')}\n         options = (\n             ('dummy_option_2', {\n                 'type': 'string',\n                 'metavar': '<string>',\n                 'help': 'Dummy option 2',\n             }),\n         )\n\n\n     def register(linter):\n         linter.register_checker(DummyPlugin1(linter))\n         linter.register_checker(DummyPlugin2(linter))\n\n* We do not yield ``unused-argument`` for singledispatch implementations and\n  do not warn about ``function-redefined`` for multiple implementations with same name.\n\n  .. code-block:: python\n\n     from functools import singledispatch\n\n     @singledispatch\n     def f(x):\n         return 2*x\n\n     @f.register(str)\n     def _(x):\n         return -1\n\n     @f.register(int)\n     @f.register(float)\n     def _(x):\n         return -x\n\n* ``unused-variable`` checker has new functionality of warning about unused\n  variables in global module namespace. Since globals in module namespace\n  may be a part of exposed API, this check is disabled by default. For\n  enabling it, set ``allow-global-unused-variables`` option to false.\n\n* Fix a false-positive ``logging-format-interpolation`` message, when format\n  specifications are used in formatted string. In general, these operations\n  are not always convertible to old-style formatting used by logging module.\n\n* Added a new switch ``single-line-class-stmt`` to allow single-line declaration\n  of empty class bodies (as seen in the example below). Pylint won't emit a\n  ``multiple-statements`` message when this option is enabled.\n\n  .. code-block:: python\n\n     class MyError(Exception): pass\n\n  * ``too-many-format-args`` and ``too-few-format-args`` are emitted correctly\n    (or not emitted at all, when exact count of elements in RHS cannot be\n    inferred) when starred expressions are used in RHS tuple. For example,\n    code block as shown below detects correctly that the used tuple has in\n    fact three elements, not two.\n\n  .. code-block:: python\n\n    meat = ['spam', 'ham']\n    print('%s%s%s' % ('eggs', *meat))\n\n* ``cyclic-import`` checker supports local disable clauses. When one\n  of cycle imports was done in scope where disable clause was active,\n  cycle is not reported as violation.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_15",
    "repo": "pylint",
    "question": "Where are the checkers for shallow-copy-environ, keyword-arg-before-vararg, stop-iteration-return, raising-format-tuple, logging-not-lazy, simplify-boolean-expression, inconsistent-return-statements, and bad-thread-instantiation implemented, and how does their detection logic flow?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The checkers described in the documentation are spread across multiple files in the pylint codebase, each following a specific data/control flow pattern:\n\n1. **`shallow-copy-environ` (W1507)** - Located in `pylint/checkers/stdlib.py`, in the `StdlibChecker` class (which inherits from `DeprecatedMixin` and `BaseChecker`). The detection flows through `visit_call()` \u2192 infers `node.func` \u2192 checks if inferred is a `FunctionDef` with qname `COPY_COPY` (\"copy.copy\") \u2192 calls `_check_shallow_copy_environ()`. This method uses `utils.get_argument_from_call(node, position=0, keyword=\"x\")` to get the argument, and if that fails, falls back to `utils.infer_kwarg_from_call(node, keyword=\"x\")` (with `INFERENCE` confidence instead of `HIGH`). It then checks if the inferred argument's qname equals `OS_ENVIRON` which is defined as `\"os._Environ\"`.\n\n2. **`keyword-arg-before-vararg` (W1113)** - Defined in `MSGS` dict in `pylint/checkers/typecheck.py` and emitted only from `TypeChecker.visit_functiondef()`. The check is: `if node.args.vararg and node.args.defaults`. There's a special exemption for positional-only parameters: if `node.args.posonlyargs` exists but `node.args.args` is empty, no warning is emitted. The method `visit_asyncfunctiondef` is aliased to `visit_functiondef` so async functions are also checked. Note: `design_analysis.py` references this message in its `only_required_for_messages` decorator but never emits it.\n\n3. **`stop-iteration-return` (R1708)** - In `pylint/checkers/refactoring/refactoring_checker.py` in `RefactoringChecker`. Two entry points: `visit_raise()` calls `_check_stop_iteration_inside_generator()` for direct raises, and `visit_call()` calls `_check_raising_stopiteration_in_generator_next_call()` for `next()` calls. The inheritance check uses `_check_exception_inherit_from_stopiteration()` which constructs `\"builtins.StopIteration\"` (using `utils.EXCEPTIONS_MODULE = \"builtins\"`) and checks the exception's MRO. For `next()` calls, it skips the warning if the iterator is in `KNOWN_INFINITE_ITERATORS` (\"itertools.count\" and \"itertools.cycle\"), or if `next()` has a sentinel (second) argument, or if the node is within a try/except handling StopIteration.\n\n4. **`raising-format-tuple` (W0715)** - In `pylint/checkers/exceptions.py`, in `ExceptionRaiseRefVisitor.visit_call()`. Uses Python structural pattern matching: `match node.args: case [nodes.Const(value=str() as msg), _, *_]` \u2014 checking if the first arg is a string constant with 2+ args total. It then checks if `\"%\"` is in the message or both `\"{\"` and `\"}\"` are present.\n\n5. **`logging-not-lazy` (W1201)** - In `pylint/checkers/logging.py` in `LoggingChecker`. The `+` operator detection works by checking if the format argument is a `BinOp` with `op == \"+\"` and at least one operand is a literal string (`_is_operand_literal_str`). The checker handles both `logging.log(level, msg)` (format_pos=1) and convenience functions like debug/info/warning/error/critical/exception/fatal/warn (format_pos=0).\n\n6. **`simplify-boolean-expression` (R1709)** - In `RefactoringChecker.visit_return()`. Triggered when `_is_and_or_ternary()` matches the pattern `condition and truth_value or false_value`, and `truth_value.bool_value()` evaluates to `False`. The suggestion is simply `false_value.as_string()`.\n\n7. **`inconsistent-return-statements` (R1710)** - In `RefactoringChecker._check_consistent_returns()`, called from `leave_functiondef()`. It collects return nodes via `node.nodes_of_class(nodes.Return, skip_klass=nodes.FunctionDef)` and checks if explicit returns (those with `value is not None`) match total returns count AND the function is \"return ended\". The `never-returning-functions` config option (default: `sys.exit`, `argparse.parse_error`) affects this check.\n\n8. **`bad-thread-instantiation` (W1506)** - In `StdlibChecker._check_bad_thread_instantiation()`. Triggered in `visit_call()` when the inferred class is `\"threading.Thread\"`. The check passes if `\"target\"` is in keyword arguments; otherwise warns if `len(node.args) < 2`.",
    "rubric": [
      "Identifies shallow-copy-environ (W1507) in StdlibChecker in stdlib.py, noting the flow from visit_call through COPY_COPY qname check to _check_shallow_copy_environ, which uses get_argument_from_call with position=0/keyword='x' and checks if inferred qname equals OS_ENVIRON ('os._Environ')",
      "Identifies keyword-arg-before-vararg (W1113) in TypeChecker in typecheck.py, noting visit_functiondef checks node.args.vararg and node.args.defaults, with the posonlyargs exemption (skips if posonlyargs exists but args is empty), and visit_asyncfunctiondef is aliased to visit_functiondef",
      "Identifies stop-iteration-return (R1708) in RefactoringChecker in refactoring_checker.py with two entry points: visit_raise calling _check_stop_iteration_inside_generator and visit_call calling _check_raising_stopiteration_in_generator_next_call, noting KNOWN_INFINITE_ITERATORS contains itertools.count and itertools.cycle",
      "Identifies raising-format-tuple (W0715) in ExceptionRaiseRefVisitor.visit_call in exceptions.py, noting the pattern match on node.args checking for a string Const as first arg with 2+ total args, then checking for '%' or both '{' and '}' in the message",
      "Identifies logging-not-lazy (W1201) in LoggingChecker in logging.py, noting detection of '+' BinOp with at least one operand being a literal string via _is_operand_literal_str",
      "Identifies simplify-boolean-expression (R1709) in RefactoringChecker.visit_return, noting it triggers when _is_and_or_ternary matches and truth_value.bool_value() is False, suggesting false_value.as_string()",
      "Identifies inconsistent-return-statements (R1710) in RefactoringChecker._check_consistent_returns called from leave_functiondef, collecting returns via nodes_of_class(nodes.Return, skip_klass=nodes.FunctionDef) and checking explicit returns count vs total returns plus _is_node_return_ended",
      "Identifies bad-thread-instantiation (W1506) in StdlibChecker._check_bad_thread_instantiation in stdlib.py, noting it checks if 'target' is in keyword args and otherwise warns if len(node.args) < 2"
    ],
    "key_files": [
      "pylint/checkers/stdlib.py",
      "pylint/checkers/typecheck.py",
      "pylint/checkers/refactoring/refactoring_checker.py",
      "pylint/checkers/exceptions.py",
      "pylint/checkers/logging.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/1/1.8/summary.rst] New checkers\n* A new check was added, ``shallow-copy-environ``.\n\n  This warning message is emitted when shallow copy of os.environ is created.\n  Shallow copy of os.environ doesn't work as people may expect. os.environ\n  is not a dict object but rather a proxy object, so any changes made\n  on copy may have unexpected effects on os.environ\n\n  Instead of copy.copy(os.environ) method os.environ.copy() should be used.\n\n  See https://bugs.python.org/issue15373 for details.\n\n  .. code-block:: python\n\n     import copy\n     import os\n     wrong_env_copy = copy.copy(os.environ)  # will emit pylint warning\n     wrong_env_copy['ENV_VAR'] = 'new_value'  # changes os.environ\n     assert os.environ['ENV_VAR'] == 'new_value'\n\n     good_env_copy = dict(os.environ)  # the right way\n     good_env_copy['ENV_VAR'] = 'different_value'  # doesn't change os.environ\n     assert os.environ['ENV_VAR'] == 'new_value'\n\n* A new check was added, ``keyword-arg-before-vararg``.\n\n  This warning message is emitted when a function is defined with a keyword\n  argument appearing before variable-length positional arguments (\\*args).\n  This may lead to args list getting modified if keyword argument's value is\n  not provided in the function call assuming it will take default value provided\n  in the definition.\n\n  .. rstcheck: ignore-next-code-block\n  .. code-block:: python\n\n     def foo(a, b=3, *args):\n         print(a, b, args)\n\n     # Case1: a=0, b=2, args=(4,5)\n     foo(0,2,4,5) # 0 2 (4,5) ==> Observed values are same as expected values\n\n     # Case2: a=0, b=<default_value>, args=(4,5)\n     foo(0,4,5) # 0 4 (5,) ==> args list got modified as well as the observed value of b\n\n     # Case3: Syntax Error if tried as follows\n     foo(0,b=2,4,5) # syntax error\n\n* A new check was added, ``simplify-boolean-expression``.\n\n  This message is emitted when ``consider-using-ternary`` check would emit\n  not equivalent code, due to truthy element being falsy in boolean context.\n\n  .. code-block:: python\n\n     value = condition and False or other_value\n\n  This flawed construct may be simplified to:\n\n  .. code-block:: python\n\n     value = other_value\n\n* A new check was added, ``bad-thread-instantiation``.\n\n  This message is emitted when the threading.Thread class does not\n  receive the target argument, but receives just one argument, which\n  is by default the group parameter.\n\n  In the following example, the instantiation will fail, which is definitely\n  not desired:\n\n  .. code-block:: python\n\n     import threading\n     threading.Thread(lambda: print(1)) # Oups, this is the group parameter\n\n* A new Python 3 checker was added to warn about accessing functions that have been\n  removed from the itertools module ``izip``, ``imap``, ``iflter``, ``izip_longest``, and ``ifilterfalse``.\n\n  .. code-block:: python\n\n      from itertools import izip\n      print(list(izip([1, 2], [3])))\n\n  Instead use ``six.moves`` to import a Python 2 and Python 3 compatible function:\n\n  .. code-block:: python\n\n      from six.moves import zip\n      print(list(zip([1, 2], [3])))\n\n* A new Python 3 checker was added to warn about accessing deprecated fields from\n  the types module like ``ListType`` or ``IntType``\n\n  .. code-block:: python\n\n      from types import ListType\n      print(isinstance([], ListType))\n\n  Instead use the declarations in the builtin namespace:\n\n  .. code-block:: python\n\n      print(isinstance([], list))\n\n* A new Python 3 checker was added to warn about declaring a ``next`` method that\n  would have implemented the ``Iterator`` protocol in Python 2 but is now a normal\n  method in Python 3.\n\n  .. code-block:: python\n\n      class Foo(object):\n          def next(self):\n              return 42\n\n  Instead implement a ``__next__`` method and use ``six.Iterator`` as a base class\n  or alias ``next`` to ``__next__``:\n\n  .. code-block:: python\n\n      class Foo(object):\n          def __next__(self):\n              return 42\n          next = __next__\n\n* Three new Python 3 checkers were added to warn about using dictionary methods\n  in non-iterating contexts.\n\n  For example, the following are returning iterators in Python 3::\n\n  .. code-block:: python\n\n     d = {}\n     d.keys()[0]\n     d.items()[0]\n     d.values() + d.keys()\n\n* A new Python 3 porting check was added, ``non-ascii-bytes-literals``\n\n  This message is emitted whenever we detect that a bytes string contain\n  non-ASCII characters, which results in a SyntaxError on Python 3.\n\n* A new warning, ``raising-format-tuple``, will catch situations where the\n  intent was likely raising an exception with a formatted message string,\n  but the actual code did omit the formatting and instead passes template\n  string and value parameters as separate arguments to the exception\n  constructor.  So it detects things like\n\n  .. code-block:: python\n\n      raise SomeError('message about %s', foo)\n      raise SomeError('message about {}', foo)\n\n  which likely were meant instead as\n\n  .. code-block:: python\n\n      raise SomeError('message about %s' % foo)\n      raise SomeError('message about {}'.format(foo))\n\n  This warning can be ignored on projects which deliberately use lazy\n  formatting of messages in all user-facing exception handlers.\n\n* Following the recommendations of PEP479_ ,a new Python 3.0 checker was added to warn about raising a ``StopIteration`` inside\n  a generator. Raising a ``StopIteration`` inside a generator may be due a direct call\n  to ``raise StopIteration``:\n\n  .. code-block:: python\n\n      def gen_stopiter():\n          yield 1\n          yield 2\n          yield 3\n          raise StopIteration\n\n  Instead use a simple ``return`` statement\n\n  .. code-block:: python\n\n      def gen_stopiter():\n          yield 1\n          yield 2\n          yield 3\n          return\n\n  Raising a ``StopIteration`` may also be due to the call to ``next`` function with a generator\n  as argument:\n\n  .. code-block:: python\n\n      def gen_next_raises_stopiter():\n          g = gen_ok()\n          while True:\n              yield next(g)\n\n  In this case, surround the call to ``next`` with a try/except block:\n\n  .. code-block:: python\n\n      def gen_next_raises_stopiter():\n          g = gen_ok()\n          while True:\n              try:\n                  yield next(g)\n              except StopIteration:\n                  return\n\n* The check about raising a StopIteration inside a generator is also valid if the exception\n  raised inherit from StopIteration.\n\n  Closes #1385\n\n .. _PEP479: https://peps.python.org/pep-0479\n\n* A new Python checker was added to warn about using a ``+`` operator inside call of logging methods\n  when one of the operands is a literal string:\n\n  .. code-block:: python\n\n     import logging\n     var = \"123\"\n     logging.log(logging.INFO, \"Var: \" + var)\n\n  Instead use formatted string and positional arguments :\n\n  .. code-block:: python\n\n     import logging\n     var = \"123\"\n     logging.log(logging.INFO, \"Var: %s\", var)\n\n* A new Python checker was added to warn about ``inconsistent-return-statements``. A function or a method\n  has inconsistent return statements if it returns both explicit and implicit values :\n\n  .. code-block:: python\n\n    def mix_implicit_explicit_returns(arg):\n        if arg < 10:\n            return True\n        elif arg < 20:\n            return\n\n  According to PEP8_, if any return statement returns an expression,\n  any return statements where no value is returned should explicitly state this as return None,\n  and an explicit return statement should be present at the end of the function (if reachable).\n  Thus, the previous function should be written:\n\n  .. code-block:: python\n\n    def mix_implicit_explicit_returns(arg):\n        if arg < 10:\n            return True\n        elif arg < 20:\n            return None\n\n  Closes #1267\n\n .. _PEP8: https://peps.python.org/pep-0008",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the specific line number reference needed minor adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_16",
    "repo": "pylint",
    "question": "How does pylint handle the asymmetry between `too-few-public-methods` and `too-many-public-methods` regarding dunder methods and enum classes?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "Here's how the changes described in the pylint 1.9 changelog were implemented:\n\n## 1. Enum classes exempt from `too-few-public-methods`\n\nIn `pylint/checkers/design_analysis.py`, the function `_is_exempt_from_public_methods(node)` checks if a class inherits from `enum.Enum` by iterating through `node.ancestors()` and calling `is_enum(ancestor)`. The `is_enum` function is imported from `pylint/checkers/utils.py`. If an ancestor is an Enum, the function returns `True`, causing `leave_classdef` to skip the too-few-public-methods check entirely.\n\n## 2. Special methods count for `too-few-public-methods` but NOT for `too-many-public-methods`\n\nThe asymmetry is implemented through two different counting mechanisms in `leave_classdef`:\n\n- **For `too-many-public-methods`**: The count `my_methods` is computed as `sum(1 for method in node.mymethods() if not method.name.startswith(\"_\"))`. Since all dunder methods start with `_`, they are excluded from this count.\n\n- **For `too-few-public-methods`**: It calls `_count_methods_in_class(node)`, which first counts all inherited public methods (not starting with `_`) via `node.methods()`, then **adds** dunder methods (matching regex `^_{2}[a-z]+_{2}$`) from `node.mymethods()`, **excluding `__init__`**. The `SPECIAL_OBJ` regex is defined as `re.compile(\"^_{2}[a-z]+_{2}$\")`. So `__len__`, `__getitem__`, etc. count towards the minimum but `__init__` explicitly does not.\n\n## 3. Abstract methods in docparams\n\nIn `pylint/extensions/docparams.py`:\n- `check_functiondef_returns`: If `node.is_abstract()`, the method returns immediately. The check for `redundant-returns-doc` is completely skipped.\n- `check_functiondef_yields`: Similarly skips the `redundant-yields-doc` check for abstract methods.\n- `_add_raise_message`: This function handles abstract methods' exception documentation (implementation is in the truncated portion of the file).\n\n## 4. Dict-iter handling (current state)\n\nThe old `dict-iter-method` (W1620) and `dict-view-method` (W1621) messages have been deleted (recorded in `pylint/message/_deleted_message_ids.py`). The current `consider-iterating-dictionary` check in `pylint/checkers/refactoring/recommendation_checker.py` uses `utils.safe_infer(node.func)` with a structural pattern match `case astroid.BoundMethod(bound=nodes.Dict())` to detect `.keys()` calls on dict-like objects. The `dict-iter-missing-items` (E1141) message is defined in `typecheck.py`'s MSGS dictionary.",
    "rubric": [
      "Explains that enum classes are exempted from too-few-public-methods via _is_exempt_from_public_methods checking node.ancestors() with is_enum()",
      "Explains that too-many-public-methods counts methods not starting with '_' from node.mymethods(), thereby excluding all dunder methods",
      "Explains that too-few-public-methods uses _count_methods_in_class which adds dunder methods (matching SPECIAL_OBJ regex ^_{2}[a-z]+_{2}$) except __init__ to the count",
      "Mentions that the SPECIAL_OBJ regex specifically matches double-underscore methods and __init__ is explicitly excluded from being counted"
    ],
    "key_files": [
      "pylint/checkers/design_analysis.py",
      "pylint/extensions/docparams.py",
      "pylint/checkers/refactoring/recommendation_checker.py",
      "pylint/checkers/typecheck.py",
      "pylint/checkers/utils.py",
      "pylint/message/_deleted_message_ids.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/1/1.9/summary.rst] Other Changes\n* ``defaultdict`` and subclasses of ``dict`` are now handled for `dict-iter-*` checks. That\n  means that the following code will now emit warnings for when ``iteritems`` and friends\n  are accessed:\n\n  .. code-block:: python\n\n      some_dict = defaultdict(list)\n      ...\n      some_dict.iterkeys()\n\n* Enum classes no longer trigger ``too-few-methods``\n\n* Special methods now count towards ``too-few-methods``,\n  and are considered part of the public API.\n  They are still not counted towards the number of methods for\n  ``too-many-methods``.\n\n* docparams allows abstract methods to document returns documentation even\n  if the default implementation does not return something.\n  They also no longer need to document raising a NotImplementedError.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few line-number references, test file references, and documentation-only rationale needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_18",
    "repo": "pylint",
    "question": "Why does pylint use a Chain of Responsibility pattern for classifying relationships in pyreverse, and why are some name types excluded from having a style option?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The stripped source code reveals several features. Here's the analysis based on actual source code:\n\n**1. `check-fixme-in-docstring`:**\nThe implementation in `pylint/checkers/misc.py` uses three distinct regex patterns to handle docstrings: `_comment_fixme_pattern` for regular comments, `_docstring_fixme_pattern` for single-line docstrings (matching `'''...'''` or `\"\"\"...\"\"\"`), and `_multiline_docstring_fixme_pattern` for lines within multiline docstrings. The option defaults to `False` (`\"type\": \"yn\"`). The `_is_multiline_docstring` method checks if `token_info.type == tokenize.STRING` AND the line starts with triple quotes AND contains a newline, distinguishing multiline from single-line docstrings.\n\n**2. `use-implicit-booleaness-not-comparison` boolean context distinction:**\nThe `_in_boolean_context` method in `pylint/checkers/refactoring/implicit_booleaness_checker.py` determines whether a comparison is used in a boolean context by traversing parent nodes using Python's structural pattern matching (`match/case`). When not in a boolean context, `_get_suggestion` wraps the suggestion with `bool()` (e.g., `bool(name)`) instead of just the bare name. The method recognizes `If`, `While`, `Assert`, `IfExp` (test position), `UnaryOp(not)`, `Comprehension` ifs, `GeneratorExp` elements passed to `all()`/`any()`, `Lambda` body passed to `filter()`, and `bool()` calls. It also follows `BoolOp` chains upward.\n\n**3. Colorized reporter fail-on:**\nThe `ColorizedTextReporter` in `pylint/reporters/text.py` uses an internal color key `\"X\"` mapped to `MessageStyle(\"red\", (\"bold\", \"inverse\"))` to highlight fail-on messages. The `handle_message` method checks `if msg.symbol in self.fail_on_symbols` and uses `_get_decoration(\"X\")` instead of the message's category-based color. The `set_fail_on_symbols()` method is defined on `ColorizedTextReporter` to receive the symbols.\n\n**4. `comparison-of-constants` unicode fix:**\nThe message format string for `comparison-of-constants` (R0133) uses `\"%s %s %s\"` in `pylint/checkers/base/comparison_checker.py`. The `_check_constants_comparison` method calls `left_operand.as_string()` and `right_operand.as_string()` which uses astroid's AST representation.\n\n**5. Pyreverse relationship handlers:**\nThe relationship classification uses a Chain of Responsibility pattern (`AbstractRelationshipHandler`) in `pylint/pyreverse/inspector.py`. The chain order is Composition \u2192 Aggregation \u2192 Association. `CompositionsHandler` detects `nodes.Call` assignments (object creation), `AggregationsHandler` detects `nodes.Name` assignments, and `AssociationsHandler` handles `nodes.AnnAssign` with `value is None` (type annotation without assignment) as the primary association case, with everything else as a fallback association. The code references `compositions_type`, `aggregations_type`, and `associations_type` fields on `ClassInfo`.\n\n**6. `--max-depth` option:**\nThe option defaults to `None` (meaning show everything). In `pylint/pyreverse/diadefslib.py`, `_should_include_by_depth` returns `True` immediately when `max_depth is None`. When set, depth is calculated relative to 'leaf nodes' (the deepest specified packages) using `name.count(\".\")`. If nested packages are specified, a warning is emitted and only leaf nodes are used as the depth base.\n\n**7. ParamSpec/TypeVarTuple naming:**\nIn `pylint/checkers/base/name_checker/checker.py`, default regex patterns are defined: `paramspec` matches `^_{0,2}(?:[A-Z]+|(?:[A-Z]+[a-z]+)+(?:P)?(?<!Type))$` and `typevartuple` matches `^_{0,2}(?:[A-Z]+|(?:[A-Z]+[a-z]+)+(?:Ts)?(?<!Type))$`. These are in the `KNOWN_NAME_TYPES` set (in `naming_style.py`) but NOT in `KNOWN_NAME_TYPES_WITH_STYLE`, meaning they only get a `rgx` option (no style dropdown), unlike regular variables which support named styles like `snake_case`.\n\n**8. Verbose filenames:**\nIn `pylint/lint/pylinter.py`, there is a `verbose` attribute and a `stats` attribute of type `LinterStats`. The verbose output mechanism uses the stats object for module tracking.",
    "rubric": [
      "Explains that the Chain of Responsibility (AbstractRelationshipHandler) orders handlers as Composition \u2192 Aggregation \u2192 Association, allowing each to check its condition and pass to the next if not matched",
      "Mentions that CompositionsHandler detects nodes.Call assignments (object creation), AggregationsHandler detects nodes.Name assignments (receiving existing objects), and AssociationsHandler handles type-annotation-only (AnnAssign with value is None) plus a fallback for everything else",
      "Explains this pattern is designed to be extensible for adding more distinct relationship types in the future",
      "Identifies that paramspec, typevartuple, typevar, and typealias are in KNOWN_NAME_TYPES but NOT in KNOWN_NAME_TYPES_WITH_STYLE",
      "Explains that types not in KNOWN_NAME_TYPES_WITH_STYLE only get a regex (rgx) option, not a named style dropdown (like snake_case/PascalCase), because they have specialized default regex patterns (e.g., paramspec requires PascalCase ending with optional P suffix, typevartuple with optional Ts suffix)"
    ],
    "key_files": [
      "pylint/checkers/misc.py",
      "pylint/checkers/refactoring/implicit_booleaness_checker.py",
      "pylint/reporters/text.py",
      "pylint/pyreverse/inspector.py",
      "pylint/checkers/base/comparison_checker.py",
      "pylint/pyreverse/diadefslib.py",
      "pylint/checkers/base/name_checker/checker.py",
      "pylint/lint/pylinter.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/4/4.0/index.rst] New Features\n- ``comparison-of-constants`` now uses the unicode from the ast instead of reformatting from\n   the node's values preventing some bad formatting due to ``utf-8`` limitation. The message now uses\n   ``\"`` instead of ``'`` to better work with what the python ast returns.\n\n  Refs #8736 (`#8736 <https://github.com/pylint-dev/pylint/issues/8736>`_)\n\n- Enhanced pyreverse to properly distinguish between UML relationship types (association, aggregation, composition) based on object ownership semantics. Type annotations without assignment are now treated as associations, parameter assignments as aggregations, and object instantiation as compositions.\n\n  Closes #9045\n  Closes #9267 (`#9045 <https://github.com/pylint-dev/pylint/issues/9045>`_)\n\n- The ``fixme`` check can now search through docstrings as well as comments, by using\n  ``check-fixme-in-docstring = true`` in the ``[tool.pylint.miscellaneous]`` section.\n\n  Closes #9255 (`#9255 <https://github.com/pylint-dev/pylint/issues/9255>`_)\n\n- The ``use-implicit-booleaness-not-x`` checks now distinguish between comparisons\n  used in boolean contexts and those that are not, enabling them to provide more accurate refactoring suggestions.\n\n  Closes #9353 (`#9353 <https://github.com/pylint-dev/pylint/issues/9353>`_)\n\n- The verbose option now outputs the filenames of the files that have been checked.\n  Previously, it only included the number of checked and skipped files.\n\n  Closes #9357 (`#9357 <https://github.com/pylint-dev/pylint/issues/9357>`_)\n\n- colorized reporter now colorizes messages/categories that have been configured as ``fail-on`` in red inverse.\n  This makes it easier to quickly find the errors that are causing pylint CI job failures.\n\n  Closes #9898 (`#9898 <https://github.com/pylint-dev/pylint/issues/9898>`_)\n\n- Enhanced support for @property decorator in pyreverse to correctly display return types of annotated properties when generating class diagrams.\n\n  Closes #10057 (`#10057 <https://github.com/pylint-dev/pylint/issues/10057>`_)\n\n- Add --max-depth option to pyreverse to control diagram complexity. A depth of 0 shows only top-level packages, 1 shows one level of subpackages, etc.\n  This helps manage visualization of large codebases by limiting the depth of displayed packages and classes.\n\n  Refs #10077 (`#10077 <https://github.com/pylint-dev/pylint/issues/10077>`_)\n\n- Handle deferred evaluation of annotations in Python 3.14.\n\n  Closes #10149 (`#10149 <https://github.com/pylint-dev/pylint/issues/10149>`_)\n\n- Enhanced pyreverse to properly detect aggregations for comprehensions (list, dict, set, generator).\n\n  Closes #10236 (`#10236 <https://github.com/pylint-dev/pylint/issues/10236>`_)\n\n- ``pyreverse``: add support for colorized output when using output format ``mmd`` (MermaidJS) and ``html``.\n\n  Closes #10242 (`#10242 <https://github.com/pylint-dev/pylint/issues/10242>`_)\n\n- pypy 3.11 is now officially supported.\n\n  Refs #10287 (`#10287 <https://github.com/pylint-dev/pylint/issues/10287>`_)\n\n- Add support for Python 3.14.\n\n  Refs #10467 (`#10467 <https://github.com/pylint-dev/pylint/issues/10467>`_)\n\n- Add naming styles for ``ParamSpec`` and ``TypeVarTuple`` that align with the ``TypeVar`` style.\n\n  Refs #10541 (`#10541 <https://github.com/pylint-dev/pylint/issues/10541>`_)",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Many claims are verifiable from code structure, but issue references, historical rationales, and some specific line numbers rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_19",
    "repo": "pylint",
    "question": "Where is the `property-with-parameters` check implemented and how does it detect violations?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `property-with-parameters` check (message code R0206) is implemented in the `ClassChecker` class in `pylint/checkers/classes/class_checker.py`. The check is triggered from the `visit_functiondef` method (line 1268), which first verifies the node is a method (`node.is_method()`), and then calls `_check_property_with_parameters` (line 1275).\n\nThe detection logic in `_check_property_with_parameters` (line 1451) checks three conditions:\n1. `len(node.args.arguments) > 1` \u2014 the method has more than just `self`\n2. `decorated_with_property(node)` \u2014 the method is decorated with `@property`\n3. `not is_property_setter(node)` \u2014 the method is NOT a property setter\n\nThe `decorated_with_property` function (in `pylint/checkers/utils.py`, line 805) uses `_is_property_decorator` to infer whether a decorator resolves to `builtins.property` or `functools.cached_property`, or a class inheriting from `builtins.property`. It also handles the case where a decorator is a function that returns a property-decorated function.\n\nThe `is_property_setter` function (line 828) delegates to `_is_property_kind(node, \"setter\")`, which checks if any decorator is a `nodes.Attribute` with `attrname` equal to \"setter\".\n\nThe good example shows the fix: split the property into a parameter-less `@property` method `bore` and a separate regular method `bore_with_depth(depth)`. The bad example shows `bore(self, depth)` decorated with `@property`, which triggers the message.\n\nThe `ClassChecker` class inherits from `BaseChecker` and has `name = \"classes\"`. The same `visit_functiondef` handler is also aliased as `visit_asyncfunctiondef` (line 1361), so async properties would also be checked. The message is emitted with `confidence=HIGH`.",
    "rubric": [
      "Identifies that the check is implemented in the ClassChecker class in pylint/checkers/classes/class_checker.py",
      "Explains that visit_functiondef triggers the check by calling _check_property_with_parameters",
      "Notes that the detection checks if the method has more than one argument (more than just self)",
      "Notes that the detection checks if the method is decorated with @property (via decorated_with_property)",
      "Notes that the detection excludes property setters (via is_property_setter)",
      "Explains how decorated_with_property works (infers decorator to check for builtins.property or functools.cached_property)",
      "Explains how is_property_setter works (checks for decorator attribute with attrname 'setter')",
      "Mentions that visit_asyncfunctiondef is aliased to the same handler, so async properties are also checked"
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/utils.py",
      "doc/data/messages/p/property-with-parameters/good.py",
      "doc/data/messages/p/property-with-parameters/bad.py",
      "tests/functional/p/property_with_parameters.py"
    ],
    "source_doc": "[docstring: doc/data/messages/p/property-with-parameters/good.py] doc.data.messages.p.property-with-parameters.good.Worm\ndoc.data.messages.p.property-with-parameters.good.Worm.bore:\n  Property accessed with '.bore'.\n\ndoc.data.messages.p.property-with-parameters.good.Worm.bore_with_depth:\n  Function called with .bore_with_depth(depth).",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only a minor rationale explanation about setters was trimmed.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_17",
    "repo": "pylint",
    "question": "What are the key pylint configuration options and how do their implementations connect to the actual checker logic at runtime?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "Here are the key code-level details discovered by reading the source:\n\n**`limit-inference-results`**: Defined in `pylint/lint/base_options.py` as an integer option with a default value of **100**. In `pylint/lint/pylinter.py`, `MANAGER = astroid.MANAGER` is assigned at module level. The `open()` method sets `MANAGER.max_inferable_values = self.config.limit_inference_results`, bridging pylint's command-line option to astroid's inference engine.\n\n**`exit-zero`**: Defined in `pylint/lint/base_options.py` with `action: \"store_true\"` and `default: False`. In `pylint/lint/run.py`, when `linter.config.exit_zero` is True, `sys.exit(0)` is called unconditionally. Otherwise, the exit logic considers `any_fail_on_issues()`, `score_value >= linter.config.fail_under`, and `self.linter.msg_status`.\n\n**`ignore-none`**: Defined in `pylint/checkers/typecheck.py` as a `yn` type option with `default: True`. The `_emit_no_member` function receives `ignored_none` and checks: `if ignored_none and isinstance(owner, nodes.Const) and owner.value is None: return False` \u2014 meaning when True (default), no-member is NOT emitted for None owners.\n\n**`inconsistent-return-statements` (R1710)**: Checked in `pylint/checkers/refactoring/refactoring_checker.py`. The `_check_consistent_returns` method counts explicit returns (those with non-None values). If there are explicit returns but the node is not \"return-ended\", it emits the message. The `_is_node_return_ended` method uses Python `match/case` on node types: it handles `Return`, `Call` (checking for never-returning functions and `is_terminating_func`), `While` (truthy test + no break), `Raise`, `If`, `Try`, and `Assert(test=Const(value=False|0))`.\n\n**`never-returning-functions`**: Configured in `refactoring_checker.py` with default value `(\"sys.exit\", \"argparse.parse_error\")`. The `_is_function_def_never_returning` method checks both the qualified name against this config list AND whether the function's return annotation is `NoReturn` or `Never` (via `nodes.Attribute(attrname=name) | nodes.Name(name=name)` pattern matching).\n\n**`is_terminating_func`** (in `pylint/checkers/utils.py`): Detects calls to `exit()`, `quit()`, `os._exit()`, `sys.exit()`, and functions annotated with `typing.NoReturn` or `typing.Never`. It checks against `TERMINATING_FUNCS_QNAMES` which is a frozenset containing `{\"_sitebuiltins.Quitter\", \"sys.exit\", \"posix._exit\", \"nt._exit\", \"unittest.case.TestCase.fail\"}`.\n\n**`too-few-public-methods` vs `too-many-public-methods`**: In `pylint/checkers/design_analysis.py`, `leave_classdef` counts methods differently for each: `too-many-public-methods` uses `node.mymethods()` filtering only non-underscore methods. `too-few-public-methods` uses `_count_methods_in_class` which counts non-underscore methods from ALL ancestors via `node.methods()`, then adds special methods (`__xxx__` matching regex `^_{2}[a-z]+_{2}$`, excluding `__init__`) from the current class's own methods (`node.mymethods()`). Enum classes are exempted via `_is_exempt_from_public_methods` which checks ancestors using `is_enum()`. The function also checks for `TYPING_NAMEDTUPLE`, `TYPING_TYPEDDICT`, `TYPING_EXTENSIONS_TYPEDDICT`, and dataclass/attrs decorators.\n\n**`singleton-comparison` for negative conditions**: In `pylint/checkers/base/comparison_checker.py`, `_check_singleton_comparison` handles both `==` and `!=` via a `checking_for_absence` boolean. For True/False singletons, it generates suggestions using `truthiness_example = {False: \"not {}\", True: \"{}\"}` and `truthiness_phrase = {True: \"truthiness\", False: \"falsiness\"}`, providing context-aware suggestions.\n\n**`undefined-loop-variable` non-empty iterable check**: In `pylint/checkers/variables.py`, after inferring the iterable, the checker considers `range()` objects always safe. For sequences (`List`, `Tuple`, `Dict`, `Set`, `FrozenSet`), it checks `elements = getattr(inferred, \"elts\", getattr(inferred, \"items\", []))` and only emits the message if `not elements`.\n\n**`ignored-argument-names`**: Defaults to regex `_.*|^ignored_|^unused_` (defined as `IGNORED_ARGUMENT_NAMES` in `pylint/checkers/variables.py`). The matching in `_is_name_ignored` applies to `nodes.AssignName(parent=nodes.Arguments())` or `nodes.Arguments()` nodes, covering all argument types including starred/keyword arguments.",
    "rubric": [
      "Explains `limit-inference-results` defaults to 100 and is bridged to astroid via `MANAGER.max_inferable_values` in the `open()` method of pylinter",
      "Explains `exit-zero` uses `store_true`/default False, and when True causes unconditional `sys.exit(0)`, otherwise considers `any_fail_on_issues()`, `fail_under` score comparison, and `msg_status`",
      "Explains `ignore-none` is a `yn` option defaulting to True in typecheck.py, suppressing no-member for None owners by checking `isinstance(owner, nodes.Const) and owner.value is None`",
      "Describes `inconsistent-return-statements` (R1710) checking for explicit returns and using `_is_node_return_ended` with match/case handling Return, Call (never-returning/terminating), While, Raise, If, Try, and Assert(False/0) nodes",
      "Describes `never-returning-functions` defaulting to `('sys.exit', 'argparse.parse_error')` and additionally checking for NoReturn/Never return annotations",
      "Describes `is_terminating_func` checking against `TERMINATING_FUNCS_QNAMES` frozenset containing `_sitebuiltins.Quitter`, `sys.exit`, `posix._exit`, `nt._exit`, `unittest.case.TestCase.fail`",
      "Explains `too-few-public-methods` uses `_count_methods_in_class` counting non-underscore methods from ALL ancestors (via `node.methods()`) plus special methods (excluding `__init__`) from `node.mymethods()`, while `too-many-public-methods` only counts non-underscore methods via `node.mymethods()`",
      "Describes `singleton-comparison` using `checking_for_absence` boolean and truthiness dictionaries for generating context-aware suggestions for both == and != cases",
      "Explains `undefined-loop-variable` treats `range()` objects as always safe, and for sequences (List, Tuple, Dict, Set, FrozenSet) only warns when the container has no elements",
      "Explains `ignored-argument-names` defaults to regex `_.*|^ignored_|^unused_` and matches argument names including starred/keyword arguments"
    ],
    "key_files": [
      "pylint/lint/base_options.py",
      "pylint/lint/run.py",
      "pylint/lint/pylinter.py",
      "pylint/checkers/refactoring/refactoring_checker.py",
      "pylint/checkers/typecheck.py",
      "pylint/checkers/design_analysis.py",
      "pylint/checkers/base/comparison_checker.py",
      "pylint/checkers/variables.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[doc_file: doc/whatsnew/2/2.0/summary.rst] Other Changes\n* A couple of performance improvements brought to ``astroid`` should make\n  ``pylint`` should be a bit faster as well.\n\n  We added a new flag, ``max_inferable_values`` on ``astroid.MANAGER`` for\n  limiting the maximum amount of values that ``astroid`` can infer when inferring\n  values. This change should improve the performance when dealing with large frameworks\n  such as ``django``.\n  You can also control this behaviour with ``pylint --limit-inference-results``\n\n  We also rewrote how ``nodes_of_class`` and ``get_children`` methods operate which\n  should result in a performance boost for a couple of checks.\n\n* Fix a false positive ``inconsistent-return-statements`` message when exception is raised inside\n  an else statement.\n\n* Don't warn for ``missing-type-doc`` and/or ``missing-return-type-doc``, if type annotations\n  exist on the function signature for a parameter and/or return type.\n\n* Fix a false positive ``inconsistent-return-statements`` message when if\n  statement is inside try/except.\n\n* Fix a false positive ``inconsistent-return-statements`` message when\n  ``while`` loop are used.\n\n* Fix emission of false positive ``no-member`` message for class with\n  \"private\" attributes whose name is mangled.\n\n* Fix ``unused-argument`` false positives with overshadowed variable in dictionary comprehension.\n\n* Fixing false positive ``inconsistent-return-statements`` when\n  never returning functions are used (i.e such as sys.exit).\n\n* Fix false positive ``inconsistent-return-statements`` message when a\n  function is defined under an if statement.\n\n* Fix false positive ``inconsistent-return-statements`` message by\n  avoiding useless exception inference if the exception is not handled.\n\n* Fix false positive ``undefined-variable`` for lambda argument in class definitions\n\n* Suppress false-positive ``not-callable`` messages from certain staticmethod descriptors\n\n* Expand ``ignored-argument-names`` include starred arguments and keyword arguments\n\n* ``singleton-comparison`` will suggest better boolean conditions for negative conditions.\n\n* ``undefined-loop-variable`` takes in consideration non-empty iterred objects before emitting.\n\n  For instance, if the loop iterable is not empty, this check will no longer be emitted.\n\n* Enum classes no longer trigger ``too-few-methods``\n\n* Special methods now count towards ``too-few-methods``,\n  and are considered part of the public API.\n  They are still not counted towards the number of methods for\n  ``too-many-methods``.\n\n* ``docparams`` extension allows abstract methods to document returns\n  documentation even if the default implementation does not return something.\n  They also no longer need to document raising a ``NotImplementedError.``\n\n* Skip wildcard import check for ``__init__.py``.\n\n* Don't warn :ref:`useless-super-delegation`' if the subclass method has different type annotations.\n\n* Don't warn that a global variable is unused if it is defined by an import\n\n  .. code-block:: python\n\n    def func():\n        global sys\n        import sys\n\n* Added basic support for postponed evaluation of function annotations.\n\n  If ``pylint`` detects the corresponding ``from __future__ import annotations`` import,\n  it will not emit ``used-before-assignment`` and ``undefined-variable`` in the cases\n  triggered by the annotations.\n\n  More details on the postponed evaluation of annotations can be read in\n  `PEP 563`_.\n\n* A new command line option was added, ``--exit-zero``, for the use of continuous integration\n  scripts which abort if a command returns a non-zero status code.  If the\n  option is specified, and Pylint runs successfully, it will exit with 0\n  regardless of the number of lint issues detected.\n\n  Configuration errors, parse errors, and calling Pylint with invalid\n  command-line options all still return a non-zero error code, even if\n  ``--exit-zero`` is specified.\n\n* Don't emit ``unused-import`` anymore for typing imports used in type comments. For instance,\n  in the following example pylint used to complain that ``Any`` and ``List`` are not used,\n  while they should be considered used by a type checker.\n\n  .. code-block:: python\n\n      from typing import Any, List\n      a = 1 # type: List[Any]\n\n* Fix false positive ``line-too-long`` for commented lines at the end of module\n\n* Fix emitting ``useless-super-delegation`` when changing the default value of keyword arguments.\n\n* Support ``typing.TYPE_CHECKING`` for *:ref:`unused-import`* errors\n\n  When modules are imported under ``typing.TYPE_CHECKING`` guard, ``pylint``\n  will no longer emit *:ref:`unused-import`*.\n\n* Fix false positive ``unused-variable`` in lambda default arguments\n\n* ``assignment-from-no-return`` considers methods as well as functions.\n\n  If you have a method that doesn't return a value, but later on you assign\n  a value to a function call to that method (so basically it will be ``None``),\n  then ``pylint`` is going to emit an ``assignment-from-no-return`` error.\n\n* A new flag was added, ``--ignore-none`` which controls the ``no-member``\n  behaviour with respect to ``None`` values.\n\n  Previously ``pylint`` was not emitting ``no-member`` if it inferred that\n  the owner of an attribute access is a ``None`` value. In some cases,\n  this might actually cause bugs, so if you want to check for ``None`` values\n  as well, pass ``--ignore-none=n`` to pylint.\n\n* Fix false-positive ``bad-continuation`` for with statements\n\n* Fix false-positive ``bad-whitespace`` message for typing annoatations\n  with ellipses in them\n\n* Fix false-positive ``undefined-variable`` for nested lambdas\n\n\n.. _PEP 563: https://peps.python.org/pep-0563/\n.. _style guide: https://docs.python-guide.org/writing/style/",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the 'pylint 2.0 changes' framing and some exact line numbers were documentation-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_20",
    "repo": "pylint",
    "question": "How does the BasicChecker class implement its key algorithms for detecting unreachable code, bad reversed sequences, and unnecessary lambdas?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `BasicChecker` class in `pylint/checkers/base/basic_checker.py` implements a comprehensive set of code quality checks through AST visitor methods. Here's how its key algorithms work:\n\n**Inheritance and Registration:**\n- `BasicChecker` extends `_BasicChecker`, which extends `BaseChecker` (from `pylint/checkers/base_checker.py`), which in turn extends `_ArgumentsProvider`. The `_BasicChecker` intermediary class exists solely to set `name = \"basic\"` and allow multiple checker classes to share the same checker name.\n- It is registered along with 6 other checkers (BasicErrorChecker, NameChecker, DocStringChecker, PassChecker, ComparisonChecker, FunctionChecker) via the `register()` function in `pylint/checkers/base/__init__.py`.\n\n**`_check_unreachable` Algorithm:**\n- Uses `node.next_sibling()` to detect code after `return`, `break`, `continue`, `raise`, or terminating function calls.\n- Has a special exception: if a `Return` node is followed by a bare `Yield` expression (an empty generator pattern), it skips the warning unless there's yet another sibling after the yield.\n- For terminating function calls (detected via `utils.is_terminating_func`), unreachable code is reported with `INFERENCE` confidence instead of `HIGH`.\n\n**`_check_not_in_finally` Algorithm:**\n- Maintains a `_trys` stack (list of `nodes.Try`) that tracks try block nesting via `visit_try`/`leave_try`.\n- Walks up the parent chain from the node looking for a parent with a `finalbody` attribute where the current node is in that finalbody.\n- Uses `breaker_classes` to short-circuit: for `return`, the breaker is `(nodes.FunctionDef,)`, and for `break`, the breakers are `(nodes.For, nodes.While)`. If a breaker class parent is encountered before finding a `finalbody`, the check aborts.\n\n**`_check_reversed` Algorithm (for `bad-reversed-sequence`):**\n- Uses a two-tier protocol check: `REVERSED_METHODS` is a tuple of two tuples \u2014 `((\"__getitem__\", \"__len__\"), (\"__reversed__\",))`. The algorithm iterates through each method group, trying `argument.getattr(meth)` for each method in the group. If all methods in any group exist (inner for-else break), the argument is valid. If no group passes (outer for-else), it emits the message.\n- Special handling for dicts: when `_py38_plus` is False (Python < 3.8), dict-based classes are checked for `__reversed__` in `argument.locals` rather than using `getattr`.\n- Also detects `reversed(iter(...))` by inferring the function name of the inner call and checking it's the builtin `iter`.\n\n**`_check_dangerous_default` Algorithm:**\n- Combines both `node.args.defaults` and `node.args.kw_defaults` into a single list to check.\n- Infers the value via `next(default.infer())` and checks if its `qname()` is in `DEFAULT_ARGUMENT_SYMBOLS`.\n- `DEFAULT_ARGUMENT_SYMBOLS` maps `builtins.set` \u2192 `\"set()\"`, `builtins.dict` \u2192 `\"{}\"`, `builtins.list` \u2192 `\"[]\"`, plus 7 `collections` types (deque, ChainMap, Counter, OrderedDict, defaultdict, UserDict, UserList).\n\n**`visit_expr` for pointless statements:**\n- PEP-257 attribute docstrings are recognized: a string after an `Assign`, `AnnAssign`, or `TypeAlias` at module/class/`__init__` scope is NOT flagged.\n- For `pointless-exception-statement` (W0133), a performance heuristic is used: inference is only run if the function/attribute name starts with an uppercase character.\n- Ellipsis constants are explicitly allowed (no pointless-statement warning).\n- If the expression is the sole child of a `Try` or `TryStar` body, it's also allowed.\n\n**`visit_lambda` for `unnecessary-lambda`:**\n- Bails out early if lambda has defaults (can't verify they match the called function's defaults).\n- Uses `_filter_vararg` to exclude starred args matching the lambda's vararg name.\n- Uses `_has_variadic_argument` to check if keyword/star args use the same variadic name.\n- Uses structural pattern matching to detect chained calls (`nodes.Attribute(expr=nodes.Call())`) and skip them.\n- Final check: if any `nodes.Name` inside `call.func` looks up to the lambda itself, the lambda is considered necessary (e.g., `lambda foo: (func1 if foo else func2)(foo)`).\n\n**`_name_holds_generator` Algorithm:**\n- Looks up the variable name in the enclosing frame via `test.frame().lookup(test.name)`.\n- Creates a lazy generator checking if each assignment's parent value is a `GeneratorExp`.\n- Uses `itertools.chain` with the first item to avoid consuming it, then checks `all()` on the chain.\n- If exactly one assignment exists and it's a Call, returns that call as `maybe_generator_call` for further testing.\n\n**`is_terminating_func` (in `utils.py`):**\n- Checks against `TERMINATING_FUNCS_QNAMES`: `{\"_sitebuiltins.Quitter\", \"sys.exit\", \"posix._exit\", \"nt._exit\", \"unittest.case.TestCase.fail\"}`.\n- Also checks for functions annotated with `typing.NoReturn` or `typing.Never` return types.",
    "rubric": [
      "Explains _check_unreachable uses node.next_sibling() and has a special exception for Return followed by a bare Yield (empty generator pattern)",
      "Mentions that terminating function calls (detected via is_terminating_func) report unreachable code with INFERENCE confidence instead of HIGH",
      "Describes that _check_reversed uses REVERSED_METHODS as a two-tier protocol check: (__getitem__, __len__) or (__reversed__,), iterating method groups with a for-else pattern",
      "Explains the special dict handling in _check_reversed when _py38_plus is False, checking __reversed__ in argument.locals",
      "Describes _check_not_in_finally walking up the parent chain looking for finalbody membership, with breaker_classes to short-circuit (FunctionDef for return, For/While for break)",
      "Explains visit_lambda bails out if defaults exist, uses _filter_vararg and _has_variadic_argument for starred/keyword args, and checks if any Name in call.func looks up to the lambda itself",
      "Mentions the structural pattern matching to detect chained calls (nodes.Attribute(expr=nodes.Call())) in visit_lambda",
      "Describes _check_dangerous_default combining args.defaults and args.kw_defaults, inferring values and checking qname() against DEFAULT_ARGUMENT_SYMBOLS mapping",
      "Explains visit_expr recognizes PEP-257 attribute docstrings (string after Assign/AnnAssign/TypeAlias at module/class/__init__ scope) and doesn't flag them",
      "Mentions the uppercase heuristic for pointless-exception-statement: inference only runs if the function/attribute name starts with an uppercase character"
    ],
    "key_files": [
      "pylint/checkers/base/basic_checker.py",
      "pylint/checkers/utils.py",
      "pylint/checkers/base_checker.py",
      "pylint/checkers/base/__init__.py"
    ],
    "source_doc": "[docstring: pylint/checkers/base/basic_checker.py] pylint.checkers.base.basic_checker.BasicChecker\npylint.checkers.base.basic_checker.BasicChecker:\n  Basic checker.\n\n    Checks for :\n    * doc strings\n    * number of arguments, local variables, branches, returns and statements in\n    functions, methods\n    * required module attributes\n    * dangerous default values as arguments\n    * redefinition of function / method / class\n    * uses of the global statement\n\npylint.checkers.base.basic_checker.BasicChecker.open:\n  Initialize visit variables and statistics.\n\npylint.checkers.base.basic_checker.BasicChecker._name_holds_generator:\n  Return whether `test` tests a name certain to hold a generator, or optionally\n        a call that should be then tested to see if *it* returns only generators.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_module:\n  Check module name, docstring and required arguments.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_classdef:\n  Check module name, docstring and redefinition\n        increment branch counter.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_expr:\n  Check for various kind of statements without effect.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_lambda:\n  Check whether the lambda is suspicious.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_functiondef:\n  Check function name, docstring, arguments, redefinition,\n        variable names, max locals.\n\npylint.checkers.base.basic_checker.BasicChecker._check_dangerous_default:\n  Check for dangerous default values as arguments.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_return:\n  Return node visitor.\n\n        1 - check if the node has a right sibling (if so, that's some\n        unreachable code)\n        2 - check if the node is inside the 'finally' clause of a 'try...finally'\n        block\n\npylint.checkers.base.basic_checker.BasicChecker.visit_continue:\n  Check is the node has a right sibling (if so, that's some unreachable\n        code).\n\npylint.checkers.base.basic_checker.BasicChecker.visit_break:\n  Break node visitor.\n\n        1 - check if the node has a right sibling (if so, that's some\n        unreachable code)\n        2 - check if the node is inside the 'finally' clause of a 'try...finally'\n        block\n\npylint.checkers.base.basic_checker.BasicChecker.visit_raise:\n  Check if the node has a right sibling (if so, that's some unreachable\n        code).\n\npylint.checkers.base.basic_checker.BasicChecker.visit_call:\n  Visit a Call node.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_assert:\n  Check whether assert is used on a tuple or string literal.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_dict:\n  Check duplicate key in dictionary.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_set:\n  Check duplicate value in set.\n\npylint.checkers.base.basic_checker.BasicChecker.visit_try:\n  Update try block flag.\n\npylint.checkers.base.basic_checker.BasicChecker.leave_try:\n  Update try block flag.\n\npylint.checkers.base.basic_checker.BasicChecker._check_unreachable:\n  Check unreachable code.\n\npylint.checkers.base.basic_checker.BasicChecker._check_not_in_finally:\n  Check that a node is not inside a 'finally' clause of a\n        'try...finally' statement.\n\n        If we find a parent which type is in breaker_classes before\n        a 'try...finally' block we skip the whole check.\n\npylint.checkers.base.basic_checker.BasicChecker._check_reversed:\n  Check that the argument to `reversed` is a sequence.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides high-level docstrings for methods. The generated answer includes extremely detailed implementation specifics (e.g., DEFAULT_ARGUMENT_SYMBOLS mapping, REVERSED_METHODS tuple structure, TERMINATING_FUNCS_QNAMES set contents, itertools.chain usage in _name_holds_generator, specific breaker_classes values, _py38_plus handling) that cannot be verified from the provided documentation alone.",
      "The documentation says visit_classdef checks 'module name, docstring and redefinition increment branch counter' but the answer doesn't discuss visit_classdef behavior.",
      "Claims about inheritance chain (_BasicChecker, BaseChecker, _ArgumentsProvider), registration with 6 other checkers, and the register() function are not verifiable from the provided documentation.",
      "The detailed algorithm descriptions appear plausible for pylint's codebase but are extrapolations beyond what the documentation explicitly states."
    ],
    "strip_verify_leakage": null,
    "strip_verify_summary": null,
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_21",
    "repo": "pylint",
    "question": "What helper functions does `_is_singledispatchmethod_registration` depend on, and how does it fit into the checker's logic?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `_is_singledispatchmethod_registration` function in `pylint/checkers/base/basic_error_checker.py` is a module-level helper function that determines whether a function node is decorated with a `.register(...)` pattern where the base function uses `@singledispatchmethod`. It depends on two other helper functions in the same file:\n\n1. **`_extract_register_target(dec)`** (lines 98-112): Extracts the target expression from a decorator that looks like `@func.register(...)` (as a `nodes.Call`) or `@func.register` (as a `nodes.Attribute`). It checks if the decorator is a `Call` with a function part that is an `Attribute` with `attrname == \"register\"`, returning `func_part.expr`. If the decorator is directly an `Attribute` with `attrname == \"register\"`, it returns `dec.expr`. Otherwise returns `None`.\n\n2. **`_inferred_has_singledispatchmethod(target)`** (lines 115-135): Uses `utils.safe_infer(target)` to resolve the target, then checks if the inferred object is a `FunctionDef` or `AsyncFunctionDef` whose decorators include one that infers to the qname `\"functools.singledispatchmethod\"`. Notably, this only checks for `functools.singledispatchmethod`, unlike the similar `is_registered_in_singledispatchmethod_function` in `pylint/checkers/utils.py` which also checks for `\"singledispatch.singledispatchmethod\"`.\n\nThe function is called at line 604 inside `BasicErrorChecker._check_redefinition` to suppress the `\"function-redefined\"` (E0102) message when a method is a singledispatchmethod registration (i.e., it early-returns without emitting the message). This is distinct from the `utils.is_registered_in_singledispatch_function` check done earlier in `visit_functiondef` at line 338 which guards against `singledispatch` (not `singledispatchmethod`).\n\n`BasicErrorChecker` inherits from `_BasicChecker` (defined in `basic_checker.py`), which itself inherits from `BaseChecker` (in `pylint/checkers/base_checker.py`) and has `name = \"basic\"`. The `BaseChecker` class inherits from `_ArgumentsProvider`.\n\nThe `safe_infer` function (from `pylint/checkers/utils.py`) is used by `_inferred_has_singledispatchmethod`. The result is checked for truthiness before proceeding with further inference.",
    "rubric": [
      "Identifies `_extract_register_target` as a helper that extracts the target from `@func.register(...)` Call or `@func.register` Attribute decorators",
      "Identifies `_inferred_has_singledispatchmethod` as a helper that uses `safe_infer` to check if the target is decorated with `functools.singledispatchmethod`",
      "Explains that `_is_singledispatchmethod_registration` is called in `_check_redefinition` to suppress the `function-redefined` (E0102) message",
      "Notes that `_inferred_has_singledispatchmethod` only checks for `functools.singledispatchmethod` (not `singledispatch.singledispatchmethod`)",
      "Mentions the inheritance chain: `BasicErrorChecker` inherits from `_BasicChecker` which inherits from `BaseChecker`"
    ],
    "key_files": [
      "pylint/checkers/base/basic_error_checker.py",
      "pylint/checkers/utils.py",
      "pylint/checkers/base/basic_checker.py"
    ],
    "source_doc": "[docstring: pylint/checkers/base/basic_error_checker.py] pylint.checkers.base.basic_error_checker._is_singledispatchmethod_registration\nReturn True if `node` is a function decorated like:\n\n        @func.register(...)\n        def _(\u2026): ...\n\n    where `func` is a singledispatchmethod (i.e. its base was decorated\n    with @singledispatchmethod).",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the detailed description of safe_infer's return semantics partially relies on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_22",
    "repo": "pylint",
    "question": "Why does the ComparisonChecker use `@only_required_for_messages`, frozensets, and early-return guards to avoid unnecessary work during AST traversal?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `ComparisonChecker` in `pylint/checkers/base/comparison_checker.py` employs several performance-oriented design decisions to avoid unnecessary work during AST traversal:\n\n**1. The `@only_required_for_messages` decorator as a method-level circuit breaker:**\nThe `visit_compare` method is decorated with `@utils.only_required_for_messages(...)` listing all 7 message IDs it can emit. This decorator (defined at `pylint/checkers/utils.py:480`) sets a `checks_msgs` attribute on the function object. When `ASTWalker.add_checker()` (`pylint/utils/ast_walker.py:42`) registers the checker, it calls `_is_method_enabled()` (line 37), which checks `any(self.linter.is_message_enabled(m) for m in method.checks_msgs)`. If **none** of the 7 messages are enabled by the user's configuration, the entire `visit_compare` callback is never registered into the `visit_events` dictionary and is never called for any `Compare` node in the AST. This avoids invoking the method on every comparison node in a codebase where all its messages are disabled.\n\n**2. Early returns using operator guards before expensive inference:**\nWithin `visit_compare`, multiple sub-checks use cheap `isinstance` and operator-set membership tests to bail out before performing costly type inference (via `utils.safe_infer`). For example:\n- `_check_callable_comparison` immediately returns (line 266-267) if the operator is not in the `COMPARISON_OPERATORS` frozenset before calling `safe_infer` on both operands.\n- `_check_unidiomatic_typecheck` checks `operator in TYPECHECK_COMPARISON_OPERATORS` and `_is_one_arg_pos_call(left)` (a simple `isinstance` + `len` check) before calling `safe_infer` in `_check_type_x_is_y`.\n- `_check_constants_comparison` returns immediately if either operand is not a `nodes.Const`, avoiding any message construction.\n- The `len(node.ops) != 1` guard at line 303 skips singleton comparison, NaN comparison, and literal comparison checks for chained comparisons like `x == y == 42`.\n\n**3. `frozenset` for O(1) operator lookups:**\n`COMPARISON_OPERATORS` and `TYPECHECK_COMPARISON_OPERATORS` are defined as `frozenset` objects (lines 15-16), providing O(1) `in` membership testing rather than linear tuple/list scanning. The `visit_compare` method body also uses set literals (`{\"==\", \"!=\"}`, `{\"is\", \"is not\"}`) for the same reason.\n\n**4. Structural pattern matching in `_is_float_nan` and `_is_numpy_nan` avoids unnecessary inference:**\nIn `_check_nan_comparison`, the nested `_is_numpy_nan` function uses structural pattern matching (`nodes.Attribute(attrname=\"NaN\", expr=nodes.Name(name=name))`) to check the AST structure without calling `safe_infer` at all. The `_is_float_nan` function uses pattern matching to verify the call structure first (`nodes.Call(args=[nodes.Const(value=str() as value)])`) and only invokes inference (`node.inferred()[0].pytype()`) if the structural match succeeds. The short-circuit `if not nan_left and not _is_nan(right_value)` at line 167 avoids running the right-side NaN check if left already matched.\n\n**5. `_check_type_x_is_y` gates inference behind cheap structural checks:**\nIn `_check_unidiomatic_typecheck`, the helper `_is_one_arg_pos_call` (line 20-22) is a purely structural check (`isinstance(call, nodes.Call) and len(call.args) == 1 and not call.keywords`) that runs before any inference. Only when this gate passes does `_check_type_x_is_y` invoke `safe_infer(left.func)`. Furthermore, when both sides are `type()` calls (line 341-351), it only calls `safe_infer(right.args[0])` after first confirming both function calls resolve to `builtins.type`.\n\n**6. Four checks run before the `len(node.ops) != 1` guard, but are designed to be cheap:**\n`_check_callable_comparison`, `_check_logical_tautology`, `_check_unidiomatic_typecheck`, and `_check_constants_comparison` all run before the guard at line 303. However, `_check_logical_tautology` and `_check_constants_comparison` avoid inference entirely (they only examine `nodes.Const` and `nodes.Name` types), and the other two have their own early-return guards.\n\n**7. The R0133 message itself advises on performance:**\nThe `comparison-of-constants` message (R0133) states: \"Using the constant directly is both easier to read and more performant\" \u2014 meaning the checker is not just about correctness but actively encourages users to avoid runtime overhead of comparing constants that always yield the same boolean result.",
    "rubric": [
      "Explains that `@only_required_for_messages` sets a `checks_msgs` attribute so that `ASTWalker._is_method_enabled` can skip registering `visit_compare` entirely when none of its 7 messages are enabled",
      "Explains that `COMPARISON_OPERATORS` and `TYPECHECK_COMPARISON_OPERATORS` are frozensets (or set literals) providing O(1) membership testing for operator checks",
      "Explains that early returns (e.g., operator not in COMPARISON_OPERATORS in `_check_callable_comparison`, isinstance checks for nodes.Const in `_check_constants_comparison`) bail out before expensive operations like `safe_infer`",
      "Mentions that `_is_numpy_nan` uses structural pattern matching on the AST node to avoid calling inference entirely",
      "Mentions the `len(node.ops) != 1` guard that skips singleton/NaN/literal checks for chained comparisons",
      "Mentions that `_is_one_arg_pos_call` is a cheap structural check (isinstance + len) gating the more expensive `safe_infer` call in `_check_type_x_is_y`"
    ],
    "key_files": [
      "pylint/checkers/base/comparison_checker.py",
      "pylint/utils/ast_walker.py",
      "pylint/checkers/utils.py",
      "pylint/checkers/base/basic_checker.py"
    ],
    "source_doc": "[docstring: pylint/checkers/base/comparison_checker.py] pylint.checkers.base.comparison_checker.ComparisonChecker\npylint.checkers.base.comparison_checker.ComparisonChecker:\n  Checks for comparisons.\n\n    - singleton comparison: 'expr == True', 'expr == False' and 'expr == None'\n    - yoda condition: 'const \"comp\" right' where comp can be '==', '!=', '<',\n      '<=', '>' or '>=', and right can be a variable, an attribute, a method or\n      a function\n\npylint.checkers.base.comparison_checker.ComparisonChecker._check_singleton_comparison:\n  Check if == or != is being used to compare a singleton value.\n\npylint.checkers.base.comparison_checker.ComparisonChecker._check_literal_comparison:\n  Check if we compare to a literal, which is usually what we do not want to do.\n\npylint.checkers.base.comparison_checker.ComparisonChecker._check_logical_tautology:\n  Check if identifier is compared against itself.\n\n        :param node: Compare node\n        :Example:\n        val = 786\n        if val == val:  # [comparison-with-itself]\n            pass\n\npylint.checkers.base.comparison_checker.ComparisonChecker._check_constants_comparison:\n  When two constants are being compared it is always a logical tautology.\n\npylint.checkers.base.comparison_checker.ComparisonChecker._check_type_x_is_y:\n  Check for expressions like type(x) == Y.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific decorators, frozenset definitions, pattern matching details, etc.) that go far beyond what the provided documentation describes. These cannot be verified from the documentation alone.",
      "The documentation only describes the high-level purpose of the ComparisonChecker and its sub-methods (_check_singleton_comparison, _check_literal_comparison, _check_logical_tautology, _check_constants_comparison, _check_type_x_is_y). The answer discusses many methods and implementation details not mentioned in the documentation (e.g., _check_callable_comparison, _check_nan_comparison, _check_unidiomatic_typecheck, _is_float_nan, _is_numpy_nan, _is_one_arg_pos_call).",
      "The answer mentions 7 message IDs but the documentation doesn't specify how many messages the checker emits.",
      "The R0133 message text 'Using the constant directly is both easier to read and more performant' is not mentioned in the documentation and cannot be verified.",
      "Claims about specific line numbers (e.g., line 266-267, line 303, lines 15-16, line 167, etc.) cannot be verified from the documentation.",
      "However, the claims that ARE verifiable from the documentation are consistent: the checker checks singleton comparisons (True/False/None), yoda conditions, comparison-with-itself (_check_logical_tautology), literal comparison, constants comparison (logical tautology), and type(x)==Y checks."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code \u2014 they reference code structure, control flow, variable definitions, and string literals in the msgs dict.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_23",
    "repo": "pylint",
    "question": "Where is the `FunctionChecker` class defined, how is it registered with the linter, and what's its full inheritance chain?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `FunctionChecker` class is located in `pylint/checkers/base/function_checker.py`. It inherits from `_BasicChecker` (defined in `pylint/checkers/base/basic_checker.py`), which in turn inherits from `BaseChecker` (in `pylint/checkers/base_checker.py`), which itself inherits from `_ArgumentsProvider`. The `_BasicChecker` class sets `name = \"basic\"`, which multiple checker classes share.\n\n**Registration:** `FunctionChecker` is registered with the linter in `pylint/checkers/base/__init__.py` via the `register()` function, which calls `linter.register_checker(FunctionChecker(linter))`. It is the last of 7 checkers registered in that function (after BasicErrorChecker, BasicChecker, NameChecker, DocStringChecker, PassChecker, and ComparisonChecker).\n\n**Message:** The checker emits a single message with ID `W0135` and symbol `contextmanager-generator-missing-cleanup`, with the template `\"The context used in function %r will not be exited.\"`\n\n**Visitor methods:** It defines two visitor methods: `visit_functiondef` (for `nodes.FunctionDef`) and `visit_asyncfunctiondef` (for `nodes.AsyncFunctionDef`), both decorated with `@utils.only_required_for_messages(\"contextmanager-generator-missing-cleanup\")` and both delegating to `_check_contextmanager_generator_missing_cleanup`.\n\n**`_check_contextmanager_generator_missing_cleanup` flow:**\n1. Collects all `nodes.With` nodes in the function using `node.nodes_of_class(nodes.With)`.\n2. Returns early if no With nodes found.\n3. Collects Yield nodes nested inside those With nodes using `chain.from_iterable`.\n4. Returns early if no Yield nodes inside With blocks.\n5. For each With node, iterates over `with_node.items` (tuples of `(call, held)`).\n6. Skips items where `held is None` (discarded context value).\n7. Calls `utils.safe_infer(call)` and accesses `.parent` on the result via `getattr(..., \"parent\", None)` to get the inferred function definition.\n8. Only continues if the inferred parent is a `nodes.FunctionDef`.\n9. Calls `_node_fails_contextmanager_cleanup(inferred_node, yield_nodes)` to decide whether to emit the message.\n10. The message is added on the `with_node` (not the function node), with `args=(node.name,)` (the generator function's name).\n\n**`_node_fails_contextmanager_cleanup` logic (static method):**\n1. Returns `False` (no failure) if *any* yield node has `value is None` or is a `nodes.Const` \u2014 i.e., yielding `None` or a constant means no warning.\n2. Re-collects yield nodes from the context manager's own body (`node.nodes_of_class(nodes.Yield)`).\n3. If there's exactly 1 yield, walks up from the yield's parent to the function node checking `n.next_sibling()`. If no sibling is found all the way up (yield is last statement), returns `False`.\n4. Filters Try nodes to those containing a Yield.\n5. If no Try nodes at all, returns `True` (fails cleanup \u2014 no try/finally or exception handling).\n6. Returns `False` if all relevant Try nodes have a `finalbody` (finally block).\n7. Contains inner function `check_handles_generator_exceptions` that checks for bare except (`handler.type is None`), or handlers catching `builtins.GeneratorExit` or `builtins.Exception` (checked via `inferred.qname()`).\n8. Returns `False` if all Try nodes handle generator exceptions; otherwise returns `True`.\n\n**Confidence level:** The `add_message` call does not explicitly set a `confidence` argument (it defaults to `None` in the `add_message` signature).",
    "rubric": [
      "States that FunctionChecker is defined in pylint/checkers/base/function_checker.py",
      "Identifies that FunctionChecker inherits from _BasicChecker, which inherits from BaseChecker (which inherits from _ArgumentsProvider)",
      "Mentions that _BasicChecker sets name = 'basic', shared across multiple checker classes",
      "Explains it is registered in the register() function in pylint/checkers/base/__init__.py via linter.register_checker(FunctionChecker(linter))",
      "Notes it is the last of 7 checkers registered (after BasicErrorChecker, BasicChecker, NameChecker, DocStringChecker, PassChecker, ComparisonChecker)"
    ],
    "key_files": [
      "pylint/checkers/base/function_checker.py",
      "pylint/checkers/base/basic_checker.py",
      "pylint/checkers/base/__init__.py",
      "tests/functional/c/contextmanager_generator_missing_cleanup.py",
      "pylint/checkers/base_checker.py"
    ],
    "source_doc": "[docstring: pylint/checkers/base/function_checker.py] pylint.checkers.base.function_checker.FunctionChecker\npylint.checkers.base.function_checker.FunctionChecker:\n  Check if a function definition handles possible side effects.\n\npylint.checkers.base.function_checker.FunctionChecker._check_contextmanager_generator_missing_cleanup:\n  Check a FunctionDef to find if it is a generator\n        that uses a contextmanager internally.\n\n        If it is, check if the contextmanager is properly cleaned up. Otherwise, add message.\n\n        :param node: FunctionDef node to check\n        :type node: nodes.FunctionDef\n\npylint.checkers.base.function_checker.FunctionChecker._node_fails_contextmanager_cleanup:\n  Check if a node fails contextmanager cleanup.\n\n        Current checks for a contextmanager:\n            - only if the context manager yields a non-constant value\n            - only if the context manager lacks a finally, or does not catch GeneratorExit\n            - only if some statement follows the yield, some manually cleanup happens\n\n        :param node: Node to check\n        :type node: nodes.FunctionDef\n        :return: True if fails, False otherwise\n        :param yield_nodes: List of Yield nodes in the function body\n        :type yield_nodes: list[nodes.Yield]\n        :rtype: bool",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many claims in the answer go far beyond what the documentation provides (inheritance hierarchy, registration details, message IDs, visitor methods, detailed flow of _check_contextmanager_generator_missing_cleanup, detailed logic of _node_fails_contextmanager_cleanup, confidence levels). These are plausible code-level details but cannot be verified from the given documentation alone.",
      "The documentation states _node_fails_contextmanager_cleanup checks 'only if the context manager yields a non-constant value' and 'only if the context manager lacks a finally, or does not catch GeneratorExit' and 'only if some statement follows the yield, some manually cleanup happens'. The answer's detailed expansion of these checks is consistent with but far more specific than the documentation.",
      "The answer's claim about yielding None or a constant meaning no warning aligns with the doc's 'only if the context manager yields a non-constant value'.",
      "The answer's claims about checking for finally blocks and GeneratorExit align with 'only if the context manager lacks a finally, or does not catch GeneratorExit'."
    ],
    "strip_verify_leakage": 0.07,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only the design rationale for _BasicChecker and the specific default confidence value partially rely on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_24",
    "repo": "pylint",
    "question": "How does pylint's NameChecker implement naming convention validation, including TypeVar variance checking and multi-naming style resolution?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `NameChecker` class in pylint implements naming convention checking through a multi-layered system that supports TypeVar variance checking, PEP 695 type parameter syntax, and multi-naming style matching. Here's how it works:\n\n**Architecture & Inheritance:**\n- `NameChecker` inherits from `_BasicChecker` (in `basic_checker.py`), which itself inherits from `BaseChecker`.\n- The checker name is \"basic\" (shared with `BasicChecker`).\n\n**Initialization & Configuration:**\n- On `open()`, the checker creates naming rules by iterating `KNOWN_NAME_TYPES` (15 types total: 11 with style + 4 without: typevar, paramspec, typevartuple, typealias).\n- For types with styles, it retrieves a `NamingStyle` subclass regex via `get_regex()`. For types without styles (typevar, paramspec, typevartuple, typealias), it uses hardcoded `DEFAULT_PATTERNS`.\n- Custom regex options (e.g., `typevar_rgx`) override both style-based and default patterns.\n\n**Name Checking Flow (`_check_name`):**\n1. First checks if name is in `good_names` (default: i, j, k, ex, Run, _) or matches `good_names_rgxs` \u2192 skip.\n2. Then checks if name is in `bad_names` (default: foo, bar, baz, toto, tutu, tata) or matches `bad_names_rgxs` \u2192 emit `disallowed-name`.\n3. Applies the type-specific regex. If a multi-naming match is detected (match with a lastgroup not in `EXEMPT_NAME_CATEGORIES`), it's stored in `_bad_names` for resolution at `leave_module`.\n4. If the regex doesn't match and not exempted, emits `invalid-name`.\n5. If `node_type == \"typevar\"`, additionally calls `_check_typevar()`.\n\n**TypeVar Variance Checking (`_check_typevar`):**\n- Uses Python 3.10+ structural pattern matching on `node.parent`.\n- For `nodes.Assign` parent: extracts keywords/args directly from `assign_type().value`.\n- For `nodes.Tuple` parent (tuple unpacking): indexes into the tuple's elements to find the corresponding Call.\n- For PEP 695 type parameters (default/catch-all case): sets `variance = TypeVarVariance.inferred` and skips variance checks entirely.\n- Detects variance from `covariant=True`/`contravariant=True` keyword arguments.\n- If both are True, emits both `typevar-double-variance` (C0131) and `typevar-name-incorrect-variance` (C0105).\n- For covariant TypeVars, name should end with `_co`; for contravariant, with `_contra`; invariant should have neither suffix.\n- The name argument is extracted either from the `name` keyword or the first positional arg (if it's a `nodes.Const`), and compared to the assigned variable name.\n\n**`_assigns_typevar` static method:**\n- Returns the type string (\"typevar\", \"paramspec\", or \"typevartuple\") if the node is a `nodes.Call` whose inferred function is a `ClassDef` with a qname in `TYPE_VAR_QNAMES`.\n- Supports both `typing.TypeVar` and `typing_extensions.TypeVar` (same for ParamSpec and TypeVarTuple).\n\n**`_assigns_typealias` static method:**\n- Checks for `typing.TypeAlias` annotation (both as ClassDef in py3.12+ and FunctionDef in earlier versions).\n- Special-cases Union types (`.Union`, `builtins.Union`, `builtins.UnionType`): treats them as type aliases only when NOT inside an `AnnAssign` (i.e., not used as annotation).\n\n**Multi-naming Match Resolution (`leave_module`):**\n- When regex groups allow multiple naming styles, warnings are deferred until `leave_module`.\n- The \"prevalent group\" (most common match group) wins, and warnings are emitted only for the minority groups.\n- When multiple groups have the minimum count, they're sorted by earliest line number, and all but the first group get warnings.\n\n**Default TypeVar Regex:**\n- Pattern: `^_{0,2}(?!T[A-Z])(?:[A-Z]+|(?:[A-Z]+[a-z]+)+(?:T)?(?<!Type))(?:_co(?:ntra)?)?$`",
    "rubric": [
      "Explains that NameChecker inherits from _BasicChecker and creates naming rules in open() by iterating KNOWN_NAME_TYPES, using NamingStyle.get_regex() for types with styles and DEFAULT_PATTERNS for typevar/paramspec/typevartuple/typealias",
      "Describes the _check_name flow: first checks good_names/good_names_rgxs to skip, then bad_names/bad_names_rgxs to emit disallowed-name, then applies type-specific regex",
      "Explains multi-naming match handling: matches with a lastgroup not in EXEMPT_NAME_CATEGORIES are stored in _bad_names dict, then resolved in leave_module by emitting warnings only for minority groups",
      "Describes _check_typevar using pattern matching on node.parent to handle Assign, Tuple (unpacking), and PEP 695 cases, extracting keywords to determine variance",
      "Explains variance detection from covariant/contravariant keyword args and the expected suffixes (_co for covariant, _contra for contravariant, neither for invariant)",
      "Mentions that _assigns_typevar checks if a Call node's inferred function is a ClassDef with qname in TYPE_VAR_QNAMES (supporting both typing and typing_extensions)",
      "Notes that custom regex options (e.g., typevar_rgx) override both style-based and default patterns"
    ],
    "key_files": [
      "pylint/checkers/base/name_checker/checker.py",
      "pylint/checkers/base/name_checker/naming_style.py",
      "pylint/checkers/base/basic_checker.py",
      "pylint/constants.py"
    ],
    "source_doc": "[docstring: pylint/checkers/base/name_checker/checker.py] pylint.checkers.base.name_checker.checker.NameChecker\npylint.checkers.base.name_checker.checker.NameChecker.visit_assignname:\n  Check module level assigned names.\n\npylint.checkers.base.name_checker.checker.NameChecker._recursive_check_names:\n  Check names in a possibly recursive list <arg>.\n\npylint.checkers.base.name_checker.checker.NameChecker._check_name:\n  Check for a name using the type's regexp.\n\npylint.checkers.base.name_checker.checker.NameChecker._assigns_typevar:\n  Check if a node is assigning a TypeVar and return TypeVar type.\n\npylint.checkers.base.name_checker.checker.NameChecker._assigns_typealias:\n  Check if a node is assigning a TypeAlias.\n\npylint.checkers.base.name_checker.checker.NameChecker._check_typevar:\n  Check for TypeVar lint violations.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is extremely minimal \u2014 just docstrings for 6 methods of NameChecker. The generated answer contains extensive implementation details (regex patterns, specific message codes like C0131/C0105, inheritance chains, default good_names/bad_names lists, PEP 695 support, structural pattern matching usage, specific TYPE_VAR_QNAMES, Union special-casing, multi-naming resolution algorithm, etc.) that go far beyond what the documentation states.",
      "None of these detailed claims directly contradict the documentation, but they cannot be verified from it either \u2014 they appear to be based on reading the actual source code rather than the provided documentation chunk.",
      "The docstring descriptions are consistent with the answer's characterizations: visit_assignname checks module-level names, _recursive_check_names checks names in recursive lists, _check_name uses type regexps, _assigns_typevar checks TypeVar assignment, _assigns_typealias checks TypeAlias assignment, _check_typevar checks TypeVar violations.",
      "The answer claims NameChecker inherits from _BasicChecker \u2014 this is not stated in the docs but is a plausible code-level detail.",
      "The specific regex pattern, default names, message codes, and algorithmic details are all extrapolations/code-level knowledge not verifiable from the provided documentation."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor rewrites needed for inheritance chain detail and regex description.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_25",
    "repo": "pylint",
    "question": "What does `_is_attribute_property` check for, and what methods in the class checker call it?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `_is_attribute_property` function is a module-level helper function defined in `pylint/checkers/classes/class_checker.py`. It takes two parameters: `name: str` (the attribute name) and `klass: nodes.ClassDef` (the class to check). The function uses `klass.getattr(name)` from astroid to retrieve attributes, catching `astroid.NotFoundError` if the name isn't found. It iterates over retrieved attributes, skipping `util.UninferableBase` instances and catching `astroid.InferenceError` during inference. For each attribute it can infer, it checks two conditions: (1) if the inferred node is a `nodes.FunctionDef` decorated with a property (using `decorated_with_property`), or (2) if the inferred node's `pytype()` equals `\"builtins.property\"` (for direct `property()` calls).\n\nThe function is called from two methods within the `ClassChecker` class (which extends `BaseChecker`):\n\n1. **`_check_in_slots`**: When checking attribute assignments related to `__slots__` definitions, it calls `_is_attribute_property`, which can prevent the `\"assigning-non-slot\"` warning from being emitted.\n\n2. **`_check_protected_attribute_access`**: When checking for protected member access, it uses `_is_attribute_property` in relation to the `\"protected-access\"` warning.",
    "rubric": [
      "Explains that _is_attribute_property is a module-level helper that takes a name (str) and klass (ClassDef) and determines if the attribute is a property",
      "Mentions it uses klass.getattr(name) to retrieve attributes, catching astroid.NotFoundError if not found",
      "Describes it skips UninferableBase instances and catches InferenceError during inference",
      "Notes it checks if the inferred node is a FunctionDef decorated with a property (via decorated_with_property)",
      "Notes it checks if the inferred node's pytype() equals 'builtins.property' (for direct property() calls)",
      "Identifies _check_in_slots as a caller, where it prevents the assigning-non-slot warning for properties",
      "Identifies _check_protected_attribute_access as a caller, where it's used in relation to the protected-access warning"
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/utils.py",
      "pylint/checkers/classes/__init__.py"
    ],
    "source_doc": "[docstring: pylint/checkers/classes/class_checker.py] pylint.checkers.classes.class_checker._is_attribute_property\nCheck if the given attribute *name* is a property in the given *klass*.\n\n    It will look for `property` calls or for functions\n    with the given name, decorated by `property` or `property`\n    subclasses.\n    Returns ``True`` if the name is a property in the given klass,\n    ``False`` otherwise.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (line numbers, specific method names, pattern matching logic, etc.) that cannot be verified from the provided documentation alone. These could be accurate code-level details or could be hallucinated.",
      "The core description of the function's purpose is consistent with the documentation: checking if an attribute name is a property in a given class by looking for property calls or property-decorated functions.",
      "The claim about checking `pytype() == 'builtins.property'` aligns with the doc's mention of looking for 'property calls'.",
      "The claim about checking `decorated_with_property` aligns with the doc's mention of 'functions with the given name, decorated by property or property subclasses'.",
      "Specific line numbers (451, 1798, 1953, 805) and detailed implementation specifics cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed unverifiable details about decorated_with_property internals and simplified documentation-derived purpose explanations.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_26",
    "repo": "pylint",
    "question": "Why does the implicit booleaness checker distinguish between boolean and non-boolean contexts when suggesting replacements for comparisons like `x != []`?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `_in_boolean_context` method provides different suggestions based on how a comparison result is used. The core behavior is:\n\nWhen a comparison like `x != []` uses the `!=` operator (a \"negation redundant\" operator), the simplification depends on context:\n- **In a boolean context** (e.g., `if x != []:`), the comparison is simplified to just `x` (returning `f\"{name}\"`).\n- **Outside a boolean context** (e.g., `_ = x != []` or `return x != []`), the suggestion is `bool(x)` (returning `f\"bool({name})\"`) to preserve the boolean return type.\n\nThis is implemented in `_get_suggestion`: when the operator is in `negation_redundant_ops`, it returns `f\"{name}\"` if in boolean context, or `f\"bool({name})\"` otherwise. For equality operators (`==`), it always suggests `not {name}` regardless of context.\n\nOther key implementation details in this checker:\n\n1. **`_is_constant_zero` explicitly checks `node.value is not False`** because in Python, `False == 0` is `True`, so without the identity check, `False` would be incorrectly treated as the constant zero.\n\n2. **`visit_unaryop` always emits with `HIGH` confidence**, while `visit_call` uses `INFERENCE` confidence when it needs to infer the argument type.\n\n3. **The XOR check (`is_left_empty_literal ^ is_right_empty_literal`)** in `_check_use_implicit_booleaness_not_comparison` ensures that exactly one side is an empty literal - if both sides are empty literals (e.g., `[] == ()`) or neither is, the check is skipped.\n\n4. **The `instance_has_bool` check** bypasses the warning for classes that define `__bool__` but are NOT base collection types (`tuple`, `list`, `dict`, `set`). The logic is: `if not is_base_comprehension_type and self.instance_has_bool(target_instance): return`.\n\n5. **`BoolOp` traversal in `_in_boolean_context`** uses a `continue` loop to walk up through nested `and`/`or` operations, so that a comparison inside a boolean operation that itself is in a boolean context (like `if x != [] and y:`) is still treated as being in a boolean context.\n\n6. **There is a likely bug**: the code checks `self.linter.is_message_enabled(\"use-implicit-booleaness-not-comparison-to-str\")` with a truncated name instead of the correct `\"use-implicit-booleaness-not-comparison-to-string\"`. Due to the `UnknownMessageError` fallback in `is_message_enabled` (which catches the exception and uses `msgids = [msg_descr]`, then `_is_one_message_enabled` defaults to `True` for unknown msgids), this effectively means the string comparison check always runs when queried this way.\n\n7. **Messages C1804 and C1805 have `\"default_enabled\": False`** in their definition options, making them opt-in extensions that users must explicitly enable, unlike C1802 and C1803 which are enabled by default.",
    "rubric": [
      "Explains that in a boolean context (if/while/assert/etc.), `x != []` is simplified to just `x` because truthiness is already being evaluated",
      "Explains that outside a boolean context (assignment, return, etc.), the suggestion is `bool(x)` to preserve the boolean return type",
      "Mentions the `_get_suggestion` method's logic: for negation-redundant operators (!=, is not), it returns `name` vs `bool(name)` based on `_in_boolean_context`",
      "Notes that for equality operators (==, is), the suggestion is always `not name` regardless of context",
      "Describes how `_in_boolean_context` traverses through `BoolOp` parents (and/or) via a continue loop so nested boolean operations are handled",
      "Identifies at least some of the constructs treated as boolean contexts (if, while, assert, ternary test, not, comprehension filters, all/any generators, filter lambdas, bool() calls)",
      "Mentions the `_is_constant_zero` identity check against False to avoid treating False as zero",
      "Notes that C1804 and C1805 have `default_enabled: False`, making them opt-in extensions unlike C1802 and C1803"
    ],
    "key_files": [
      "pylint/checkers/refactoring/implicit_booleaness_checker.py",
      "pylint/checkers/utils.py",
      "pylint/lint/message_state_handler.py",
      "tests/functional/u/use/use_implicit_booleaness_not_comparison.txt"
    ],
    "source_doc": "[docstring: pylint/checkers/refactoring/implicit_booleaness_checker.py] pylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker:\n  Checks for incorrect usage of comparisons or len() inside conditions.\n\n    Incorrect usage of len()\n    Pep8 states:\n    For sequences, (strings, lists, tuples), use the fact that empty sequences are false.\n\n        Yes: if not seq:\n             if seq:\n\n        No: if len(seq):\n            if not len(seq):\n\n    Problems detected:\n    * if len(sequence):\n    * if not len(sequence):\n    * elif len(sequence):\n    * elif not len(sequence):\n    * while len(sequence):\n    * while not len(sequence):\n    * assert len(sequence):\n    * assert not len(sequence):\n    * bool(len(sequence))\n\n    Incorrect usage of empty literal sequences; (), [], {},\n\n    For empty sequences, (dicts, lists, tuples), use the fact that empty sequences are false.\n\n        Yes: if variable:\n             if not variable\n\n        No: if variable == empty_literal:\n            if variable != empty_literal:\n\n    Problems detected:\n    * comparison such as variable == empty_literal:\n    * comparison such as variable != empty_literal:\n\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker.visit_unaryop:\n  `not len(S)` must become `not S` regardless if the parent block is a test\n        condition or something else (boolean expression) e.g. `if not len(S):`.\n\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker._check_use_implicit_booleaness_not_comparison:\n  Check for left side and right side of the node for empty literals.\n\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker._implicit_booleaness_message_args:\n  Helper to get the right message for \"use-implicit-booleaness-not-comparison\".\n\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker._in_boolean_context:\n  Returns True if the comparison is used in a boolean context; False otherwise.\n\n        A comparison is considered to be in a boolean context when it appears in constructs\n        that evaluate its truthiness directly, such as:\n        - control flow statements (`if`, `while`, `assert`)\n        - ternary expressions (`x if condition else y`)\n        - logical negation (`not`)\n        - comprehension filters (e.g. `[x for x in items if cond]`)\n        - generator expressions passed to `all()` or `any()`\n        - lambdas expressions passed to `filter()`\n        - `bool()` cast\n        - boolean operations `and`, `or` nested within any of the above\n\n        In contrast, a comparison is not in a boolean context when its result is used as a value,\n        such as when it is assigned to a variable, returned from a function, passed as an argument,\n        or used in an expression that does not depend on its truthiness.\n\npylint.checkers.refactoring.implicit_booleaness_checker.ImplicitBooleanessChecker.base_names_of_instance:\n  Return all names inherited by a class instance or those returned by a\n        function.\n\n        The inherited names include 'object'.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claims about specific line numbers (186, 231, 339-341) cannot be verified from the documentation alone",
      "Claims about `_is_constant_zero`, `_get_suggestion`, XOR check, `instance_has_bool`, specific confidence levels, and message IDs (C1804, C1805, C1802, C1803) are implementation details not present in the provided documentation",
      "The claim about a 'likely bug' with truncated message names is entirely unsupported by the documentation and could be a hallucination",
      "The claim about `default_enabled: False` for certain messages cannot be verified from the documentation",
      "The core explanation of `_in_boolean_context` is consistent with the documentation's description of what constitutes a boolean context vs non-boolean context",
      "The `visit_unaryop` description aligns with the documented behavior that 'not len(S) must become not S regardless if the parent block is a test condition or something else'"
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rationale explanations about design intent were trimmed while preserving all code-observable behavior.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_27",
    "repo": "pylint",
    "question": "Where does data and control flow through the RecommendationChecker, from registration to its visit methods being dispatched?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `RecommendationChecker` in `pylint/checkers/refactoring/recommendation_checker.py` has the following data/control flow:\n\n**Registration and Initialization:**\n- It is registered in `pylint/checkers/refactoring/__init__.py` via `linter.register_checker(RecommendationChecker(linter))` in the `register()` function.\n- It inherits from `checkers.BaseChecker` (defined in `pylint/checkers/base_checker.py`), which itself inherits from `_ArgumentsProvider`.\n- The `open()` method reads `self.linter.config.py_version` and sets `self._py36_plus` for f-string detection gating.\n\n**AST Walking Dispatch:**\n- The `ASTWalker` (in `pylint/utils/ast_walker.py`) collects `visit_*` methods from each checker via `add_checker()`. It strips the `visit_` prefix, uses the remaining string as a node type key (e.g., `\"call\"`, `\"for\"`, `\"comprehension\"`, `\"const\"`), and stores callbacks in `visit_events` dict.\n- The `@utils.only_required_for_messages` decorator sets a `checks_msgs` attribute on the method. `ASTWalker._is_method_enabled()` checks if any message in `checks_msgs` is enabled before adding the callback \u2014 if none are enabled, the visit method is skipped entirely.\n\n**Control flow for `visit_call`:**\n- Dispatches to `_check_consider_iterating_dictionary` and `_check_use_maxsplit_arg`.\n- `_check_use_maxsplit_arg` uses `utils.get_argument_from_call(node, 0, \"sep\")` to extract the separator argument. If that raises `NoSuchArgumentError`, it falls back to `utils.infer_kwarg_from_call(node, keyword=\"sep\")`. If the fallback path is used, confidence is downgraded from `HIGH` to `INFERENCE`.\n- `_check_use_maxsplit_arg` calls `utils.get_subscript_const_value(node.parent)` which raises `InferredTypeError` (a simple Exception subclass defined in utils.py) if the value cannot be determined.\n\n**Control flow for `visit_for`:**\n- Dispatches to three checks: `_check_consider_using_enumerate`, `_check_consider_using_dict_items`, `_check_use_sequence_for_iteration`.\n- `_check_consider_using_enumerate` calls the static method `_is_builtin(node, function)` which calls `utils.safe_infer(node)` then `utils.is_builtin_object(inferred)` \u2014 the latter simply checks `node.root().name == \"builtins\"`.\n- `_check_consider_using_dict_items` delegates to `utils.get_iterating_dictionary_name()` to determine if the iteration target is a dictionary.\n- `_check_use_sequence_for_iteration` checks if `node.iter` is a `nodes.Set` and uses `utils.has_starred_node_recursive()` to check for starred nodes.\n\n**Control flow for `visit_comprehension`:**\n- Decorated with `@only_required_for_messages(\"consider-using-dict-items\", \"use-sequence-for-iteration\")`.\n- Dispatches to `_check_consider_using_dict_items_comprehension` and `_check_use_sequence_for_iteration`.\n\n**Control flow for `visit_const`:**\n- Gated by `self._py36_plus` (set in `open()`). Only processes nodes where `node.pytype() == \"builtins.str\"` and the parent is not a `JoinedStr` (already an f-string).\n- Calls `_detect_replacable_format_call` which handles both `.format()` calls (checking via `utils.parse_format_method_string`) and `%` operator formatting.\n\n**Data flows through `pylint/checkers/utils.py` utilities:**\n- `safe_infer` is used throughout to safely infer node types.\n- `get_argument_from_call` is used to extract arguments from call nodes, raising `NoSuchArgumentError` on failure.",
    "rubric": [
      "Explains registration via linter.register_checker(RecommendationChecker(linter)) in the refactoring __init__.py register() function",
      "Describes how ASTWalker.add_checker() collects visit_* methods, strips the prefix to get a node-type key, and stores callbacks in visit_events dict",
      "Mentions that @only_required_for_messages sets checks_msgs attribute and ASTWalker._is_method_enabled() checks if any message is enabled before adding the callback",
      "Explains the open() method reads linter.config.py_version to set _py36_plus for gating f-string detection in visit_const",
      "Describes visit_call dispatching to _check_consider_iterating_dictionary and _check_use_maxsplit_arg, including the NoSuchArgumentError fallback that downgrades confidence from HIGH to INFERENCE",
      "Describes visit_for dispatching to _check_consider_using_enumerate, _check_consider_using_dict_items, and _check_use_sequence_for_iteration",
      "Mentions _is_builtin using utils.safe_infer and utils.is_builtin_object to check if a node refers to a builtin function",
      "Describes visit_comprehension dispatching to _check_consider_using_dict_items_comprehension and _check_use_sequence_for_iteration",
      "Mentions visit_const is gated by _py36_plus and checks node.pytype() == 'builtins.str' and parent is not JoinedStr before calling _detect_replacable_format_call"
    ],
    "key_files": [
      "pylint/checkers/refactoring/recommendation_checker.py",
      "pylint/checkers/utils.py",
      "pylint/utils/ast_walker.py",
      "pylint/checkers/refactoring/__init__.py",
      "pylint/checkers/base_checker.py"
    ],
    "source_doc": "[docstring: pylint/checkers/refactoring/recommendation_checker.py] pylint.checkers.refactoring.recommendation_checker.RecommendationChecker\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._check_use_maxsplit_arg:\n  Add message when accessing first or last elements of a str.split() or\n        str.rsplit().\n\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._check_consider_using_enumerate:\n  Emit a convention whenever range and len are used for indexing.\n\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._check_consider_using_dict_items:\n  Add message when accessing dict values by index lookup.\n\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._check_consider_using_dict_items_comprehension:\n  Add message when accessing dict values by index lookup.\n\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._check_use_sequence_for_iteration:\n  Check if code iterates over an in-place defined set.\n\n        Sets using `*` are not considered in-place.\n\npylint.checkers.refactoring.recommendation_checker.RecommendationChecker._detect_replacable_format_call:\n  Check whether a string is used in a call to format() or '%' and whether it\n        can be replaced by an f-string.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (line numbers, specific utility function signatures, decorator behavior, etc.) that go far beyond what the documentation chunk provides. These cannot be verified from the given documentation alone.",
      "The documentation says _check_consider_using_dict_items is about 'dict values by index lookup' but the answer describes it as delegating to utils.get_iterating_dictionary_name() with pattern matching details - this is plausible but unverifiable from the doc.",
      "The documentation says _check_use_sequence_for_iteration checks 'if code iterates over an in-place defined set' and 'Sets using `*` are not considered in-place.' The answer's claim about utils.has_starred_node_recursive() is consistent with this but adds unverifiable specifics.",
      "The answer's claim about visit_call dispatching to _check_consider_iterating_dictionary is not supported by the documentation chunk, which doesn't mention this method at all.",
      "The documentation describes _detect_replacable_format_call as checking 'whether a string is used in a call to format() or \"%\"' which is consistent with the answer's description of handling both .format() calls and % operator formatting.",
      "The answer's descriptions of _check_use_maxsplit_arg, _check_consider_using_enumerate, _check_consider_using_dict_items_comprehension are broadly consistent with the docstrings provided."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few line number references and internal implementation details of truncated utility functions needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_28",
    "repo": "pylint",
    "question": "How does the `ConsiderUsingWithStack` track context manager assignments and trigger warnings across different scopes?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `ConsiderUsingWithStack` is a `NamedTuple` subclass (imported from `typing`) with three fields: `module_scope`, `class_scope`, and `function_scope`, each typed as `dict[str, nodes.NodeNG]` with default empty dict values. It tracks variables assigned to context manager calls (like `open()`, `subprocess.Popen`, etc.) and is used in conjunction with the `consider-using-with` message (R1732).\n\n**How the stack works in the system:**\n\n1. **Initialization and lifecycle**: The stack is created in `RefactoringChecker.__init__()` (line 550) and cleared via `clear_all()` in the `_init()` method (line 560). The `_init()` method is also called from `leave_module`.\n\n2. **Population**: When `visit_assign` is called, it delegates to `_append_context_managers_to_stack()` (line 1625). This method checks if the right-hand side of an assignment is a call to a function whose `qname()` is in `CALLS_RETURNING_CONTEXT_MANAGERS` (e.g., `_io.open`, `pathlib.Path.open`, `subprocess.Popen`, `tempfile.NamedTemporaryFile`, etc.). If so, the variable name and the call node are stored in the appropriate scope dictionary obtained via `get_stack_for_frame()`.\n\n3. **Scope resolution via `get_stack_for_frame()`**: Uses Python structural pattern matching (`match`/`case`). For `nodes.FunctionDef` it returns `function_scope`, for `nodes.ClassDef` it returns `class_scope`, and for anything else (effectively `nodes.Module`) it returns `module_scope`.\n\n4. **Removal on `with` usage**: In `visit_with()` (line 776), when a variable is used in a `with` statement, the method iterates through all three scopes using the custom `__iter__` method (which yields in order: `function_scope`, `class_scope`, `module_scope` \u2014 inner to outer). If the variable name is found in any stack, it is deleted and the search breaks.\n\n5. **Emission on scope exit**: When leaving a scope (`leave_functiondef`, `leave_classdef`, `leave_module`), `_emit_consider_using_with_if_needed()` is called with the corresponding scope dict. This method iterates over the remaining values in the dict and emits a `consider-using-with` message for each.\n\n6. **Redefinition handling**: In `_append_context_managers_to_stack()`, if a variable already exists in the stack (i.e., reassigned without using `with`), it checks `astroid.are_exclusive(node, existing_node)`. If the two assignments are mutually exclusive (e.g., in different branches of an if/else), it simply replaces the entry. Otherwise, it immediately emits the `consider-using-with` message for the old node before recording the new one.\n\n7. **Direct call checking**: `_check_consider_using_with()` (called from `visit_call`) handles the case where a context manager is called but never assigned. It checks that the node is not already in the stack's values (meaning it was already tracked via assignment), not inside a `__enter__` method or `@contextlib.contextmanager` decorated function, not a return statement's child, not part of `with` items, and not passed to `contextlib.ExitStack.enter_context()` or `contextlib._BaseExitStack.enter_context()`. The method checks both `CALLS_THAT_COULD_BE_REPLACED_BY_WITH` (like `threading.lock.acquire`) and `CALLS_RETURNING_CONTEXT_MANAGERS`.\n\n8. **`RefactoringChecker` extends `BaseTokenChecker`** (which extends `BaseChecker`).",
    "rubric": [
      "Explains that ConsiderUsingWithStack is a NamedTuple with three dict fields: module_scope, class_scope, and function_scope, mapping variable names to AST nodes",
      "Describes that get_stack_for_frame uses pattern matching to return the appropriate scope dict based on whether the frame is a FunctionDef, ClassDef, or Module",
      "Explains that _append_context_managers_to_stack (called from visit_assign) checks if a call's inferred qname is in CALLS_RETURNING_CONTEXT_MANAGERS and adds variable-to-node mappings to the correct scope",
      "Explains that visit_with removes entries from the stack (iterating inner-to-outer: function, class, module) when a tracked variable appears in a with statement",
      "Explains that leave_functiondef, leave_classdef, and leave_module emit consider-using-with messages for any remaining entries in their respective scope dicts",
      "Describes the redefinition handling: if a variable is already in the stack and the assignments are not mutually exclusive (via are_exclusive), a warning is emitted immediately for the old node",
      "Mentions _check_consider_using_with handles unassigned context manager calls, checking they aren't already tracked, inside a context manager, a return value, part of with items, or released via ExitStack.enter_context"
    ],
    "key_files": [
      "pylint/checkers/refactoring/refactoring_checker.py",
      "pylint/checkers/base_checker.py"
    ],
    "source_doc": "[docstring: pylint/checkers/refactoring/refactoring_checker.py] pylint.checkers.refactoring.refactoring_checker.ConsiderUsingWithStack\npylint.checkers.refactoring.refactoring_checker.ConsiderUsingWithStack:\n  Stack for objects that may potentially trigger a R1732 message\n    if they are not used in a ``with`` block later on.\n\npylint.checkers.refactoring.refactoring_checker.ConsiderUsingWithStack.get_stack_for_frame:\n  Get the stack corresponding to the scope of the given frame.\n\npylint.checkers.refactoring.refactoring_checker.ConsiderUsingWithStack.clear_all:\n  Convenience method to clear all stacks.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes ConsiderUsingWithStack as a 'Stack for objects that may potentially trigger a R1732 message if they are not used in a with block later on', with two methods: get_stack_for_frame and clear_all. The answer claims it is a NamedTuple subclass with three specific fields (module_scope, class_scope, function_scope) typed as dict[str, nodes.NodeNG] - this is not in the documentation but is a plausible code-level detail.",
      "The extensive implementation details about visit_assign, _append_context_managers_to_stack, visit_with, leave_functiondef, structural pattern matching, __iter__ method, CALLS_RETURNING_CONTEXT_MANAGERS, astroid.are_exclusive, _check_consider_using_with, contextlib.ExitStack, threading.lock.acquire, BaseTokenChecker, etc. are all beyond what the documentation states. These appear to be plausible code-level extrapolations but cannot be verified from the provided documentation.",
      "The documentation does not contradict any claims, but the vast majority of the answer's content goes far beyond what is documented."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc leakage in design rationale comments and purpose descriptions; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_29",
    "repo": "pylint",
    "question": "What is the TypeChecker class and how is it structured in terms of messages, options, helper functions, and version-aware behavior?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `TypeChecker` class in `pylint/checkers/typecheck.py` is a checker that inherits from `BaseChecker` (which itself inherits from `_ArgumentsProvider`). It is one of two checkers registered by the module's `register()` function \u2014 the other being `IterableChecker`. Here is a comprehensive definition:\n\n**Core Identity:**\n- `name = \"typecheck\"` (shared with `IterableChecker`)\n- Contains 25 message definitions in its `MSGS` dict, ranging from E1101 to W1117\n- Defines 12 configuration options including `ignore-on-opaque-inference`, `mixin-class-rgx`, `generated-members`, `signature-mutators`, `missing-member-hint-distance` (default 1), `missing-member-max-choices` (default 1), and `contextmanager-decorators` (default `[\"contextlib.contextmanager\"]`)\n\n**Architecture:**\n- Uses the `CallableObjects` type alias which is a union of `bases.BoundMethod | bases.UnboundMethod | nodes.FunctionDef | nodes.Lambda | nodes.ClassDef`\n- The `_determine_callable()` helper function uses Python `match` statements to dispatch on callable type and returns a tuple of `(CallableObjects, int, str)`\n- The `_string_distance()` function implements the Levenshtein edit distance algorithm (dynamic programming row-by-row), used by `_similar_names()` (decorated with `@lru_cache(maxsize=256)`) to find similar attribute names\n- The `_similar_names()` function uses `heapq.nsmallest()` to select the closest matches\n\n**Version-aware behavior:**\n- `open()` sets `_py310_plus` and `_py314_plus` based on `linter.config.py_version`\n- `visit_module()` sets `_postponed_evaluation_enabled` to True if `_py314_plus` or if the module has `from __future__ import annotations`\n- The `_includes_version_compatible_overload()` method checks if `__or__`/`__ror__` overloads on `builtins.type` should be considered valid for the configured Python version\n- A `VERSION_COMPATIBLE_OVERLOAD` sentinel class and its singleton instance `VERSION_COMPATIBLE_OVERLOAD_SENTINEL` are defined and used in operator overload checking\n\n**Disabled checks:**\n- `_visit_binop` and `_visit_augassign` are prefixed with underscore to disable them (preventing AST walker dispatch)\n\n**Built-in handling:**\n- `BUILTINS_IMPLICIT_RETURN_NONE` maps `builtins.dict`, `builtins.list`, and `builtins.set` to sets of method names known to return None (e.g., list.sort, dict.clear, set.add)\n- `_is_ignored_function` skips `AsyncFunctionDef`, error functions, generators, and abstract functions (with `pass_is_abstract=False`)\n\n**`visit_call` specifics:**\n- Calls `safe_infer(node.func, compare_constructors=True)`\n- Uses `arguments.CallSite.from_call(node)` from astroid to analyze call arguments\n- Supports `functools.partial` via `filled_positionals` and `filled_keywords` attributes on the called object\n- Has special handling for `str.format` (via the `STR_FORMAT` constant) with `self=self` keyword arguments",
    "rubric": [
      "Identifies TypeChecker as inheriting from BaseChecker with name='typecheck'",
      "Mentions it is registered alongside IterableChecker by the module's register() function",
      "Notes the MSGS dict contains 25 message definitions (E1101 through W1117)",
      "Describes at least 3 of the 12 configuration options (e.g., ignore-on-opaque-inference, mixin-class-rgx, generated-members, contextmanager-decorators, missing-member-hint-distance, signature-mutators)",
      "Explains the CallableObjects type alias as a union of BoundMethod, UnboundMethod, FunctionDef, Lambda, and ClassDef",
      "Describes _determine_callable() using match statements to dispatch on callable type, returning a tuple of (CallableObjects, int, str)",
      "Explains _string_distance() implements Levenshtein edit distance and _similar_names() is decorated with @lru_cache(maxsize=256) using heapq.nsmallest()",
      "Describes version-aware behavior: open() sets _py310_plus and _py314_plus, visit_module() sets _postponed_evaluation_enabled",
      "Mentions _visit_binop and _visit_augassign are disabled (underscore-prefixed) to prevent AST walker dispatch",
      "Notes BUILTINS_IMPLICIT_RETURN_NONE maps builtins.dict/list/set to sets of method names returning None",
      "Describes visit_call using safe_infer with compare_constructors=True and arguments.CallSite.from_call for argument analysis",
      "Mentions functools.partial support via filled_positionals and filled_keywords attributes"
    ],
    "key_files": [
      "pylint/checkers/typecheck.py",
      "pylint/checkers/base_checker.py",
      "pylint/interfaces.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: pylint/checkers/typecheck.py] pylint.checkers.typecheck.TypeChecker\npylint.checkers.typecheck.TypeChecker:\n  Try to find bugs in the code using type inference.\n\npylint.checkers.typecheck.TypeChecker.visit_attribute:\n  Check that the accessed attribute exists.\n\n        to avoid too much false positives for now, we'll consider the code as\n        correct if a single of the inferred nodes has the accessed attribute.\n\n        function/method, super call and metaclasses are ignored\n\npylint.checkers.typecheck.TypeChecker.visit_assign:\n  Process assignments in the AST.\n\npylint.checkers.typecheck.TypeChecker._check_assignment_from_function_call:\n  When assigning to a function call, check that the function returns a valid\n        value.\n\npylint.checkers.typecheck.TypeChecker._check_dundername_is_string:\n  Check a string is assigned to self.__name__.\n\npylint.checkers.typecheck.TypeChecker._check_uninferable_call:\n  Check that the given uninferable Call node does not\n        call an actual function.\n\npylint.checkers.typecheck.TypeChecker._check_argument_order:\n  Match the supplied argument names against the function parameters.\n\n        Warn if some argument names are not in the same order as they are in\n        the function signature.\n\npylint.checkers.typecheck.TypeChecker.visit_call:\n  Check that called functions/methods are inferred to callable objects,\n        and that passed arguments match the parameters in the inferred function.\n\npylint.checkers.typecheck.TypeChecker._keyword_argument_is_in_all_decorator_returns:\n  Check if the keyword argument exists in all signatures of the\n        return values of all decorators of the function.\n\npylint.checkers.typecheck.TypeChecker._check_not_callable:\n  Checks to see if the not-callable message should be emitted.\n\n        Only functions, generators and objects defining __call__ are \"callable\"\n        We ignore instances of descriptors since astroid cannot properly handle them yet\n\npylint.checkers.typecheck.TypeChecker.visit_unaryop:\n  Detect TypeErrors for unary operands.\n\npylint.checkers.typecheck.TypeChecker._detect_unsupported_alternative_union_syntax:\n  Detect if unsupported alternative Union syntax (PEP 604) was used.\n\npylint.checkers.typecheck.TypeChecker._includes_version_compatible_overload:\n  Check if a set of overloads of an operator includes one that\n        can be relied upon for our configured Python version.\n\n        If we are running under a Python 3.10+ runtime but configured for\n        pre-3.10 compatibility then Astroid will have inferred the\n        existence of __or__ / __ror__ on builtins.type, but these aren't\n        available in the configured version of Python.\n\npylint.checkers.typecheck.TypeChecker._check_unsupported_alternative_union_syntax:\n  Check if left or right node is of type `type`.\n\n        If either is, and doesn't support an or operator via a metaclass,\n        infer that this is a mistaken attempt to use alternative union\n        syntax when not supported.\n\npylint.checkers.typecheck.TypeChecker._visit_binop:\n  Detect TypeErrors for binary arithmetic operands.\n\npylint.checkers.typecheck.TypeChecker._visit_augassign:\n  Detect TypeErrors for augmented binary arithmetic operands.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few semantic explanations about purpose/rationale come from documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_30",
    "repo": "pylint",
    "question": "Why does the CodeStyleChecker suppress walrus operator suggestions in certain cases, disable some messages by default, and use a cutoff of 0 for difflib matching?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `CodeStyleChecker` in `pylint/extensions/code_style.py` exists as a separate extension (located in `pylint/extensions/` rather than `pylint/checkers/`), and some of its messages are disabled by default. The design decisions behind its key suppression/guard logic reveal several purposeful choices:\n\n1. **Why `_check_ignore_assignment_expr_suggestion` suppresses the walrus operator suggestion**: The method checks if the `if` node's test is a `nodes.Compare` AND whether the next logical block (either an `elif` or a separate subsequent `if` statement) also tests the same variable name. When this pattern is detected (e.g., `if x == 1: ... elif x == 2: ...`), the suggestion is suppressed. Notably, it only suppresses when `node.test` is specifically a `nodes.Compare` \u2014 plain `Name` tests or `UnaryOp(not)` tests are never suppressed by this logic.\n\n2. **Why `_check_prev_sibling_to_if_stmt` rejects multi-line assignments**: The method checks `prev_sibling.tolineno - prev_sibling.fromlineno != 0` to reject any assignment spanning multiple lines. This prevents generating walrus operator suggestions for multi-line assignments.\n\n3. **Why `max-line-length-suggestions` defaults to 0**: The `open()` method uses `self.linter.config.max_line_length_suggestions or self.linter.config.max_line_length` \u2014 meaning a default of 0 (falsy) causes a fallback to the general `max_line_length` setting. The help string in the options indicates this prevents the checker from emitting suggestions that would be split by code formatters (e.g., black).\n\n4. **Why R6104 and R6105 are disabled by default**: Both `consider-using-augmented-assign` and `prefer-typing-namedtuple` have `\"default_enabled\": False` in their message options dictionary. Their message description strings note they are \"Disabled by default!\".\n\n5. **Why `visit_call` uses `difflib.get_close_matches` with `cutoff=0`**: For the `consider-math-not-float` check (R6106), when a `float()` call contains a string argument with alphabetic characters (excluding 'e' for scientific notation), and the string isn't directly 'nan' or 'inf', the method uses `difflib.get_close_matches(value, [\"inf\", \"nan\"], n=1, cutoff=0)` to find the closest match. The `cutoff=0` ensures a match is ALWAYS returned, which catches typos like `float('nna')` or `float('inff')`.\n\n6. **Why the checker inherits from `BaseChecker` (which inherits from `_ArgumentsProvider`)**: This gives it access to the linter's configuration system. `BaseChecker.__init__` calls `_ArgumentsProvider.__init__(self, linter)`, which integrates the checker's options with the linter.\n\n7. **The `_check_consider_using_assignment_expr` uses Python 3.10 structural pattern matching (`match/case`)** on the AST node.test to check for three specific patterns: bare `Name`, `UnaryOp(op=\"not\", operand=Name)`, and `Compare(left=Name, ops=[_])` (single comparison operator only). Multi-comparison expressions like `a < b < c` are excluded because they have multiple ops.",
    "rubric": [
      "Explains that _check_ignore_assignment_expr_suggestion suppresses the walrus operator suggestion when node.test is a Compare and the next if/elif tests the same variable name (match/case pattern better fit)",
      "Explains that _check_prev_sibling_to_if_stmt rejects multi-line assignments by checking tolineno - fromlineno != 0",
      "Explains that max-line-length-suggestions defaults to 0 (falsy) so the open() method falls back to max_line_length via the `or` expression",
      "Explains that R6104 (consider-using-augmented-assign) and R6105 (prefer-typing-namedtuple) have default_enabled set to False in their message option dictionaries",
      "Explains that difflib.get_close_matches uses cutoff=0 to always return a match, catching typos like float('nna') or float('inff')",
      "Notes that _check_consider_using_assignment_expr uses Python 3.10 structural pattern matching (match/case) and only handles Name, UnaryOp(not, Name), and Compare(left=Name, ops=[_]) \u2014 excluding multi-comparison expressions"
    ],
    "key_files": [
      "pylint/extensions/code_style.py",
      "pylint/checkers/utils.py",
      "pylint/checkers/base_checker.py",
      "pylint/interfaces.py",
      "tests/functional/ext/code_style/cs_consider_using_assignment_expr.py"
    ],
    "source_doc": "[docstring: pylint/extensions/code_style.py] pylint.extensions.code_style.CodeStyleChecker\npylint.extensions.code_style.CodeStyleChecker:\n  Checkers that can improve code consistency.\n\n    As such they don't necessarily provide a performance benefit and\n    are often times opinionated.\n\n    Before adding another checker here, consider this:\n    1. Does the checker provide a clear benefit,\n       i.e. detect a common issue or improve performance\n       => it should probably be part of the core checker classes\n    2. Is it something that would improve code consistency,\n       maybe because it's slightly better with regard to performance\n       and therefore preferred => this is the right place\n    3. Everything else should go into another extension\n\npylint.extensions.code_style.CodeStyleChecker._check_dict_consider_namedtuple_dataclass:\n  Check if dictionary values can be replaced by Namedtuple or Dataclass.\n\npylint.extensions.code_style.CodeStyleChecker._check_consider_using_assignment_expr:\n  Check if an assignment expression (walrus operator) can be used.\n\n        For example if an assignment is directly followed by an if statement:\n        >>> x = 2\n        >>> if x:\n        >>>     ...\n\n        Can be replaced by:\n        >>> if (x := 2):\n        >>>     ...\n\n        Note: Assignment expressions were added in Python 3.8\n\npylint.extensions.code_style.CodeStyleChecker._check_prev_sibling_to_if_stmt:\n  Check if previous sibling is an assignment with the same name.\n\n        Ignore statements which span multiple lines.\n\npylint.extensions.code_style.CodeStyleChecker._check_ignore_assignment_expr_suggestion:\n  Return True if suggestion for assignment expr should be ignored.\n\n        E.g., in cases where a match statement would be a better fit\n        (multiple conditions).",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim 1 about `_check_ignore_assignment_expr_suggestion` checking for `nodes.Compare` and next logical block testing same variable name: The documentation only says 'Return True if suggestion for assignment expr should be ignored. E.g., in cases where a match statement would be a better fit (multiple conditions).' The detailed implementation specifics (nodes.Compare, elif checks, etc.) are plausible extrapolations from source code but not supported by the provided documentation.",
      "Claim 2 about `_check_prev_sibling_to_if_stmt` rejecting multi-line assignments via `tolineno - fromlineno != 0`: The documentation says 'Ignore statements which span multiple lines' which supports the concept, but the specific implementation detail is an extrapolation beyond the doc.",
      "Claim 3 about `max-line-length-suggestions` defaulting to 0 and `open()` method fallback logic: Not mentioned in the documentation at all \u2014 this is entirely extrapolated from source code.",
      "Claim 4 about R6104 and R6105 being disabled by default with `default_enabled: False`: Not mentioned in the documentation \u2014 extrapolated from source code.",
      "Claim 5 about `visit_call` using `difflib.get_close_matches` with `cutoff=0` for R6106 `consider-math-not-float`: Not mentioned in the documentation at all \u2014 entirely fabricated or extrapolated from source code not provided.",
      "Claim 6 about inheritance from `BaseChecker` and `_ArgumentsProvider`: Not mentioned in the documentation \u2014 plausible but unsupported.",
      "Claim 7 about `_check_consider_using_assignment_expr` using Python 3.10 structural pattern matching (`match/case`): Not mentioned in the documentation \u2014 this is an extrapolation from source code. The documentation only describes the walrus operator suggestion logic conceptually."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale statements and one test configuration claim needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_31",
    "repo": "pylint",
    "question": "Where is the docstring parameter checking logic implemented, and where do the different docstring format parsers live in the class hierarchy?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `DocstringParameterChecker` in `pylint/extensions/docparams.py` is located in the `pylint.extensions` package and checks docstring parameter, return, yield, and exception documentation. Here is where the key components are located and how they connect:\n\n**Primary file**: `pylint/extensions/docparams.py` \u2014 Contains the `DocstringParameterChecker` class, which inherits from `BaseChecker` (defined in `pylint/checkers/base_checker.py`). `BaseChecker` itself inherits from `_ArgumentsProvider`.\n\n**Registration**: The checker is registered via a `register()` function, which calls `linter.register_checker(DocstringParameterChecker(linter))`.\n\n**Docstring parsing utilities**: Located in `pylint/extensions/_check_docs_utils.py`. The `docstringify()` function determines docstring type. It tries all four docstring types (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) and picks the one with the highest `matching_sections()` count. If none match, it falls back to the `default_type` parameter, using the `DOCSTRING_TYPES` dictionary which maps type name strings to their respective classes.\n\n**Class hierarchy for docstring types**:\n- `Docstring` (base) \u2014 returns 0 for `matching_sections()`, has `supports_yields = False`\n- `SphinxDocstring(Docstring)` \u2014 `supports_yields = False`, counts matches against 5 regex patterns in `matching_sections()`\n- `EpytextDocstring(SphinxDocstring)` \u2014 replaces initial `:` with `@` in Sphinx regex patterns\n- `GoogleDocstring(Docstring)` \u2014 uses section-based parsing\n- `NumpyDocstring(GoogleDocstring)` \u2014 overrides section header detection and indent behavior\n\n**Visit methods**: The checker handles 6 AST node types: `functiondef`, `asyncfunctiondef` (aliased to `visit_functiondef`), `raise`, `return`, `yield`, and `yieldfrom` (aliased to `visit_yield`).\n\n**Key flow in `visit_functiondef`**: First checks `is_overload_stub` (skips overload-decorated stubs), then checks `no-docstring-rgx` config, then `_is_shorter_than_min_length`. Then delegates to `check_functiondef_params`, `check_functiondef_returns`, and `check_functiondef_yields`.\n\n**Exception handling in `visit_raise`**: For property setters without docstrings, it looks up the property getter via `get_setters_property()` and checks the getter's docstring instead. There is a special case where `found_exc == \"error\"` matches `expected.name == \"PatternError\"` (handling the `re.error`/`re.PatternError` aliasing).\n\n**Message codes**: The checker defines 17 messages from W9005 to W9021, with the checker name `\"parameter_documentation\"`. Some messages have `old_names` for backward compatibility (e.g., W9015/missing-param-doc has old name W9003/old-missing-param-doc).\n\n**Configuration options**: 5 options \u2014 `accept-no-param-doc`, `accept-no-raise-doc`, `accept-no-return-doc`, `accept-no-yields-doc` (all default `True`), and `default-docstring-type` (default `\"default\"`, choices from `DOCSTRING_TYPES` keys).",
    "rubric": [
      "Identifies DocstringParameterChecker class in pylint/extensions/docparams.py as the main checker, inheriting from BaseChecker",
      "Identifies pylint/extensions/_check_docs_utils.py as the location of docstring parsing utilities and the docstringify() function",
      "Describes the docstring class hierarchy: Docstring (base), SphinxDocstring(Docstring), EpytextDocstring(SphinxDocstring), GoogleDocstring(Docstring), NumpyDocstring(GoogleDocstring)",
      "Explains how docstringify() selects the best docstring type by comparing matching_sections() counts across all four types, falling back to default_type",
      "Mentions the DOCSTRING_TYPES dictionary mapping type name strings to their classes",
      "Mentions the register() function that registers DocstringParameterChecker with the linter",
      "Identifies the visit methods handled: functiondef (aliased for asyncfunctiondef), raise, return, yield (aliased for yieldfrom)",
      "Notes that visit_functiondef delegates to check_functiondef_params, check_functiondef_returns, and check_functiondef_yields after checking overload stubs, no-docstring-rgx, and min-length",
      "Mentions property setter handling in visit_raise via get_setters_property() to check getter docstrings"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/checkers/base_checker.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: pylint/extensions/docparams.py] pylint.extensions.docparams.DocstringParameterChecker\npylint.extensions.docparams.DocstringParameterChecker:\n  Checker for Sphinx, Google, or Numpy style docstrings.\n\n    * Check that all function, method and constructor parameters are mentioned\n      in the params and types part of the docstring.  Constructor parameters\n      can be documented in either the class docstring or ``__init__`` docstring,\n      but not both.\n    * Check that there are no naming inconsistencies between the signature and\n      the documentation, i.e. also report documented parameters that are missing\n      in the signature. This is important to find cases where parameters are\n      renamed only in the code, not in the documentation.\n    * Check that all explicitly raised exceptions in a function are documented\n      in the function docstring. Caught exceptions are ignored.\n\n    Activate this checker by adding the line::\n\n        load-plugins=pylint.extensions.docparams\n\n    to the ``MAIN`` section of your ``.pylintrc``.\n\npylint.extensions.docparams.DocstringParameterChecker.visit_functiondef:\n  Called for function and method definitions (def).\n\n        :param node: Node for a function or method definition in the AST\n        :type node: :class:`astroid.scoped_nodes.Function`\n\npylint.extensions.docparams.DocstringParameterChecker._compare_missing_args:\n  Compare the found argument names with the expected ones and\n        generate a message if there are arguments missing.\n\n        :param found_argument_names: argument names found in the docstring\n\n        :param message_id: pylint message id\n\n        :param not_needed_names: names that may be omitted\n\n        :param expected_argument_names: Expected argument names\n\n        :param warning_node: The node to be analyzed\n\npylint.extensions.docparams.DocstringParameterChecker._compare_different_args:\n  Compare the found argument names with the expected ones and\n        generate a message if there are extra arguments found.\n\n        :param found_argument_names: argument names found in the docstring\n\n        :param message_id: pylint message id\n\n        :param not_needed_names: names that may be omitted\n\n        :param expected_argument_names: Expected argument names\n\n        :param warning_node: The node to be analyzed\n\npylint.extensions.docparams.DocstringParameterChecker._compare_ignored_args:\n  Compare the found argument names with the ignored ones and\n        generate a message if there are ignored arguments found.\n\n        :param found_argument_names: argument names found in the docstring\n        :param message_id: pylint message id\n        :param ignored_argument_names: Expected argument names\n        :param warning_node: The node to be analyzed\n\npylint.extensions.docparams.DocstringParameterChecker.check_arguments_in_docstring:\n  Check that all parameters are consistent with the parameters mentioned\n        in the parameter documentation (e.g. the Sphinx tags 'param' and 'type').\n\n        * Undocumented parameters except 'self' are noticed.\n        * Undocumented parameter types except for 'self' and the ``*<args>``\n          and ``**<kwargs>`` parameters are noticed.\n        * Parameters mentioned in the parameter documentation that don't or no\n          longer exist in the function parameter list are noticed.\n        * If the text \"For the parameters, see\" or \"For the other parameters,\n          see\" (ignoring additional white-space) is mentioned in the docstring,\n          missing parameter documentation is tolerated.\n        * If there's no Sphinx style, Google style or NumPy style parameter\n          documentation at all, i.e. ``:param`` is never mentioned etc., the\n          checker assumes that the parameters are documented in another format\n          and the absence is tolerated.\n\n        :param doc: Docstring for the function, method or class.\n        :type doc: :class:`Docstring`\n\n        :param arguments_node: Arguments node for the function, method or\n            class constructor.\n        :type arguments_node: :class:`astroid.nodes.Arguments`\n\n        :param warning_node: The node to assign the warnings to\n        :type warning_node: :class:`astroid.nodes.NodeNG`\n\n        :param accept_no_param_doc: Whether to allow no parameters to be\n            documented. If None then this value is read from the configuration.\n        :type accept_no_param_doc: bool or None\n\npylint.extensions.docparams.DocstringParameterChecker._add_raise_message:\n  Adds a message on :param:`node` for the missing exception type.\n\n        :param missing_exceptions: A list of missing exception types.\n        :param node: The node to show the message on.\n\npylint.extensions.docparams.DocstringParameterChecker._is_shorter_than_min_length:\n  Returns true on functions smaller than 'docstring-min-length'.\n\n        :param node: Node for a function or method definition in the AST\n        :type node: :class:`astroid.nodes.FunctionDef`\n\n        :rtype: bool",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, exact number of messages, inheritance chains, etc.) that cannot be verified from the provided documentation. These could be accurate code-level details or could be hallucinated.",
      "The documentation states the checker handles 'Sphinx, Google, or Numpy style docstrings' but does not mention Epytext style. The answer claims EpytextDocstring is also supported, which cannot be verified from the doc.",
      "The documentation does not mention anything about `supports_yields`, `matching_sections()`, `_parse_section()`, or the regex pattern details described in the answer.",
      "The documentation does not mention `NotImplementedError` special handling for abstract functions, Python 3.13 re.error aliases, or property setter/getter docstring lookup.",
      "The claim about 17 messages from W9005 to W9021 cannot be verified from the documentation.",
      "The claim about 5 configuration options and their defaults cannot be fully verified from the documentation, though `accept-no-param-doc` is indirectly referenced in the `check_arguments_in_docstring` docstring."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for line numbers, truncated implementation details, and documentation-derived purpose descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_32",
    "repo": "pylint",
    "question": "How does BaseWriter use the Visitor pattern and stream patching to format report layouts and compute child content?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `BaseWriter` class implements a Visitor pattern-based report formatting system. Here's how the algorithms work in detail:\n\n**`format()` method (lines 31-49):** It first resolves encoding by checking if the stream has an `encoding` attribute (via `getattr(stream, 'encoding', 'UTF-8')`), falling back to `'UTF-8'` twice (once from `getattr` default, once from the `or` clause if encoding is falsy). It then stores the stream as `self.out`, calls `self.begin_format()`, invokes `layout.accept(self)` (the Visitor pattern entry point), and finally calls `self.end_format()`.\n\n**Visitor pattern dispatch (nodes.py, `VNode.accept`):** When `layout.accept(self)` is called, the `VNode.accept` method uses `getattr(visitor, f'visit_{self.visitor_name}')` to dynamically look up a method on the writer. The `visitor_name` is set in `VNode.__init__` as `self.__class__.__name__.lower()`. So for example, a `Table` node dispatches to `visit_table`, a `Section` to `visit_section`, etc.\n\n**`compute_content()` method (lines 90-107):** This uses a stream-patching trick. It saves the current `self.out`, then for each child of the layout, creates a fresh `StringIO()` instance, temporarily sets `self.out` to this new stream, calls `child.accept(self)` (which writes to the temporary stream), and yields `stream.getvalue()`. A `try/finally` block ensures `self.out` is always restored to the original stream, even if an exception occurs.\n\n**`get_table_content()` method (lines 73-88):** It initializes `result` as `[[]]` (a list with one empty list) and a countdown counter `cols = table.cols`. For each cell string from `compute_content(table)`, when `cols` reaches 0 it starts a new row and resets `cols`. After the loop, it fills missing cells in the last row with empty strings using the expression `result[-1] += [''] * (cols - len(result[-1]))` \u2014 note this formula uses the difference between remaining column count and current row length.\n\n**`begin_format()` initializes `self.section = 0`** \u2014 this counter is used by `TextWriter` to track nesting depth (incremented/decremented in `visit_section` and `visit_evaluationsection`), and it selects the title underline character from `TITLE_UNDERLINES = ['', '=', '-', '`', '.', '~', '^']`.\n\n**`BaseWriter` has no `__init__` method** \u2014 attributes like `self.out`, `self.encoding`, and `self.section` are only created when `format()` or `begin_format()` is called.\n\n**`TextWriter` (the only subclass) calls `get_table_content` in `visit_table` and `compute_content` in `visit_title`.** In `visit_title`, it joins all computed content strings together and writes the title followed by an underline character repeated to match the title length, using `TITLE_UNDERLINES[self.section]`. If the section depth exceeds the underlines list length (>6), it catches `IndexError` and prints a FIXME message.",
    "rubric": [
      "Explains that format() resolves encoding via getattr(stream, 'encoding', 'UTF-8') with a fallback 'or UTF-8' for falsy values, stores the stream in self.out, then calls begin_format(), layout.accept(self), and end_format()",
      "Explains that VNode.accept dynamically dispatches to visit_{classname.lower()} on the writer via getattr, where visitor_name is set to self.__class__.__name__.lower() in VNode.__init__",
      "Explains that compute_content() saves self.out, then for each child creates a temporary StringIO, sets self.out to it, calls child.accept(self), yields stream.getvalue(), and restores self.out in a finally block",
      "Explains that get_table_content() builds a list of rows using a cols countdown that resets when reaching 0, starting a new row, and fills missing cells in the last row with empty strings",
      "Explains that begin_format() initializes self.section = 0, which TextWriter increments/decrements in visit_section and visit_evaluationsection to track nesting depth for selecting title underline characters from TITLE_UNDERLINES"
    ],
    "key_files": [
      "pylint/reporters/ureports/base_writer.py",
      "pylint/reporters/ureports/nodes.py",
      "pylint/reporters/ureports/text_writer.py"
    ],
    "source_doc": "[docstring: pylint/reporters/ureports/base_writer.py] pylint.reporters.ureports.base_writer.BaseWriter\npylint.reporters.ureports.base_writer.BaseWriter:\n  Base class for ureport writers.\n\npylint.reporters.ureports.base_writer.BaseWriter.format:\n  Format and write the given layout into the stream object.\n\n        unicode policy: unicode strings may be found in the layout;\n        try to call 'stream.write' with it, but give it back encoded using\n        the given encoding if it fails\n\npylint.reporters.ureports.base_writer.BaseWriter.format_children:\n  Recurse on the layout children and call their accept method\n        (see the Visitor pattern).\n\npylint.reporters.ureports.base_writer.BaseWriter.writeln:\n  Write a line in the output buffer.\n\npylint.reporters.ureports.base_writer.BaseWriter.write:\n  Write a string in the output buffer.\n\npylint.reporters.ureports.base_writer.BaseWriter.begin_format:\n  Begin to format a layout.\n\npylint.reporters.ureports.base_writer.BaseWriter.end_format:\n  Finished formatting a layout.\n\npylint.reporters.ureports.base_writer.BaseWriter.get_table_content:\n  Trick to get table content without actually writing it.\n\n        return an aligned list of lists containing table cells values as string\n\npylint.reporters.ureports.base_writer.BaseWriter.compute_content:\n  Trick to compute the formatting of children layout before actually\n        writing it.\n\n        return an iterator on strings (one for each child element)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, specific code patterns like `getattr(stream, 'encoding', 'UTF-8')`, `StringIO()` usage, `TITLE_UNDERLINES` list, `TextWriter` specifics) that are not present in the documentation and appear to be derived from reading the actual source code rather than the provided documentation chunk.",
      "The documentation only provides high-level docstrings for each method. The answer's claims about specific implementation details (e.g., encoding fallback logic, stream-patching trick, section counter, TextWriter behavior) cannot be verified from the provided documentation alone.",
      "However, none of the claims directly contradict the documentation - the high-level descriptions in the answer align with the docstrings (e.g., format() writing to stream, format_children using Visitor pattern, compute_content returning iterator of strings, get_table_content returning aligned list of lists)."
    ],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code alone; no documentation leakage detected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_33",
    "repo": "pylint",
    "question": "What are the key dependencies and class relationships that make the TextWriter ureport formatting system work end-to-end?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `TextWriter` class in `pylint/reporters/ureports/text_writer.py` inherits from `BaseWriter` (defined in `pylint/reporters/ureports/base_writer.py`) and is the concrete text-formatting implementation for pylint's ureport system. Here are the key dependency relationships:\n\n1. **Parent class `BaseWriter`** provides core infrastructure: `format()` method (which sets encoding, output stream, calls `begin_format()`, invokes `layout.accept(self)`, and calls `end_format()`), `format_children()` (iterates over layout's children calling `child.accept(self)`), `writeln()`, `write()`, `begin_format()` (initializes `self.section = 0`), `get_table_content()` (uses `compute_content()` to gather table cells into a list of lists), and `compute_content()` (temporarily redirects `self.out` to `StringIO` instances to capture each child's formatted output).\n\n2. **Visitor dispatch mechanism via `VNode.accept()`** in `nodes.py`: Each node's `accept()` method calls `getattr(visitor, f\"visit_{self.visitor_name}\")` where `visitor_name` is set to `self.__class__.__name__.lower()` in `VNode.__init__`. This is how `TextWriter.visit_section`, `visit_table`, etc. get invoked dynamically.\n\n3. **Node hierarchy in `nodes.py`**: `VNode` \u2192 `BaseLayout` \u2192 concrete nodes (`Section`, `EvaluationSection`, `Title`, `Paragraph`, `Table`). `Text` inherits from `VNode` directly (not `BaseLayout`), and `VerbatimText` inherits from `Text`. `EvaluationSection` is a subclass of `Section` that constructs itself with a dashed-line paragraph and a message paragraph.\n\n4. **Usage in `TextReporter._display()`** (in `pylint/reporters/text.py`): The `TextReporter` class instantiates `TextWriter()` and calls `.format(layout, self.out)` to render report layouts. `TextWriter` is imported directly from `pylint.reporters.ureports.text_writer` in `text.py`.\n\n5. **`TextWriter.__init__`** adds `self.list_level = 0` beyond what `BaseWriter` provides.\n\n6. **`TITLE_UNDERLINES`** in `text_writer.py` is `[\"\", \"=\", \"-\", \"`\", \".\", \"~\", \"^\"]` (7 levels), while a separate `TITLE_UNDERLINES` in `pylint/reporters/text.py` is `[\"\", \"=\", \"-\", \".\"]` (only 4 levels).\n\n7. **Table formatting**: `default_table` adds 1 to each column width, uses `\"%%-%ss\"` format strings for left-justified columns, and differentiates header rows (using `\"=\"` separators) from data rows (using `\"-\"` separators) based on `layout.rheaders`.\n\n8. **`Table` node** has attributes `cols`, `title`, `rheaders`, and `cheaders`, all set in its constructor.",
    "rubric": [
      "Explains that TextWriter inherits from BaseWriter and adds self.list_level in __init__",
      "Describes BaseWriter's core infrastructure: format() sets encoding/output, calls begin_format(), layout.accept(self), and end_format(); format_children() iterates children calling accept(); compute_content() temporarily redirects self.out to StringIO to capture output; get_table_content() uses compute_content() to build list of lists",
      "Explains the visitor dispatch mechanism: VNode.accept() calls getattr(visitor, f'visit_{self.visitor_name}') where visitor_name is set to self.__class__.__name__.lower() in VNode.__init__",
      "Describes the node hierarchy: VNode \u2192 BaseLayout \u2192 Section, Title, Paragraph, Table; Text inherits from VNode directly (not BaseLayout); VerbatimText inherits from Text; EvaluationSection is a subclass of Section",
      "Mentions that TextReporter._display() instantiates TextWriter and calls .format(layout, self.out) to render layouts",
      "Describes how TextWriter's table formatting works: default_table adds 1 to column widths, uses left-justified format strings, and differentiates header rows (= separators) from data rows (- separators) based on layout.rheaders",
      "Notes the TITLE_UNDERLINES constant has 7 levels in text_writer.py versus a separate 4-level version in the text reporter"
    ],
    "key_files": [
      "pylint/reporters/ureports/text_writer.py",
      "pylint/reporters/ureports/base_writer.py",
      "pylint/reporters/ureports/nodes.py",
      "pylint/reporters/text.py"
    ],
    "source_doc": "[docstring: pylint/reporters/ureports/text_writer.py] pylint.reporters.ureports.text_writer.TextWriter\npylint.reporters.ureports.text_writer.TextWriter:\n  Format layouts as text\n    (ReStructured inspiration but not totally handled yet).\n\npylint.reporters.ureports.text_writer.TextWriter.visit_section:\n  Display a section as text.\n\npylint.reporters.ureports.text_writer.TextWriter.visit_evaluationsection:\n  Display an evaluation section as a text.\n\npylint.reporters.ureports.text_writer.TextWriter.visit_paragraph:\n  Enter a paragraph.\n\npylint.reporters.ureports.text_writer.TextWriter.visit_table:\n  Display a table as text.\n\npylint.reporters.ureports.text_writer.TextWriter.default_table:\n  Format a table.\n\npylint.reporters.ureports.text_writer.TextWriter.visit_verbatimtext:\n  Display a verbatim layout as text (so difficult ;).\n\npylint.reporters.ureports.text_writer.TextWriter.visit_text:\n  Add some text.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minimal doc leakage: only 'micro-report' terminology and an unverifiable __init__.py export claim needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_34",
    "repo": "pylint",
    "question": "Why does pylint use `_check_regexp_csv` with character-by-character parsing instead of just reusing `_splitstrip` for regex pattern lists?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "_splitstrip uses a simple list comprehension with `str.split()` and `str.strip()` rather than a more complex parsing approach.\n\nThe `word.strip()` call appears twice in the list comprehension \u2014 once for the `if` filter check and once for the output list \u2014 meaning each word gets stripped twice.\n\nThe function is called in:\n1. `_config_initialization` (config_initialization.py) to split plugin names from `load-plugins` config\n2. `_add_plugins` (config/utils.py) to parse `--load-plugins` CLI argument\n3. `_check_csv` (utils.py) which acts as an early-return guard, short-circuiting the split entirely if the value is already a list/tuple \u2014 this avoids redundant re-parsing\n\nFor regex patterns that may contain commas within braces (like `\\d{1,2}`), the codebase uses `_check_regexp_csv` instead, which performs character-by-character parsing with `collections.deque`. The `_check_regexp_csv` function uses `deque[deque[str] | None]` with `None` as a sentinel value to track group boundaries, and it is a generator (uses `yield from`) rather than building a list. The function tracks open braces to avoid splitting on commas inside `{}` groups, and uses `deque` for appending characters.",
    "rubric": [
      "Explains that _splitstrip naively splits on commas using str.split(), which would incorrectly split regex quantifiers like \\d{1,2} that contain commas inside braces",
      "Mentions that _check_regexp_csv tracks open/close braces to avoid splitting on commas inside {} groups",
      "Describes that _check_regexp_csv uses collections.deque for character-by-character parsing with None as a sentinel value for group boundaries",
      "Notes that _check_regexp_csv is a generator (uses yield from) rather than building and returning a list like _splitstrip",
      "Mentions that _splitstrip is used for simpler cases like plugin name lists where commas in values aren't expected"
    ],
    "key_files": [
      "pylint/utils/utils.py",
      "pylint/config/config_initialization.py",
      "pylint/config/utils.py",
      "pylint/config/argument.py"
    ],
    "source_doc": "[docstring: pylint/utils/utils.py] pylint.utils.utils._splitstrip\nReturn a list of stripped string by splitting the string given as\n    argument on `sep` (',' by default), empty strings are discarded.\n\n    >>> _splitstrip('a, b, c   ,  4,,')\n    ['a', 'b', 'c', '4']\n    >>> _splitstrip('a')\n    ['a']\n    >>> _splitstrip('a,\\nb,\\nc,')\n    ['a', 'b', 'c']\n\n    :type string: str or unicode\n    :param string: a csv line\n\n    :type sep: str or unicode\n    :param sep: field separator, default to the comma (',')\n\n    :rtype: str or unicode\n    :return: the unquoted string (or the input string if it wasn't quoted)",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes the function's interface (parameters, return value, and examples). The generated answer makes extensive claims about implementation details (double word.strip() call, line numbers, calling contexts, _check_regexp_csv internals, deque usage, etc.) that cannot be verified from the provided documentation alone.",
      "The claims about specific line numbers (line 206), specific calling locations (config_initialization.py:64, config/utils.py:170, utils.py:224-227), and implementation details of _check_regexp_csv (deque[deque[str] | None], yield from, etc.) are not supported by the documentation and could be hallucinations.",
      "The documentation's :return: description says 'the unquoted string (or the input string if it wasn't quoted)' which seems inconsistent with a function that returns a list of stripped strings - but the answer doesn't address this discrepancy.",
      "The answer's core claim about the function's purpose (splitting and stripping CSV values) is consistent with the documentation's docstring and examples."
    ],
    "strip_verify_leakage": 0.35,
    "strip_verify_summary": "Several claims about design rationale, performance trade-off reasoning, and philosophy are not recoverable from stripped code, while structural/behavioral claims about the functions are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_35",
    "repo": "pylint",
    "question": "Where is the `SeeTelnet` class defined, and how does the checker avoid false positives for attributes initialized in parent classes?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `SeeTelnet` class is located in `tests/functional/a/access/access_attr_before_def_false_positive.py`. It is defined at line 11 and inherits from `telnetlib.Telnet`. This file is a functional test related to pylint's `access-member-before-definition` (E0203) checker, which is implemented in `pylint/checkers/classes/class_checker.py` within the `ClassChecker` class (line 765). The file name includes 'false_positive', indicating it is a false-positive regression test.\n\nThe `SeeTelnet` class's `__init__` calls `telnetlib.Telnet.__init__(self, host, port)` (line 26). The `readUntilArray` method calls `self.process_rawq()` (line 33), which is a method inherited from `telnetlib.Telnet`. The checker avoids false positives by using `node.instance_attr_ancestors(attr)` (line 2034 of class_checker.py) to detect attributes defined in parent class `__init__` methods.\n\nThe test file contains four other classes: `Base` (line 39), `Derived(Base)` (line 46), `QoSALConnection` (line 68), and `DefinedOutsideInit` (line 81). The `.rc` file sets `max_pyver=3.13`. The `_check_attribute_defined_outside_init` method (line 1194) calls `_check_accessed_members` (line 2019) which performs three checks in sequence: class attribute via `local_attr`, parent instance attribute via `instance_attr_ancestors`, and then same-class instance attribute via `instance_attr`.",
    "rubric": [
      "Identifies SeeTelnet is located in the functional test file for access_attr_before_def_false_positive",
      "States SeeTelnet inherits from telnetlib.Telnet",
      "Mentions the file is a false-positive regression test for pylint's access-member-before-definition (E0203) checker",
      "Notes that the checker is implemented in ClassChecker class in class_checker.py",
      "Explains that _check_accessed_members uses node.instance_attr_ancestors(attr) to detect attributes defined in parent class __init__ methods",
      "Mentions SeeTelnet.__init__ calls telnetlib.Telnet.__init__ and readUntilArray calls self.process_rawq() which is inherited",
      "Identifies that the test file also contains classes Base, Derived, QoSALConnection, and DefinedOutsideInit"
    ],
    "key_files": [
      "tests/functional/a/access/access_attr_before_def_false_positive.py",
      "pylint/checkers/classes/class_checker.py",
      "tests/functional/a/access/access_attr_before_def_false_positive.rc"
    ],
    "source_doc": "[docstring: tests/functional/a/access/access_attr_before_def_false_positive.py] tests.functional.a.access.access_attr_before_def_false_positive.SeeTelnet\ntests.functional.a.access.access_attr_before_def_false_positive.SeeTelnet:\n  Extension of telnetlib.\n\ntests.functional.a.access.access_attr_before_def_false_positive.SeeTelnet.__init__:\n  Constructor.\n        When called without arguments, create an unconnected instance.\n        With a hostname argument, it connects the instance; a port\n        number is optional.\n        Parameter:\n        - host: IP address of the host\n        - port: Port number\n\ntests.functional.a.access.access_attr_before_def_false_positive.SeeTelnet.readUntilArray:\n  Read until a given string is encountered or until timeout.\n        ...",
    "verification_verdict": "warn",
    "verification_issues": [
      "Many specific claims about line numbers (line 11, 26, 33, 39, 46, 68, 81, 765, 1194, 2019, 2034), file paths (pylint/checkers/classes/class_checker.py), method names (_check_attribute_defined_outside_init, _check_accessed_members), class names (Base, Derived, QoSALConnection, DefinedOutsideInit), and implementation details cannot be verified from the provided documentation alone",
      "The claim about .rc file setting max_pyver=3.13 and telnetlib removal cannot be verified from the documentation",
      "The claim about no corresponding .txt file cannot be verified from the documentation",
      "However, the core claims about SeeTelnet being an extension of telnetlib, the __init__ parameters (host, port), and readUntilArray reading until a string or timeout are all supported by the documentation"
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Minor doc leakage in claims about telnetlib internals (cookedq) and the reason for max_pyver setting; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_36",
    "repo": "pylint",
    "question": "How does pylint's protected-access checker decide when to suppress warnings for accessing single-underscore members inside dunder methods?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Issue1802` test class in `tests/functional/a/access/access_to_protected_members.py` tests how pylint's `protected-access` checker (W0212) handles access to protected members inside special (dunder) methods. The implementation works as follows:\n\n1. **The checker method**: `_check_protected_attribute_access` in `pylint/checkers/classes/class_checker.py` handles the core logic.\n\n2. **Key distinction - single vs double underscore**: The variable `licit_protected_member = not attrname.startswith(\"__\")` determines whether the attribute uses a single underscore prefix (like `_foo`) versus a double underscore prefix (like `__private`). Only single-underscore members are eligible for the special method exemption.\n\n3. **Special method validation**: The static method `_is_called_inside_special_method` checks if the node's frame name is in `PYMETHODS`, which is defined in `pylint/checkers/utils.py` as `PYMETHODS = set(SPECIAL_METHODS_PARAMS)` \u2014 a set built from the `_SPECIAL_METHODS_PARAMS` dictionary. This set includes recognized Python dunder methods like `__eq__`, `__le__`, `__lt__`, etc., but NOT arbitrary dunder-named methods like `__fake_special__`.\n\n4. **Configuration option**: The `check-protected-access-in-special-methods` option (default: `False`) controls whether single-underscore protected member access inside legitimate special methods should be warned about. When False (default), accessing `other._foo` inside `__eq__` or `__le__` is allowed.\n\n5. **Why `__fake_special__` fails**: Since `__fake_special__` is not in `PYMETHODS` (which only contains recognized Python special methods defined in `_SPECIAL_METHODS_PARAMS`), `_is_called_inside_special_method` returns `False`, so the exemption doesn't apply.\n\n6. **Why `__eq__` still triggers for `__private`**: In `__eq__`, `other._foo` is allowed (single underscore + inside special method), but `other.__private` still triggers the warning because `licit_protected_member` is `False` for double-underscore prefixed names, bypassing the special method exemption.\n\n7. **The `is_attr_protected` function** (utils.py) considers an attribute protected if it starts with `_`, is not just `_`, and does NOT have both leading `__` and trailing `__` (dunder pattern is not protected). So `_foo` and `__private` are protected, but `__eq__` is not.\n\n8. **The early exit via `_uses_mandatory_method_param`**: Before protected access checking occurs, `self._foo` access is excluded because it accesses via the mandatory first parameter (`self`/`cls`). Only access via `other._foo` (where `other` is not the first method parameter) proceeds to the protected-access check.",
    "rubric": [
      "Explains that `licit_protected_member = not attrname.startswith('__')` distinguishes single-underscore (`_foo`) from double-underscore (`__private`) attributes, and only single-underscore members are eligible for the special method exemption.",
      "Describes that `_is_called_inside_special_method` checks if the enclosing method's frame name is in `PYMETHODS`, which is built from `_SPECIAL_METHODS_PARAMS` and contains only recognized Python dunder methods (not arbitrary dunder-named methods like `__fake_special__`).",
      "Mentions the `check-protected-access-in-special-methods` config option (default False) that controls whether the exemption applies \u2014 when False, accessing `other._foo` inside a recognized special method is allowed without warning.",
      "Explains that double-underscore attributes like `__private` still trigger the warning even inside legitimate special methods because `licit_protected_member` is False for them.",
      "Describes how `is_attr_protected` determines what counts as protected: starts with `_`, is not just `_`, and doesn't match the dunder pattern (`__x__`), so `_foo` and `__private` are protected but `__eq__` is not.",
      "Notes that access via `self` (the mandatory first method parameter) is excluded before the protected-access check, so only access via other objects like `other._foo` triggers the check."
    ],
    "key_files": [
      "tests/functional/a/access/access_to_protected_members.py",
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/a/access/access_to_protected_members.py] tests.functional.a.access.access_to_protected_members.Issue1802\ntests.functional.a.access.access_to_protected_members.Issue1802:\n  Test for GitHub issue 1802\n\ntests.functional.a.access.access_to_protected_members.Issue1802.__eq__:\n  Test a correct access as the access to protected member is in a special method\n\ntests.functional.a.access.access_to_protected_members.Issue1802.not_in_special:\n  Test an incorrect access as the access to protected member is not inside a special method\n\ntests.functional.a.access.access_to_protected_members.Issue1802.__le__:\n  Test a correct access as the access to protected member\n        is inside a special method even if it is deeply nested\n\ntests.functional.a.access.access_to_protected_members.Issue1802.__fake_special__:\n  Test an incorrect access as the access\n        to protected member is not inside a licit special method",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific implementation details (line numbers, variable names, function names, configuration options) that are not present in the provided documentation and cannot be verified from it. These could be accurate extrapolations from the codebase or could be hallucinated details.",
      "The documentation does not mention double-underscore (__private) behavior at all - point 6 about __eq__ triggering for __private is an extrapolation not supported by the given documentation.",
      "The core claims about what the test class tests are consistent with the documentation: __eq__ and __le__ are correct accesses (special methods), not_in_special is incorrect, and __fake_special__ is incorrect because it's not a 'licit special method'."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only specific line numbers and some option description wording needed minor rewrites.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_37",
    "repo": "pylint",
    "question": "What is the architecture for detecting and dispatching `arguments-differ` vs `arguments-renamed` warnings when a child class overrides a parent method with different parameters?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `VarargsChild` class in `tests/functional/a/arguments_differ.py` is part of a test for pylint's `ClassChecker` (defined in `pylint/checkers/classes/class_checker.py`). The architecture works as follows:\n\n1. **Class hierarchy in the test**: `Varargs` defines `has_kwargs(self, arg, **kwargs)` and `no_kwargs(self, args)`. `VarargsChild(Varargs)` overrides both methods.\n\n2. **Detection logic**: The `ClassChecker.visit_functiondef` method iterates over ancestor methods to find the parent method, then calls `_check_signature`.\n\n3. **Signature comparison flow**: `_check_signature` calls the module-level function `_different_parameters(original, overridden, dummy_parameter_regex=...)` which performs the actual signature comparison.\n\n4. **For `has_kwargs`**: The parent has `**kwargs` but the child doesn't. In `_different_parameters`, `kwarg_lost = original.args.kwarg and not overridden.args.kwarg` evaluates to True, producing the message \"Variadics removed in\". This message doesn't contain \"Number\" or \"renamed\", so it results in emitting `arguments-differ` (W0221).\n\n5. **For `no_kwargs`**: The child adds `**kwargs`. However, the positional parameter name changed from `args` to `arg`. The `_has_different_parameters` function compares positional params via `zip_longest` and detects `original_param.name != overridden_param.name`, generating the message `\"Parameter 'args' has been renamed to 'arg' in\"`. Since this message contains \"renamed\", it dispatches to `arguments-renamed` (W0237).\n\n6. **Message dispatch logic**: In `_check_signature`, the error type is determined by string matching on the message content: \"Number\" \u2192 arguments-differ, \"renamed\" \u2192 arguments-renamed, else \u2192 arguments-differ.\n\n7. **Special method exclusion**: `_different_parameters` clears all messages if `original.name in PYMETHODS`, which includes `__init_subclass__` (defined in `_SPECIAL_METHODS_PARAMS` with `None` parameter count in `pylint/checkers/utils.py`).\n\n8. **The `_dummy_rgx` property** is a `@cached_property` (imported from `functools`) used in `_has_different_parameters` to ignore parameter name differences when a parameter matches the dummy variable regex (e.g., names like `_arg` or `dummy`). The logic `if any(dummy_parameter_regex.match(name) for name in names): continue` skips the name comparison.",
    "rubric": [
      "Explains that ClassChecker.visit_functiondef iterates over ancestor methods to find the parent method and calls _check_signature",
      "Mentions that _check_signature calls _different_parameters (module-level function) which performs the actual signature comparison",
      "Describes how kwarg_lost or vararg_lost detection works (e.g., original has **kwargs but overridden doesn't)",
      "Explains that _has_different_parameters compares parameter names via zip_longest and detects renames",
      "Describes the message dispatch logic: string matching on message content ('Number' \u2192 arguments-differ, 'renamed' \u2192 arguments-renamed, else \u2192 arguments-differ)",
      "Mentions the PYMETHODS exclusion that clears messages for special methods like __init_subclass__",
      "Mentions the dummy_parameter_regex mechanism that skips name comparison when parameters match the dummy variable pattern"
    ],
    "key_files": [
      "tests/functional/a/arguments_differ.py",
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/utils.py",
      "tests/functional/a/arguments_differ.txt"
    ],
    "source_doc": "[docstring: tests/functional/a/arguments_differ.py] tests.functional.a.arguments_differ.VarargsChild\ntests.functional.a.arguments_differ.VarargsChild.has_kwargs:\n  Not okay to lose capabilities. Also, type has changed.\n\ntests.functional.a.arguments_differ.VarargsChild.no_kwargs:\n  Addition of kwargs does not violate LSP, but first argument's name has changed.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor adjustments needed for exact line numbers and one LSP design rationale reference.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_39",
    "repo": "pylint",
    "question": "Where does data flow through the `consider-using-with` check from assignment tracking to message emission, and how are exclusive branches handled?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data flow for the `consider-using-with` control flow check in `TestControlFlow` works as follows:\n\n1. **Assignment tracking**: When an `Assign` node is visited, `_append_context_managers_to_stack` checks if the assigned value is a call to a function in `CALLS_RETURNING_CONTEXT_MANAGERS` (a frozenset including `_io.open`, `pathlib.Path.open`, `tempfile.NamedTemporaryFile`, `subprocess.Popen`, etc.). If so, it stores the variable name and the call node in the appropriate scope dictionary within `ConsiderUsingWithStack`.\n\n2. **Exclusivity check**: If the same variable name is already in the stack, `astroid.are_exclusive(node, existing_node)` is called. If the two assignments are in mutually exclusive branches (e.g., if/else, try/except, different except handlers), `are_exclusive` returns `True`, and the stack entry is simply updated to point to the newer assignment without emitting a message. If they are NOT exclusive, the message is emitted for the `existing_node` (the earlier assignment).\n\n3. **With-block removal**: `visit_with` iterates through the `with` statement's items. If the context expression (`var`) is a `nodes.Name`, it searches ALL stacks (function_scope, class_scope, module_scope) for that variable name and deletes it. This prevents the message from being emitted at scope exit.\n\n4. **Scope exit emission**: At `leave_functiondef`, `_emit_consider_using_with_if_needed` iterates over all remaining entries in `function_scope` stack and emits `consider-using-with` for each. The stack is then cleared.\n\n5. **Non-Call reassignments** (like `file_handle = None` in `test_triggers_if_reassigned_after_if_else`): Since `None` is not a `nodes.Call`, `_append_context_managers_to_stack` skips it (`if not isinstance(value, nodes.Call): continue`). The open() call remains in the stack and is reported at scope exit.\n\n6. **Return suppression**: `_is_a_return_statement` checks if the call is a child of `nodes.Return`. If so, `_check_consider_using_with` returns early, suppressing the message (used in `test_multiple_return_statements` and `test_suppress_on_return`).",
    "rubric": [
      "Explains that `_append_context_managers_to_stack` stores variable names and call nodes in the appropriate scope dictionary of `ConsiderUsingWithStack` when the assigned value calls a function in `CALLS_RETURNING_CONTEXT_MANAGERS`",
      "Describes the exclusivity check: if a variable name already exists in the stack, `astroid.are_exclusive` is called, and if True the entry is updated without emitting a message; otherwise, the message is emitted for the existing/earlier node",
      "Explains that `visit_with` searches all stacks for the variable name used in the `with` statement's context expression and deletes it, preventing message emission at scope exit",
      "Describes that `_emit_consider_using_with_if_needed` emits messages for all remaining entries at scope exit (leave_functiondef, leave_classdef, or leave_module) and then the stack is cleared",
      "Mentions that non-Call reassignments (e.g., `None`) are skipped because the code checks `isinstance(value, nodes.Call)` and continues past non-calls",
      "Mentions that `_is_a_return_statement` suppresses the message when the call is a child of a Return node"
    ],
    "key_files": [
      "pylint/checkers/refactoring/refactoring_checker.py",
      "tests/functional/c/consider/consider_using_with_open.py",
      "tests/functional/c/consider/consider_using_with_open.txt"
    ],
    "source_doc": "[docstring: tests/functional/c/consider/consider_using_with_open.py] tests.functional.c.consider.consider_using_with_open.TestControlFlow\ntests.functional.c.consider.consider_using_with_open.TestControlFlow:\n  The message is triggered if a context manager is assigned to a variable, which name is later\n    reassigned without the variable being used inside a ``with`` first.\n    E.g. the following would trigger the message:\n\n        a = open(\"foo\")  # <-- would trigger here\n        a = \"something new\"\n\n    But it must not happen that the logic which checks if the same variable is assigned multiple\n    times in different code branches where only one of those assign statements is hit at runtime.\n    For example, the variable could be assigned in an if-else construct.\n\n    These tests check that the message is not triggered in those circumstances.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive implementation details (specific line numbers, function names, method names, frozenset contents) that are not present in the documentation and cannot be verified from it alone. These appear to be plausible extrapolations from the actual codebase but are not supported by the provided documentation chunk.",
      "The documentation only describes the general behavior: the message triggers when a context manager variable is reassigned without being used in a `with` statement first, and that it must NOT trigger when assignments are in mutually exclusive branches (like if-else). The answer's detailed implementation description is consistent with this described behavior.",
      "The answer's description of the exclusivity check (point 2) aligns with the documentation's statement about not triggering in different code branches where only one assign statement is hit at runtime.",
      "The answer references specific test method names like `test_triggers_if_reassigned_after_if_else` and `test_multiple_return_statements` which are not mentioned in the documentation chunk."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor line number references were removed as they may not match stripped code, but all functional claims are verifiable from the source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_38",
    "repo": "pylint",
    "question": "Why does the `arguments-differ` checker allow a child method with `*args, **kwargs` to override a parent method that has explicit positional and keyword-only parameters?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `AbstractFoo` class in the `arguments_differ.py` test file is used to test pylint's `arguments-differ` checker specifically around keyword-only parameter handling. The `kwonly_1` through `kwonly_6` methods verify that pylint correctly detects when child classes change keyword-only parameters to positional ones, remove keyword-only parameters, or remove variadics.\n\nThe core detection logic lives in `_different_parameters()` in `pylint/checkers/classes/class_checker.py`. In this function, variadics in the overriding (child) method are treated as a wildcard that can match any parameters from the parent. Specifically:\n\n1. If the overridden method has `*args` (vararg), the original's positional parameters are filtered down to only those whose names also appear in the overridden's positional parameters. This means `*args` effectively absorbs all parent positional params.\n\n2. If the overridden method has `**kwargs` (kwarg), the original's keyword-only args are filtered down to only those whose names also appear in the overridden's keyword-only args. This means `**kwargs` absorbs all parent keyword-only params.\n\nThis is why `kwonly_6` in class `Foo` (using `*args, **kwargs`) and class `Foo2` (using `first, *args, **kwargs`) are valid overrides of `AbstractFoo.kwonly_6(self, first, second, *, third)` \u2014 they don't trigger errors because they can still accept any call that the parent could accept.\n\nThe `_has_different_keyword_only_parameters()` function checks that all original keyword-only parameter names exist in the overridden method's keyword-only params, and any new keyword-only params in the overridden method must have default values (checked via `overridden[0].parent.default_value(name)`, raising `astroid.NoDefault` if there's no default). This is why `ClassWithNewNonDefaultKeywordOnly.method(self, *, arg2, arg1=None)` triggers the error \u2014 `arg2` is a new keyword-only param without a default.\n\nAdditionally, the checker explicitly skips special dunder methods listed in `PYMETHODS` (populated from `SPECIAL_METHODS_PARAMS`) \u2014 if the method name is in that set (like `__init_subclass__`), `output_messages.clear()` is called to suppress any findings. These methods are instead handled by the `unexpected-special-method-signature` checker.",
    "rubric": [
      "Explains that `_different_parameters()` filters the original's positional parameters when the overriding method has `*args` (vararg), keeping only those whose names also appear in the overridden's positional params \u2014 effectively letting `*args` absorb/match any parent positional params.",
      "Explains that `_different_parameters()` similarly filters the original's keyword-only args when the overriding method has `**kwargs` (kwarg), keeping only those whose names appear in the overridden's keyword-only args \u2014 effectively letting `**kwargs` absorb/match any parent keyword-only params.",
      "Mentions or references the test cases (e.g., `kwonly_6`) where `*args, **kwargs` or `first, *args, **kwargs` validly override a parent with explicit positional and keyword-only parameters.",
      "Explains that the rationale is LSP compliance: a method using variadics can still accept any call that the parent signature could accept, so it's a valid override."
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "tests/functional/a/arguments_differ.py",
      "tests/functional/a/arguments_differ.txt",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/a/arguments_differ.py] tests.functional.a.arguments_differ.AbstractFoo\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_1:\n  Normal positional with two positional only params.\n\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_2:\n  Two positional only parameter.\n\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_3:\n  Two positional only params.\n\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_4:\n  One positional only and another with a default.\n\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_5:\n  Keyword only and keyword variadics.\n\ntests.functional.a.arguments_differ.AbstractFoo.kwonly_6:\n  Two positional and one keyword",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer makes extensive claims about implementation details (line numbers, function names, specific logic) that cannot be verified from the provided documentation, which only contains docstrings describing the parameters of AbstractFoo's methods.",
      "The documentation describes kwonly_1 as 'Normal positional with two positional only params', kwonly_2 as 'Two positional only parameter', kwonly_3 as 'Two positional only params', kwonly_4 as 'One positional only and another with a default', kwonly_5 as 'Keyword only and keyword variadics', kwonly_6 as 'Two positional and one keyword'. The answer's characterization of kwonly_6 as having signature 'self, first, second, *, third' is plausible but not directly verifiable from the doc.",
      "The answer describes the purpose as testing 'keyword-only parameter handling' but the documentation shows several methods are described in terms of 'positional only params', suggesting the test covers positional-only parameters as well, not just keyword-only ones.",
      "Most of the answer's detailed claims about implementation (line numbers, specific function names, class names like ClassWithNewNonDefaultKeywordOnly, PYMETHODS set, etc.) are extrapolations/hallucinations beyond what the documentation provides."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from the stripped code logic, test structure, and output file; minor doc leakage in design rationale language (LSP, W0221 code).",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_40",
    "repo": "pylint",
    "question": "How does pylint's VariablesChecker avoid false positives for names used in decorators and default arguments that reference class-level attributes?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `decorator_scope.py` functional test verifies that pylint's `VariablesChecker` correctly resolves names used in decorators and function default arguments within a class body, treating them as having access to the class namespace (unlike function bodies which don't). The test has no associated `.txt` file, meaning it expects zero pylint messages.\n\nThe test file defines a class `Test` with:\n- `ident = lambda x: x` (a class-level attribute)\n- `@ident(ident)` (decorator using class-level name)\n- `def method(self, val=ident(7), func=ident):` (defaults using class-level name)\n- Function body uses `func(val)` (resolved from parameters, not class scope)\n\n**How pylint handles this:**\n\n1. **Scope consumer stack**: When pylint visits the class, it pushes a `NamesConsumer(node, \"class\")` onto `self._to_consume`. When visiting the method, it pushes `NamesConsumer(node, \"function\")`.\n\n2. **Key method: `_should_node_be_skipped`**: When checking a name used in a decorator or default argument, the method detects the function's own consumer should be skipped. Specifically, if `consumer.scope_type == \"function\"` and `_defined_in_function_definition(node, consumer.node)` is True, it returns True (skip), causing the search to start from the parent frame of the function instead of the function frame.\n\n3. **Key method: `_defined_in_function_definition`**: This static method checks whether a node is in a function's annotations, arguments, decorators, or return annotation. For decorators, it checks `frame.decorators and frame.decorators.parent_of(node)`. The condition `node.statement() is frame` ensures the node's statement is the function definition itself.\n\n4. **Key method: `_ignore_class_scope`**: When the search reaches the class-level consumer (non-start index), this method determines whether to skip the class scope. For names in decorators/defaults, `in_annotation_or_default_or_decorator` is True, so it uses `frame.parent.scope().locals` (the class's locals) to check if the name exists. The method returns False (don't ignore) when the name IS in the class's frame locals, allowing consumption.\n\n5. **`is_func_decorator` utility** (utils.py): Walks up ancestors looking for a `nodes.Decorators` parent, stopping at statements, Lambdas, or ComprehensionScopes.\n\n6. **No `.txt` output file**: When `_open_expected_file` encounters a FileNotFoundError, it returns `StringIO(\"\")`, meaning no expected messages - confirming the test passes with zero warnings.",
    "rubric": [
      "Explains that `_should_node_be_skipped` detects when a name is in a function's decorators/defaults (via `_defined_in_function_definition`) and skips the function's own consumer/scope, causing lookup to proceed to the parent (class) scope",
      "Describes `_defined_in_function_definition` checking whether the node is in frame.args, frame.decorators, or frame.returns (and that node.statement() is frame)",
      "Explains that `_ignore_class_scope` uses `in_annotation_or_default_or_decorator` to look up the name in `frame.parent.scope().locals` (the class's locals) and returns False (don't ignore) when the name exists there",
      "Mentions the NamesConsumer stack (`_to_consume`) and iteration from start_index to 0, where skipping a consumer causes the search to continue to the next outer scope",
      "Notes that when no `.txt` file exists for a functional test, `_open_expected_file` returns `StringIO(\"\")`, meaning zero expected messages (no false positives)"
    ],
    "key_files": [
      "tests/functional/d/decorator_scope.py",
      "pylint/checkers/variables.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/d/decorator_scope.py] tests.functional.d.decorator_scope\nTest that decorators sees the class namespace - just like\nfunction default values does but function body doesn't.\n\nhttps://www.logilab.net/elo/ticket/3711 - bug finding decorator arguments\nhttps://www.logilab.net/elo/ticket/5626 - name resolution bug inside classes",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor doc leakage from a comment quote in the _should_node_be_skipped description; otherwise claims are verifiable from code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_42",
    "repo": "pylint",
    "question": "Why does the docparams checker allow missing parameter documentation when a class docstring contains \"For the parameters, see\"?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `ClassFooTwo` class in the test file is related to the 'For the parameters, see' pattern in pylint's docparams checker. The code allows class docstrings to suppress missing parameter documentation warnings when `params_documented_elsewhere()` returns `True`, meaning parameters are documented elsewhere.\n\nThe mechanism works through a two-level tolerance system in `DocstringParameterChecker.check_functiondef_params()` (in `pylint/extensions/docparams.py`):\n\n1. When `__init__` is visited and belongs to a class, both the class docstring and the `__init__` docstring are checked.\n2. The class docstring's `params_documented_elsewhere()` method (defined in `_check_docs_utils.py`) uses the regex `re_for_parameters_see` which matches `For\\s+the\\s+(other)?\\s*parameters\\s*,\\s+see` (with `re.X | re.S` flags).\n3. If the class docstring matches this pattern, `node_allow_no_param` is set to `True` (via `class_doc.has_params() or class_doc.params_documented_elsewhere() or None`), which is then passed to `check_arguments_in_docstring` for the `__init__` method's own docstring check. This causes the `__init__`'s missing params to be tolerated even with `accept-no-param-doc=no`.\n4. For the class docstring check itself, `class_allow_no_param` is set based on `node_doc.has_params() or node_doc.params_documented_elsewhere() or None`, and passed to `check_arguments_in_docstring` for the class_doc check.\n\nThis mechanism prevents missing-param warnings when the docstring references other functions for parameter details instead of duplicating documentation.",
    "rubric": [
      "Explains that `params_documented_elsewhere()` checks the docstring against `re_for_parameters_see` regex matching 'For the (other) parameters, see'",
      "Describes the two-level tolerance system: `node_allow_no_param` is set True when the class docstring documents params elsewhere, and `class_allow_no_param` when __init__ docstring does",
      "Explains that when `__init__` is visited in a class, both the class docstring and the `__init__` docstring are retrieved and cross-checked",
      "Notes that if either docstring has params or references params elsewhere, the other docstring is allowed to omit parameter documentation",
      "Mentions the purpose is to avoid forcing duplication of parameter documentation when it's already documented in a referenced function"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required.py] tests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFooTwo\ntests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFooTwo:\n  test_see_sentence_for_constr_params_in_class\n    Example usage of \"For the parameters, see\" in class docstring\n\n    For the parameters, see :func:`bla`\n\ntests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFooTwo.__init__:\n  init",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes highly specific implementation details (line numbers, method names, regex patterns, variable names) that are not present in the provided documentation and cannot be verified from it. These may be accurate based on the actual codebase but are extrapolations beyond the given doc chunk.",
      "The core claim about ClassFooTwo demonstrating the 'For the parameters, see' pattern is supported by the documentation, which shows the docstring containing 'For the parameters, see :func:`bla`'.",
      "The claim about the class being in 'tests/functional/ext/docparams/parameter/missing_param_doc_required.py' is supported by the documentation.",
      "The description of ClassFooTwo having an __init__ method is supported by the documentation."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code logic; minor adjustments needed for design rationale language and references to stripped docstring content in test files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_43",
    "repo": "pylint",
    "question": "Where is the logic that generates missing-param-doc and missing-type-doc warnings for Google-style docstrings, and how does it determine which params are missing?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `missing-param-doc` and `missing-type-doc` warnings for `Foo.test_missing_method_params_in_google_docstring` in the test file are generated by the `DocstringParameterChecker` class located in `pylint/extensions/docparams.py`. The checking flow is:\n\n1. The checker is loaded via `pylint.extensions.docparams` as specified in the `.rc` file with `accept-no-param-doc=no`.\n\n2. When pylint visits the method definition, `DocstringParameterChecker.visit_functiondef()` is called, which invokes `utils.docstringify()` to determine the docstring type.\n\n3. `docstringify()` in `pylint/extensions/_check_docs_utils.py` tries each docstring type (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) and selects the one with the most `matching_sections()`. For Google-style docs, `GoogleDocstring.matching_sections()` counts matches of `re_param_section`, `re_raise_section`, `re_returns_section`, `re_yields_section`, and `re_property_returns_line`.\n\n4. `check_functiondef_params()` calls `check_arguments_in_docstring()`, which:\n   - Collects expected argument names from the AST node (excluding `self` via `not_needed_param_in_docstring = {'self', 'cls'}`)\n   - Calls `doc.match_param_docs()` which parses the Google-style \"Args:\" section using `re_param_line` regex and returns sets of documented param names and param types\n   - Compares expected vs found using `_compare_missing_args()`\n\n5. For the test case, `self` is excluded, leaving expected params `{x, y}`. The docstring only documents `x` (with description but no type), so `params_with_doc = {'x'}` and `params_with_type = set()`. This results in \"y\" missing in parameter documentation (W9015) and \"x, y\" missing in parameter type documentation (W9016).\n\n6. The `GoogleDocstring.re_param_section` regex matches section headers \"Args\", \"Arguments\", or \"Parameters\" followed by a colon. The `re_param_line` regex pattern captures an identifier (potentially with asterisks), an optional parenthesized type, and a description.\n\n7. The `_parse_section()` method splits the section content into individual entries based on indentation level, where lines at the minimum indentation start new parameter entries.",
    "rubric": [
      "Identifies DocstringParameterChecker in pylint/extensions/docparams.py as the checker class",
      "Mentions visit_functiondef() as the entry point when visiting function definitions",
      "Explains that docstringify() in _check_docs_utils.py selects the docstring type by comparing matching_sections() counts across docstring classes (Sphinx, Epytext, Google, Numpy)",
      "Describes that GoogleDocstring.matching_sections() counts matches of regex patterns like re_param_section, re_raise_section, re_returns_section, re_yields_section",
      "Explains that check_arguments_in_docstring() collects expected params from the AST and excludes 'self'/'cls' via not_needed_param_in_docstring",
      "Mentions that doc.match_param_docs() parses the Args section using re_param_line regex to find documented params and types",
      "Explains that _compare_missing_args() compares expected vs documented params to produce the warnings"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.rc",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Google.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Google.Foo\ntests.functional.ext.docparams.parameter.missing_param_doc_required_Google.Foo.test_missing_method_params_in_google_docstring:\n  Example of a class method with missing parameter documentation in\n        the Google style docstring\n\n        missing parameter documentation\n\n        Args:\n            x: bla",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only shows the docstring and method signature context \u2014 it describes a method with missing parameter documentation in Google style, with only 'x' documented in the Args section. The answer's claims about specific line numbers (192, 259, 220, 508, 400, 190, 772), regex names, method names, and internal implementation details are all extrapolations beyond what the documentation provides.",
      "The claim that expected params are {x, y} is a plausible inference (the docstring says 'missing parameter documentation' and only documents x), but the documentation doesn't explicitly list the method's parameters \u2014 it only shows the docstring content.",
      "Most of the answer describes internal pylint implementation details (specific class names, line numbers, regex patterns, method flow) that are not present in the provided documentation chunk and cannot be verified from it."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only specific line numbers needed minor adjustment as they may shift after doc removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_41",
    "repo": "pylint",
    "question": "What is the \"For the parameters, see\" tolerance mechanism in the docparams extension, and how does it suppress missing-parameter warnings?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The \"For the parameters, see\" concept in pylint's docparams extension is a mechanism that suppresses missing parameter documentation warnings when a specific pattern is found in the docstring. It is implemented via a class-level compiled regex `re_for_parameters_see` on the `Docstring` base class in `pylint/extensions/_check_docs_utils.py`, which matches the pattern `For\\s+the\\s+(other)?\\s*parameters\\s*,\\s+see` with `re.X | re.S` flags. This means it matches both \"For the parameters, see\" and \"For the other parameters, see\" (with flexible whitespace).\n\nThe `params_documented_elsewhere()` method simply checks whether this regex matches anywhere in the docstring text, returning a boolean. This method is inherited by all docstring subclasses (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) since none of them override it.\n\nThe tolerance mechanism is used in three places within `DocstringParameterChecker` (in `pylint/extensions/docparams.py`):\n\n1. In `check_arguments_in_docstring()`: `tolerate_missing_params = doc.params_documented_elsewhere()` \u2014 this sets a flag that, when True, causes the checker to skip all missing-param-doc, missing-type-doc, and missing-any-param-doc messages.\n\n2. In `check_functiondef_params()`: For constructors (`__init__` or `__new__`), the checker allows no-param documentation in either the class docstring or the `__init__` docstring if the *other* one has `params_documented_elsewhere()`. Specifically, `node_allow_no_param` is set to True if the class docstring has `params_documented_elsewhere()`, and `class_allow_no_param` is set to True if the init docstring has `params_documented_elsewhere()`.\n\nIn the test file `missing_param_doc_required.py`, even though `accept-no-param-doc=no` is configured, the `ClassFoo` and `ClassFooTwo` classes produce NO warnings (their docstrings are stripped but the output file confirms this). The expected output file confirms only the first function (`test_don_t_tolerate_no_param_documentation_at_all`) and `test_kwonlyargs_are_taken_in_account` emit warnings.\n\nThe `Docstring.__init__` method processes the raw docstring by calling `expandtabs()` on it before storing as `self.doc`. The `not_needed_param_in_docstring` set in `DocstringParameterChecker` contains `{\"self\", \"cls\"}`, and `constructor_names` is `{\"__init__\", \"__new__\"}`.",
    "rubric": [
      "Explains that `re_for_parameters_see` is a compiled regex on the `Docstring` base class matching 'For the parameters, see' and 'For the other parameters, see' with flexible whitespace (re.X | re.S flags)",
      "Describes the `params_documented_elsewhere()` method that checks if the regex matches anywhere in the docstring text, returning a boolean",
      "Notes that all docstring subclasses (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) inherit this method without overriding it",
      "Explains usage in `check_arguments_in_docstring`: when `tolerate_missing_params` is True, the checker skips missing-param-doc, missing-type-doc, and missing-any-param-doc messages",
      "Explains usage in `check_functiondef_params` for constructors: `node_allow_no_param` is set based on the class docstring's `params_documented_elsewhere()`, and `class_allow_no_param` is set based on the init docstring's `params_documented_elsewhere()`",
      "Mentions that `not_needed_param_in_docstring` is {'self', 'cls'} and `constructor_names` is {'__init__', '__new__'}"
    ],
    "key_files": [
      "pylint/extensions/_check_docs_utils.py",
      "pylint/extensions/docparams.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required.txt",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required.py] tests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFoo\ntests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFoo:\n  Example usage of \"For the parameters, see\" in init docstring\n\ntests.functional.ext.docparams.parameter.missing_param_doc_required.ClassFoo.__init__:\n  docstring foo constructor\n\n        For the parameters, see :func:`bla`",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is extremely minimal - just a docstring showing ClassFoo's usage of 'For the parameters, see' pattern in an __init__ docstring. Most of the generated answer contains detailed implementation claims (line numbers, method names, regex patterns, class hierarchies) that cannot be verified from the provided documentation chunk alone.",
      "The core claim that 'For the parameters, see :func:`bla`' in the __init__ docstring suppresses warnings is consistent with the documentation's description of ClassFoo as an 'Example usage of \"For the parameters, see\" in init docstring'.",
      "Specific line numbers (278, 330-331, 552, 234-241), regex patterns, method names, and implementation details are plausible extrapolations from source code knowledge but cannot be verified against the provided documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with line numbers and docstring content from test files that were stripped.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_44",
    "repo": "pylint",
    "question": "How does the NumPy docstring parameter checker detect missing parameter documentation and distinguish between missing docs vs missing types?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `missing_param_doc_required_Numpy.py` test file validates how pylint's `DocstringParameterChecker` (in `pylint/extensions/docparams.py`) detects missing parameter documentation in NumPy-style docstrings. Here's how the algorithm works for the `Foo.test_missing_method_params_in_numpy_docstring` case:\n\n1. **Docstring type detection** (`docstringify` in `_check_docs_utils.py`): The function iterates over all docstring types (Sphinx, Epytext, Google, Numpy), instantiates each, calls `matching_sections()`, and picks the type with the highest number of matching sections. For the Numpy type, sections are identified by a header word (e.g., 'Parameters') followed by a line of dashes/equals (`[-=]+`).\n\n2. **Section parsing** (`_parse_section` inherited from `GoogleDocstring`): After matching the section regex, individual entries are split by indentation level. For NumpyDocstring, `min_section_indent` returns `len(section_match.group(1))` (same indent as header), unlike GoogleDocstring which returns `header_indent + 1`. The parser also uses `_is_section_header` (which checks for lines matching `\\s*-+$`) to break parsing when a new section's underline is encountered.\n\n3. **Parameter matching** (`NumpyDocstring.match_param_docs`): For each entry, it first matches with `re_param_line` (named groups: `param_name`, `param_type`, `param_desc`). Then it applies a special check `re_only_desc = re.match(r\"\\s*(\\*{0,2}\\w+)\\s*:?\\n\\s*\\w*$\", entry)` \u2014 if this matches (parameter has description only, no type), it sets `param_type = None` and extracts `param_desc` from the `param_type` group. For `x:\\n    bla`, this re_only_desc pattern matches, so `x` gets added to `params_with_doc` but NOT to `params_with_type`.\n\n4. **Comparison logic** (`check_arguments_in_docstring`): \n   - Collects `expected_argument_names = {\"self\", \"x\", \"y\"}`\n   - Subtracts `not_needed_param_in_docstring = {\"self\", \"cls\"}` \n   - After `match_param_docs` returns `params_with_doc={\"x\"}`, `params_with_type=set()`\n   - `params_with_type` is augmented with `args_with_annotation(arguments_node)`, which in this case returns empty since the test function has no type annotations\n   - Since `accept-no-param-doc=no` in the `.rc` file, and `params_with_doc` is non-empty (so `tolerate_missing_params` is False)\n   - `missing_param_doc = {\"x\", \"y\"} - {\"x\"} = {\"y\"}` \u2192 emits `missing-param-doc` for `y`\n   - `missing_type_doc = {\"x\", \"y\"} - set() = {\"x\", \"y\"}` \u2192 emits `missing-type-doc` for `x, y`\n\n5. **Key difference from Google style**: The NumpyDocstring `re_param_line` regex uses named groups (`(?P<param_name>...)`) and separates param_name from type via `(\\s?(:| \\n))`, allowing either colon or newline as separator. Google's `re_param_line` uses positional groups and expects type in parentheses before colon.",
    "rubric": [
      "Explains docstring type detection via `docstringify`: iterates over docstring types, calls `matching_sections()` on each, and picks the one with the highest count",
      "Describes how NumpyDocstring sections are identified using a regex with a header word followed by a line of dashes/equals",
      "Explains `_parse_section` splits section content into entries by indentation level, using `min_section_indent` which returns `len(section_match.group(1))` for Numpy",
      "Describes how `_is_section_header` (checking for lines matching `\\s*-+$`) terminates parsing when a new section underline is encountered",
      "Explains the `re_only_desc` check in `match_param_docs`: if a param entry matches `\\s*(\\*{0,2}\\w+)\\s*:?\\n\\s*\\w*$`, the parameter is added to `params_with_doc` but NOT to `params_with_type`",
      "Describes the comparison logic in `check_arguments_in_docstring`: subtracts `not_needed_param_in_docstring` (self, cls) from expected args, then computes `missing_param_doc` and `missing_type_doc` as set differences",
      "Notes that `params_with_type` is augmented with `args_with_annotation` (type-annotated function parameters) before computing missing type docs"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.rc",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Numpy.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Numpy.Foo\ntests.functional.ext.docparams.parameter.missing_param_doc_required_Numpy.Foo.test_missing_method_params_in_numpy_docstring:\n  Example of a class method with missing parameter documentation in\n        the Numpy style docstring\n\n        missing parameter documentation\n\n        Parameters\n        ----------\n        x:\n            bla",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides a docstring and method signature showing parameter 'x' is documented in Numpy style with 'bla' as description, and mentions 'missing parameter documentation'. The answer extrapolates extensively about internal implementation details (regex patterns, parsing logic, comparison logic) that are not present in the provided documentation chunk.",
      "The answer claims the method has parameters 'self', 'x', and 'y' - while 'y' being a missing parameter is plausible given the 'missing parameter documentation' note, the documentation only explicitly shows 'x' as a documented parameter and doesn't explicitly list the method signature with 'y'.",
      "Detailed regex patterns like `re_only_desc = re.match(r\"\\s*(\\*{0,2}\\w+)\\s*:?\\n\\s*\\w*$\", entry)` and specific implementation details about `_parse_section`, `min_section_indent`, `_is_section_header` are not derivable from the provided documentation and appear to be extrapolations from source code knowledge.",
      "The claim about `.rc` file containing `accept-no-param-doc=no` is not mentioned in the documentation."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code, class structures, regex patterns, and test output files; only a minor 'PEP 484' reference was documentation-derived.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_45",
    "repo": "pylint",
    "question": "What components and configuration options are involved in detecting missing parameter documentation for Sphinx-style docstrings in the docparams extension?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The test file `missing_param_doc_required_Sphinx.py` tests the `DocstringParameterChecker` class from `pylint/extensions/docparams.py`, which is loaded as a plugin via `pylint.extensions.docparams`. The checker depends on `pylint/extensions/_check_docs_utils.py` for docstring parsing.\n\nFor the `Foo.test_missing_method_params_in_sphinx_docstring` method (parameters `self, x, y`), two messages would be emitted:\n1. `missing-param-doc` (W9015): parameters missing in parameter documentation\n2. `missing-type-doc` (W9016): parameters missing in parameter type documentation\n\nKey architectural details:\n- The `docstringify()` function auto-detects the docstring format by instantiating all four docstring types (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) and selecting the one with the most `matching_sections()`. For Sphinx, this counts matches of `re_param_in_docstring`, `re_raise_in_docstring`, `re_rtype_in_docstring`, `re_returns_in_docstring`, and `re_property_type_in_docstring`.\n- `SphinxDocstring.supports_yields` is `False`, which causes the checker to use `has_returns()`/`has_rtype()` instead of yields-specific methods for generators (as seen in `check_functiondef_yields` and `visit_yield`).\n- `DocstringParameterChecker.not_needed_param_in_docstring = {\"self\", \"cls\"}` excludes these from being required in docs.\n- The `.rc` configuration sets `accept-no-param-doc=no`, which means the checker will NOT tolerate completely missing parameter documentation (normally it would if `accept_no_param_doc` is True and no params are documented at all).\n- `SphinxDocstring.match_param_docs()` uses `re_param_in_docstring` regex which recognizes Sphinx keywords: `param`, `parameter`, `arg`, `argument`, `key`, `keyword`, followed by an optional inline type and the parameter name.\n- The `EpytextDocstring` class inherits from `SphinxDocstring` and reuses its regex patterns by replacing the leading `:` with `@`.\n- `DocstringParameterChecker` inherits from `BaseChecker` (in `pylint/checkers/base_checker.py`).\n- The `register()` function at the bottom of `docparams.py` creates and registers the checker with `linter.register_checker(DocstringParameterChecker(linter))`.\n- `params_with_type` is augmented with `utils.args_with_annotation(arguments_node)` to account for type annotations in function signatures counting as type documentation (the `args_with_annotation` function collects parameter names that have annotations).",
    "rubric": [
      "DocstringParameterChecker class inherits from BaseChecker and is the main checker",
      "The checker uses _check_docs_utils.py for docstring parsing via the docstringify() function",
      "docstringify() auto-detects format by instantiating SphinxDocstring, EpytextDocstring, GoogleDocstring, and NumpyDocstring and selecting the one with the most matching_sections()",
      "SphinxDocstring.matching_sections() counts matches of re_param_in_docstring, re_raise_in_docstring, re_rtype_in_docstring, re_returns_in_docstring, and re_property_type_in_docstring",
      "SphinxDocstring.match_param_docs() uses re_param_in_docstring regex recognizing keywords: param, parameter, arg, argument, key, keyword",
      "not_needed_param_in_docstring = {'self', 'cls'} excludes these from requiring documentation",
      "The accept-no-param-doc option (default True) controls whether completely undocumented parameters are tolerated",
      "Messages W9015 (missing-param-doc) and W9016 (missing-type-doc) are emitted for undocumented parameters",
      "params_with_type is augmented with args_with_annotation() to count type annotations as type documentation",
      "EpytextDocstring inherits from SphinxDocstring and reuses its regex patterns by replacing the leading ':' with '@'",
      "The register() function creates and registers DocstringParameterChecker with the linter",
      "SphinxDocstring.supports_yields is False, causing yield checking to fall back to has_returns()/has_rtype()"
    ],
    "key_files": [
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Sphinx.Foo\ntests.functional.ext.docparams.parameter.missing_param_doc_required_Sphinx.Foo.test_missing_method_params_in_sphinx_docstring:\n  Example of a class method with missing parameter documentation in\n        the Sphinx style docstring\n\n        missing parameter documentation\n\n        :param x: bla",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only shows a single method with docstring mentioning ':param x: bla' and notes 'missing parameter documentation'. The generated answer elaborates extensively on specific message codes (W9015, W9016), regex patterns, class hierarchies, configuration details, and architectural internals that are not present in the provided documentation chunk. These are plausible extrapolations based on knowledge of pylint's codebase but are not supported by the given documentation.",
      "The claim about two specific messages being emitted (missing-param-doc for 'y' and missing-type-doc for 'x, y') is not directly stated in the documentation, though it is a reasonable inference from the method signature and docstring shown.",
      "The method signature parameters (self, x, y) are not explicitly shown in the documentation - only ':param x: bla' and 'missing parameter documentation' are mentioned, though having parameter 'y' is a plausible inference.",
      "All architectural details about SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring, docstringify(), register(), BaseChecker, _ArgumentsProvider, etc. go far beyond the provided documentation chunk."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor leakage from stripped docstrings about specific documented parameters and PEP 484 naming; most claims verifiable from code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_46",
    "repo": "pylint",
    "question": "Why are the regex patterns in the Sphinx docstring checker compiled as class-level attributes rather than inside methods, and how does `docstringify` make that matter?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The Sphinx docstring parameter checking in pylint uses several design decisions:\n\n1. **Pre-compiled regex patterns as class-level attributes**: All regex patterns in `SphinxDocstring` (like `re_param_in_docstring`, `re_type_in_docstring`, `re_raise_in_docstring`, `re_rtype_in_docstring`, `re_returns_in_docstring`) are compiled once at class definition time using `re.compile()` as class attributes, not in `__init__` or instance methods. Since `docstringify()` creates instances of all 4 docstring types (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) for every function/method it processes, compiling regexes at the class level means they are not recompiled on each instantiation.\n\n2. **`docstringify()` auto-detection via `matching_sections()` count**: The `docstringify()` function in `_check_docs_utils.py` instantiates all four docstring types and calls `matching_sections()` on each, selecting the one with the highest count. `SphinxDocstring.matching_sections()` checks 5 regex searches (param, raise, rtype, returns, property_type) and sums the boolean results. It counts section matches to determine the most likely format.\n\n3. **Early returns to avoid unnecessary work**: In `check_functiondef_returns()`, there's an early return when `supports_yields` is `False` (as it is for `SphinxDocstring`) and the function is a generator. Similarly, `check_functiondef_yields()` returns immediately if `supports_yields` is `False`. The early return occurs before `nodes_of_class(nodes.Return)` traversal for generator functions when using Sphinx docstrings. The `visit_functiondef` method also has early-return guards: it skips overload stubs (`is_overload_stub`), functions matching `no-docstring-rgx`, and functions shorter than `docstring-min-length`.\n\n4. **`EpytextDocstring` inherits from `SphinxDocstring`**: `EpytextDocstring` extends `SphinxDocstring` and simply replaces the initial `:` with `@` in the raw regex strings (e.g., `SphinxDocstring.re_param_raw.replace(':', '@', 1)`), reusing the same parsing logic and keeping patterns compiled at class level.\n\n5. **`Docstring.__init__` calls `expandtabs()`**: The base `Docstring.__init__` normalizes tabs to spaces once via `docstring.expandtabs()`.\n\nFor the specific test function `test_existing_func_params_in_sphinx_docstring`, the regex `re_param_in_docstring` matches all four Sphinx keyword variants (`:param`, `:parameter`, `:arg`, `:keyword`) with optional inline type (e.g., `int zarg`, `my.qualified.type warg`). The `match_param_docs()` method collects both `params_with_doc` (all 4 params) and `params_with_type` (zarg and warg from inline types, plus xarg and yarg from separate `:type` directives), so no missing-doc or missing-type warnings are emitted.",
    "rubric": [
      "Explains that regex patterns (re_param_in_docstring, re_type_in_docstring, re_raise_in_docstring, etc.) are compiled once at class definition time via re.compile() as class attributes on SphinxDocstring",
      "Explains that docstringify() instantiates all four docstring types (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) for every function it processes, so class-level compilation avoids recompilation on each instantiation",
      "Mentions that matching_sections() uses these pre-compiled regexes to count how many docstring sections match, and docstringify picks the type with the highest count",
      "Notes that EpytextDocstring inherits from SphinxDocstring and reuses the same pattern structure, just replacing the initial ':' with '@' in the raw strings, also compiled at class level",
      "Mentions early-return optimizations: check_functiondef_returns exits early for generators when supports_yields is False (Sphinx), and visit_functiondef skips overload stubs, no-docstring-rgx matches, and short functions"
    ],
    "key_files": [
      "pylint/extensions/_check_docs_utils.py",
      "pylint/extensions/docparams.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_Sphinx.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_Sphinx.test_existing_func_params_in_sphinx_docstring\nExample of a function with correctly documented parameters and\n    return values (Sphinx style)\n\n    :param xarg: bla xarg\n    :type xarg: int\n\n    :parameter yarg: bla yarg\n    :type yarg: my.qualified.type\n\n    :arg int zarg: bla zarg\n\n    :keyword my.qualified.type warg: bla warg\n\n    :return: sum\n    :rtype: float",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only shows a docstring example with Sphinx-style parameter documentation. The generated answer makes extensive claims about internal implementation details (regex patterns, class hierarchies, method behaviors, line numbers) that are not present in or verifiable from the provided documentation chunk.",
      "The answer discusses performance-oriented design decisions, class-level attributes, early returns, and inheritance hierarchies that cannot be verified from the given documentation snippet.",
      "However, the final paragraph's claim about the four Sphinx keyword variants (:param, :parameter, :arg, :keyword) is directly supported by the documentation example, and the claim about inline types for zarg and warg and separate :type directives for xarg and yarg is also consistent with the documented example."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor design rationale phrasings were trimmed but nearly all factual content is recoverable from the code structure.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_47",
    "repo": "pylint",
    "question": "Where is the `no-docstring-rgx` option defined and where is it applied to skip parameter doc checks in the docparams extension?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `no-docstring-rgx` check for `__init__` in the test file `missing_param_doc_required_no_doc_rgx_check_init.py` is located in the `DocstringParameterChecker` class within `pylint/extensions/docparams.py`. Specifically, the regex matching logic is in the `visit_functiondef` method of that file. The `no-docstring-rgx` option itself is defined in `pylint/checkers/base/docstring_checker.py` with a default value of `re.compile(\"^_\")` (defined as `NO_REQUIRED_DOC_RGX`). It is also listed as a `GLOBAL_OPTION_PATTERN` in `pylint/utils/utils.py`. The test's `.rc` file configures `no-docstring-rgx=^(?!__init__$)_`, which uses a negative lookahead to match all underscore-prefixed names *except* `__init__`, so `__init__` gets checked for parameter documentation while `_private_method` does not.",
    "rubric": [
      "Mentions the option is defined in the `DocStringChecker` class in `pylint/checkers/base/docstring_checker.py`",
      "Notes the default value is `NO_REQUIRED_DOC_RGX = re.compile('^_')` (matching underscore-prefixed names)",
      "Identifies that the regex matching logic is in the `visit_functiondef` method of `DocstringParameterChecker` in `pylint/extensions/docparams.py`",
      "Mentions that `no-docstring-rgx` is listed as a `GLOBAL_OPTION_PATTERN` in `pylint/utils/utils.py`",
      "Explains that the test's `.rc` config uses `^(?!__init__$)_` (negative lookahead) to exclude `__init__` from being skipped, so it gets checked for param docs while other private methods like `_private_method` do not"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/checkers/base/docstring_checker.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_init.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_init.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_init.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_init.MyClass\ntests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_init.MyClass.__init__:\n  My init docstring\n\ntests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_init.MyClass._private_method:\n  My private method",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific claims about file paths, line numbers, class names, and regex patterns that cannot be verified from the provided documentation alone. The documentation only shows docstrings for MyClass, __init__, and _private_method.",
      "The claim about the .rc file configuring `no-docstring-rgx=^(?!__init__$)_` is plausible given the test file name but not directly supported by the provided documentation.",
      "The explanation of the regex behavior (checking __init__ but not _private_method) is consistent with the documentation showing __init__ has a docstring 'My init docstring' and _private_method has 'My private method', but the specific mechanism is not documented here."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor adjustments to remove specific line numbers that may not match stripped code; core claims are all verifiable from source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_48",
    "repo": "pylint",
    "question": "How does the `no-docstring-rgx` option work in the docparams extension, and why would setting it to an empty string cause all functions to be skipped?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `no-docstring-rgx` configuration option in pylint's `DocstringParameterChecker` (in `pylint/extensions/docparams.py`) controls which functions are exempt from docstring parameter documentation checks. Here's how it works:\n\n**Configuration flow:**\n1. The `.rc` file sets `no-docstring-rgx=` (empty string) under `[BASIC]` section.\n2. The value is transformed by `_regex_transformer()` in `pylint/config/argument.py`, which calls `re.compile(value)`. An empty string becomes `re.compile(\"\")`, a compiled regex pattern that matches the empty string at the start of ANY string.\n3. The option itself is defined in `pylint/checkers/base/docstring_checker.py` with `type: \"regexp\"` and a default of `NO_REQUIRED_DOC_RGX = re.compile(\"^_\")`, which by default skips private/system methods starting with underscore.\n\n**Skipping mechanism in `visit_functiondef`:**\nWhen visiting a function definition, the checker reads `self.linter.config.no_docstring_rgx` and performs a truthiness check followed by `re.match()`:\n```python\nif no_docstring_rgx and re.match(no_docstring_rgx, node.name):\n    return\n```\nWhen `no-docstring-rgx=` (empty string compiled to `re.compile(\"\")`), the compiled pattern object is truthy, and `re.match(re.compile(\"\"), any_name)` always succeeds because the empty pattern matches zero characters at position 0 of any string. This means **every function is skipped**, including `__init__`, so no `missing-param-doc` errors are emitted even though `accept-no-param-doc=no`.\n\nThis same guard pattern is applied in four places: `visit_functiondef`, `visit_raise`, `visit_return`, and `visit_yield`.\n\n**Contrast with other configurations:**\n- `no-docstring-rgx=^$` (\"test all\"): matches only the empty string, so no real function name matches \u2192 all functions ARE checked.\n- `no-docstring-rgx=^(?!__init__$)_` (\"check init\"): matches underscore-prefixed names except `__init__`, so `__init__` IS checked but `_private_method` is not.\n- Default `^_`: all underscore-prefixed methods are skipped.\n\nThe `DocstringParameterChecker` extends `BaseChecker`, has `constructor_names = {\"__init__\", \"__new__\"}`, and `not_needed_param_in_docstring = {\"self\", \"cls\"}`. For constructors, `check_functiondef_params` additionally checks the class-level docstring via `check_single_constructor_params`, allowing parameters to be documented in either the class or `__init__` docstring.",
    "rubric": [
      "Explains that the empty string is compiled via `_regex_transformer` using `re.compile('')`, producing a pattern that matches zero characters at position 0 of any string",
      "Describes the guard condition `if no_docstring_rgx and re.match(no_docstring_rgx, node.name): return` and explains that the compiled pattern object is truthy even when compiled from an empty string",
      "Mentions that this guard appears in four visitor methods: visit_functiondef, visit_raise, visit_return, and visit_yield",
      "Notes the option is defined in the base docstring checker with type 'regexp' and default `re.compile('^_')` which normally skips underscore-prefixed names",
      "Contrasts with `^$` which only matches the empty string so no real function names match and all functions are checked",
      "Mentions that `DocstringParameterChecker` has `constructor_names = {'__init__', '__new__'}` and for constructors, `check_functiondef_params` also checks the class-level docstring via `check_single_constructor_params`"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/config/argument.py",
      "pylint/checkers/base/docstring_checker.py",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_none.rc",
      "tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_none.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/parameter/missing_param_doc_required_no_doc_rgx_check_none.py] tests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_none.MyClass\ntests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_none.MyClass:\n  test_no_docstring_rgx\n    Function that matches \"check no functions\" 'no-docstring-rgx' config option\n    No error message is emitted.\n\ntests.functional.ext.docparams.parameter.missing_param_doc_required_no_doc_rgx_check_none.MyClass.__init__:\n  My init docstring",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation is extremely minimal - it only shows a test file with a class MyClass containing a test_no_docstring_rgx method and an __init__ method. The answer makes extensive claims about internal implementation details (specific line numbers, code patterns, configuration flow) that cannot be verified from this documentation alone.",
      "The answer claims that when no-docstring-rgx is empty, 'No error message is emitted' for the test_no_docstring_rgx function - this is consistent with the documentation which states 'No error message is emitted'.",
      "The answer's claim about the file name 'missing_param_doc_required_no_doc_rgx_check_none' aligning with 'check none' behavior (skipping all functions) is consistent with the documentation's description that no error is emitted.",
      "Most of the detailed implementation claims (specific line numbers, regex transformer details, four places the guard is applied) go far beyond what can be verified from this documentation snippet and may be hallucinated details, though they are plausible."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor adjustments to remove unverifiable line numbers; all substantive claims are verifiable from stripped code logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_49",
    "repo": "pylint",
    "question": "What is the architecture of the docstring parameter checking system, including its class hierarchies, docstring type detection, and how property setters are handled?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `missing_raises_doc_Google.py` test file is a functional test for the `DocstringParameterChecker` class (in `pylint/extensions/docparams.py`), which is a pylint extension checker loaded via the `pylint.extensions.docparams` plugin. The architecture works as follows:\n\n1. **Checker class hierarchy**: `DocstringParameterChecker` extends `BaseChecker` and is registered via the `register()` function in `docparams.py`.\n\n2. **Docstring type hierarchy**: There are 5 docstring types in `_check_docs_utils.py`: `Docstring` (base), `SphinxDocstring(Docstring)`, `EpytextDocstring(SphinxDocstring)`, `GoogleDocstring(Docstring)`, and `NumpyDocstring(GoogleDocstring)`. They are stored in the `DOCSTRING_TYPES` dict mapping names ('sphinx', 'epytext', 'google', 'numpy', 'default') to classes.\n\n3. **Docstring auto-detection**: The `docstringify()` function tries all four concrete docstring types, calling `matching_sections()` on each, and picks the one with the highest count. For `GoogleDocstring`, `matching_sections()` counts matches across 5 regex patterns: `re_param_section`, `re_raise_section`, `re_returns_section`, `re_yields_section`, and `re_property_returns_line` (applied to just the first line).\n\n4. **Property setter handling**: In `visit_raise()`, when a function has no docstring, the checker calls `utils.get_setters_property(func_node)` to check if it's a property setter. If so, it falls back to the property getter's docstring. The `get_setters_property()` function uses structural pattern matching on decorators to identify `@name.setter` patterns, then looks up the corresponding property-decorated method in the class.\n\n5. **The Foo class test case**: The first `Foo` class tests that when a property setter raises `AttributeError()` but has no docstring, the checker falls back to the property getter's docstring (which only documents `RuntimeError`). The second `Foo` class tests when the setter has its own docstring.\n\n6. **Exception matching logic**: In `visit_raise()`, found exception class names are extracted from the docstring via `doc.exceptions()`, which parses the `Raises:` section using `_parse_section()` and `re_raise_line`. The comparison strips module paths (e.g., 're.error' \u2192 'error') and also checks ancestor classes. There's a special case where `re.error` is matched against `PatternError` (visible as `if found_exc == \"error\" and expected.name == \"PatternError\": break`).\n\n7. **Abstract method exemption**: In `_add_raise_message()`, `NotImplementedError` is automatically removed from missing exceptions if the function `is_abstract()`.\n\n8. **The `_split_multiple_exc_types()` function** splits comma-separated or 'or'-separated exception types using the regex `r'(\\s*,(?:\\s*or\\s)?\\s*|\\s+or\\s+)'`.\n\n9. **Google-specific regex patterns**: The `re_raise_section` pattern matches lines like `^([ ]*)Raises\\s*:\\s*$` followed by section content. The `re_raise_line` matches exception entries in the raises section. The `re_property_returns_line` pattern matches property return type declarations at the start of the first line.",
    "rubric": [
      "DocstringParameterChecker extends BaseChecker and is registered via a register() function in docparams.py",
      "There are 5 docstring types: Docstring (base), SphinxDocstring(Docstring), EpytextDocstring(SphinxDocstring), GoogleDocstring(Docstring), and NumpyDocstring(GoogleDocstring)",
      "DOCSTRING_TYPES dict maps names ('sphinx', 'epytext', 'google', 'numpy', 'default') to their respective classes",
      "The docstringify() function auto-detects docstring type by calling matching_sections() on each concrete type and picking the one with the highest count",
      "GoogleDocstring.matching_sections() counts matches across re_param_section, re_raise_section, re_returns_section, re_yields_section, and re_property_returns_line (applied to first line only)",
      "In visit_raise(), when a function has no docstring, get_setters_property() is called to check if it's a property setter, and if so falls back to the property getter's docstring",
      "get_setters_property() uses structural pattern matching on decorators to identify @name.setter patterns, then looks up the property-decorated method in the class",
      "Exception matching strips module paths (e.g., 're.error' \u2192 'error') and also checks ancestor classes, with a special case for re.error matching PatternError",
      "_add_raise_message() automatically removes NotImplementedError from missing exceptions if the function is_abstract()",
      "_split_multiple_exc_types() splits comma-separated or 'or'-separated exception types using regex"
    ],
    "key_files": [
      "tests/functional/ext/docparams/raise/missing_raises_doc_Google.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/raise/missing_raises_doc_Google.py] tests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo\ntests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo:\n  test_finds_missing_raises_from_setter_google\n    Example of a setter having missing raises documentation in\n    the Google style docstring of the property\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo.foo_method:\n  int: docstring\n\n        Include a \"Raises\" section so that this is identified\n        as a Google docstring and not a Numpy docstring.\n\n        Raises:\n            RuntimeError: Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive implementation details (line numbers, function names, regex patterns, class hierarchies) that are not present in the provided documentation chunk. These are plausible extrapolations from the broader codebase but cannot be verified from the given doc.",
      "The documentation only shows a class `Foo` with a method `test_finds_missing_raises_from_setter_google` and a property `foo_method` with a Google-style docstring documenting `RuntimeError`. The answer's claims about line numbers (148, 160, 166, 183) and specific error behaviors cannot be verified from this doc chunk.",
      "The answer's description of `foo_method` having a return type annotation `int: docstring` in the property's docstring is consistent with what the doc shows (`int: docstring` in the docstring content).",
      "The answer correctly identifies that the `Raises:` section documents `RuntimeError: Always`, which matches the documentation.",
      "The answer mentions a 'second Foo class (line 166)' which is not supported by the documentation - only one Foo class is shown."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with specific line numbers, Python version references, and documentation examples that aren't in the code itself.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_50",
    "repo": "pylint",
    "question": "Why are there two separate `Foo` classes in the Google-style raises documentation test, and how does the property setter fallback logic treat them differently?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The design rationale behind having two separate `Foo` test classes in `missing_raises_doc_Google.py` is to test two distinct behaviors of pylint's property setter raise documentation checking logic.\n\nIn the `visit_raise` method of `DocstringParameterChecker` (pylint/extensions/docparams.py), there is a conditional fallback mechanism:\n\n```python\nif not func_node.doc_node:\n    # If this is a property setter,\n    # the property should have the docstring instead.\n    property_ = utils.get_setters_property(func_node)\n    if property_:\n        func_node = property_\n```\n\nThis design creates two distinct scenarios:\n\n1. **First Foo (no setter docstring)**: When a property setter has NO docstring of its own, pylint falls back to check the property getter's docstring. The `get_setters_property` function (in `_check_docs_utils.py`) locates the corresponding `@property` method by finding the class attribute decorated with `@property`. The error is then reported on the **property getter** node (line 148), because `func_node` was reassigned to the property. This is why the expected output shows `missing-raises-doc:148:4:148:18:Foo.foo_method` for `AttributeError`.\n\n2. **Second Foo (setter has its own docstring)**: When the setter HAS its own docstring with a Raises section, `func_node.doc_node` is truthy, so the fallback is NOT triggered. The checker uses the setter's own docstring. Since that docstring documents `RuntimeError` but not `AttributeError`, the error is reported on the **setter** node itself (line 183).\n\nAdditionally, the `accept-no-raise-doc` option (default: `True`) means: if a docstring has NO Raises section at all (`doc.exceptions()` returns empty set), the checker returns early without reporting. But once a docstring includes even a partial Raises section, ALL actually-raised exceptions must be documented. The `matching_sections()` method of `GoogleDocstring` counts recognized sections (including Raises) \u2014 if it returns > 0, full exception checking proceeds.",
    "rubric": [
      "Explains that the first Foo has a setter with NO docstring, causing the `if not func_node.doc_node` branch to trigger and fall back to the property getter's docstring",
      "Explains that `get_setters_property` locates the @property-decorated method in the class and reassigns `func_node` to it, so the error is reported on the getter",
      "Explains that the second Foo has a setter WITH its own docstring containing a Raises section, so the fallback is NOT triggered and errors are reported on the setter itself",
      "Mentions that the setter's docstring documents RuntimeError but not AttributeError, so `missing-raises-doc` fires on the setter",
      "References the `accept-no-raise-doc` option or `doc.exceptions()` early return that allows docstrings with no Raises section to pass without error"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Google.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Google.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/raise/missing_raises_doc_Google.py] tests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo\ntests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo:\n  test_finds_missing_raises_from_setter_google_2\n    Example of a setter having missing raises documentation in\n    its own Google style docstring of the property.\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo.foo_method:\n  int: docstring ...\n\n        Raises:\n            RuntimeError: Always\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Google.Foo.foo_method:\n  setter docstring ...\n\n        Raises:\n            RuntimeError: Never",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive implementation details (specific line numbers, method names, code snippets) that are not present in the provided documentation chunk and cannot be verified from it alone.",
      "The documentation only shows a single Foo class with a description about 'missing raises documentation in its own Google style docstring of the property', but the answer claims there are 'two separate Foo test classes' - this cannot be confirmed from the documentation provided.",
      "The documentation shows the getter raises RuntimeError ('Always') and setter raises RuntimeError ('Never'), but the answer discusses AttributeError which is not mentioned in the documentation chunk.",
      "The specific line numbers (148, 183, 295-300, 49-70, 306) mentioned in the answer cannot be verified from the provided documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Almost all claims are verifiable from the stripped code logic, control flow, and test output files; only specific line number references needed minor adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_51",
    "repo": "pylint",
    "question": "Where does the data flow when a `raise` statement is checked against a Numpy-style docstring, especially for property setters without their own docstring?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "When a `raise` statement is encountered in the Numpy-style test file, the data and control flow proceeds as follows:\n\n1. **Entry point**: `DocstringParameterChecker.visit_raise()` in `pylint/extensions/docparams.py` is called for each `raise` node in the AST. It first gets the enclosing function via `node.frame()`.\n\n2. **Exception type resolution**: `possible_exc_types()` from `pylint/extensions/_check_docs_utils.py` determines the set of raised exception `ClassDef` nodes. For a direct `raise RuntimeError()`, it calls `_get_raise_target()` which uses pattern matching on `nodes.Call(func=nodes.Name() | nodes.Attribute())` and calls `utils.safe_infer(func)`. For bare `raise` inside an `except` handler, it walks up the parent chain to find the `ExceptHandler` and unpacks its type.\n\n3. **Setter-to-property redirection**: If `func_node.doc_node` is None (the setter has no docstring), `get_setters_property()` is called. It uses `get_setters_property_name()` which checks decorators via pattern matching `nodes.Attribute(attrname=\"setter\", expr=nodes.Name(name=name))`. It then looks up the class attributes and finds the one decorated with `@property`. This is why in the first `Foo` class, the setter's `raise AttributeError()` is checked against the *property's* Numpy docstring.\n\n4. **Docstring type detection**: `docstringify()` iterates over `SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, and `NumpyDocstring`, instantiating each and calling `matching_sections()`. The one with the highest number of matching sections wins. `NumpyDocstring` inherits `matching_sections()` from `GoogleDocstring`, which counts boolean matches across param, raise, returns, yields sections, and property return lines.\n\n5. **Numpy-specific parsing**: `NumpyDocstring` extends `GoogleDocstring`. Its `_re_section_template` differs: it matches a section header followed by an underline of dashes (`\\s*[-=]+\\s*?$`), while Google's template matches a header followed by a colon (`{0} \\s*:`). The `re_raise_section` is compiled from this template with `r\"Raises\"`. `NumpyDocstring._is_section_header()` checks if a line matches `r\"\\s*-+$\"`, while `GoogleDocstring._is_section_header()` always returns False. `NumpyDocstring.min_section_indent()` returns `len(section_match.group(1))` (the header's indentation), while `GoogleDocstring.min_section_indent()` returns `len(section_match.group(1)) + 1`.\n\n6. **Exception comparison**: Found exception class names are extracted by splitting on `.` (taking the last part), and each expected exception is matched either by exact name, by the `re.error`/`PatternError` Python 3.13 alias, or by checking ancestor classes via `expected.ancestors()`.\n\n7. **Message emission**: `_add_raise_message()` filters out `NotImplementedError` for abstract methods, then emits `W9006` (\"missing-raises-doc\") with confidence `HIGH` and sorted missing exception names.\n\nFor the first `Foo` class in the test, the property's docstring documents `RuntimeError` in its Raises section, but the setter (without its own docstring) raises `AttributeError`. Since the setter has no docstring, `func_node` is redirected to the property getter. The property's docstring only lists `RuntimeError`, so `AttributeError` is flagged as missing \u2014 the message is reported on the *property* node (`Foo.foo`), not the setter.",
    "rubric": [
      "Explains that visit_raise() in docparams.py is the entry point, gets the enclosing function via node.frame()",
      "Describes that possible_exc_types() resolves exception types, using _get_raise_target() with pattern matching on nodes.Call for direct raises, or walking parent chain for bare raises in except handlers",
      "Explains the setter-to-property redirection: when func_node.doc_node is None, get_setters_property() is called, which uses get_setters_property_name() to match the @X.setter decorator pattern, then finds the @property-decorated method in the class",
      "Describes docstringify() iterating over Sphinx, Epytext, Google, and Numpy docstring types, choosing the one with the most matching_sections()",
      "Explains how NumpyDocstring differs from GoogleDocstring: section template uses dash underlines instead of colons, _is_section_header checks for dashes, and min_section_indent returns group(1) length without the +1",
      "Describes how found exception names are compared: splitting on dots for class name, checking exact name match, re.error/PatternError alias, and ancestor class matching",
      "Explains that _add_raise_message filters NotImplementedError for abstract methods, then emits missing-raises-doc with HIGH confidence on the (potentially redirected) func_node"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.py] tests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo\ntests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo:\n  test_finds_missing_raises_from_setter_numpy\n    Example of a setter having missing raises documentation in\n    the Numpy style docstring of the property\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo.foo:\n  int: docstring\n\n        Include a \"Raises\" section so that this is identified\n        as a Numpy docstring and not a Google docstring.\n\n        Raises\n        ------\n        RuntimeError\n            Always",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor doc leakage from line number references that cannot be verified in stripped code; all substantive claims about logic and control flow are verifiable from code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_52",
    "repo": "pylint",
    "question": "How does the missing-raises-doc check work end-to-end for Numpy-style docstrings, including docstring type detection and property setter fallback?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The missing raises documentation check for Numpy-style docstrings in pylint works through a multi-layered system involving the `DocstringParameterChecker` in `pylint/extensions/docparams.py` and the `NumpyDocstring` class in `pylint/extensions/_check_docs_utils.py`.\n\n**How the system detects missing raises documentation:**\n\n1. **Visit pattern**: The `DocstringParameterChecker.visit_raise()` method is triggered for every `raise` statement in the AST. It first gets the enclosing function via `node.frame()`, then calls `utils.possible_exc_types(node)` to determine what exception types could be raised.\n\n2. **Property setter fallback**: If the function node has no docstring (`not func_node.doc_node`), the checker calls `utils.get_setters_property(func_node)` to check if this is a property setter. If so, it redirects `func_node` to the property getter, checking the getter's docstring instead. This is why the first `Foo` class in the test reports `missing-raises-doc` on the *property getter* at line 146 (for the `AttributeError` raised in the setter), because the setter has no docstring of its own.\n\n3. **Docstring type auto-detection**: The `docstringify()` function creates instances of all four docstring types (SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring) and picks the one with the highest `matching_sections()` count. For `GoogleDocstring`, `matching_sections()` sums up boolean matches across 5 regex patterns (param, raises, returns, yields, property_returns_line). `NumpyDocstring` inherits this method.\n\n4. **Numpy vs Google section template difference**: The key difference between Numpy and Google section detection is the section template regex. Google uses `^([ ]*)   {0} \\s*:   \\s*$` (requires a colon after the header), while Numpy uses `^([ ]*)   {0}   \\s*?$` followed by `\\s*     [-=]+   \\s*?$` (requires an underline of dashes/equals instead of a colon).\n\n5. **Exception extraction**: The `exceptions()` method (inherited from `GoogleDocstring`) parses the Raises section using `_parse_section(self.re_raise_section)`, then matches each entry against `re_raise_line`. For Numpy, `re_raise_line` matches a type on its own line followed by an optional description, while Google's expects `type : description` format.\n\n6. **Exception comparison logic**: After extracting documented exceptions, the checker compares them against expected exceptions. It extracts just the class name from dotted paths (e.g., `\"error\"` from `\"re.error\"`). It also checks ancestor classes (`any(found_exc == ancestor.name for ancestor in expected.ancestors())`), and has a special case for Python 3.13 where `re.error` aliases `re.PatternError`.\n\n7. **Abstract method handling**: In `_add_raise_message()`, if the node is abstract, `NotImplementedError` is automatically removed from the missing exceptions set.\n\n8. **Second `Foo` class behavior**: The second `Foo` class has a setter with its own docstring documenting `RuntimeError` but not `AttributeError`. Since the setter HAS a docstring, the fallback to the property getter's docstring does NOT occur. The checker evaluates the setter's own docstring and reports `AttributeError` as missing (line 185 in the expected output).\n\n9. **`accept-no-raise-doc` option**: Defaults to `True`. When `True` and `doc.exceptions()` returns an empty set, the checker returns early without flagging. This means functions with docstrings that have NO raises section at all are silently accepted by default.\n\n10. **`_parse_section` and `min_section_indent`**: For NumpyDocstring, `min_section_indent` returns `len(section_match.group(1))` (the indentation of the section header itself), while GoogleDocstring returns `len(section_match.group(1)) + 1` (one more than the header). NumpyDocstring also overrides `_is_section_header` to detect lines matching `\\s*-+$` (dashes), which terminates section parsing when a new section header is encountered.",
    "rubric": [
      "Explains that visit_raise() gets the enclosing function via node.frame() and calls possible_exc_types(node) to determine expected exceptions",
      "Describes the property setter fallback: if the function has no docstring, get_setters_property() is called to redirect checking to the property getter's docstring",
      "Explains how docstringify() auto-detects docstring type by instantiating all four types and selecting the one with the highest matching_sections() count",
      "Identifies the key regex difference: Numpy's section template uses an underline of dashes/equals instead of Google's colon after the header",
      "Describes how exceptions() uses _parse_section(re_raise_section) and matches entries against re_raise_line to extract exception types",
      "Explains the Numpy re_raise_line matches a type on its own line (no colon required) vs Google's type:description format",
      "Mentions the comparison logic: extracts class name from dotted paths, checks ancestor classes, and has a special case for re.error aliasing re.PatternError",
      "Notes that _add_raise_message removes NotImplementedError from missing exceptions if the function is abstract",
      "Explains accept-no-raise-doc option: when True and doc.exceptions() is empty, the checker returns early without flagging",
      "Describes NumpyDocstring.min_section_indent returning the section header's indentation level (vs GoogleDocstring adding 1) and _is_section_header detecting dash underlines to terminate section parsing"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.py",
      "tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/raise/missing_raises_doc_Numpy.py] tests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo\ntests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo:\n  test_finds_missing_raises_from_setter_numpy_2\n    Example of a setter having missing raises documentation in\n    its own Numpy style docstring of the property\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo.foo:\n  int: docstring ...\n\n        Raises\n        ------\n        RuntimeError\n            Always\n\ntests.functional.ext.docparams.raise.missing_raises_doc_Numpy.Foo.foo:\n  setter docstring ...\n\n        Raises\n        ------\n        RuntimeError\n            Never",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation is very limited - it only shows a test class with property getter/setter docstrings documenting RuntimeError in both, but the generated answer makes extensive claims about internal implementation details (line numbers, method names, regex patterns, etc.) that cannot be verified from the provided documentation chunk alone.",
      "The answer claims the setter in the second Foo class documents RuntimeError but not AttributeError, and that AttributeError is reported as missing at line 185. The documentation only shows the docstrings mentioning RuntimeError in both getter ('Always') and setter ('Never') sections - it doesn't explicitly mention AttributeError or line numbers for error reports.",
      "The answer's claim about the first Foo class reporting missing-raises-doc on the property getter at line 146 for an AttributeError raised in the setter at line 163 is not directly supported by the documentation chunk provided.",
      "Most of the detailed implementation claims (specific line numbers, regex patterns, method signatures) are plausible extrapolations from the broader pylint codebase but cannot be verified from this documentation chunk."
    ],
    "strip_verify_leakage": 0.04,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only specific line number references needed minor rewrites.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_53",
    "repo": "pylint",
    "question": "What mechanisms handle property return documentation in the docparams checker, and how does `check_functiondef_returns` differ from `visit_return` regarding property-specific checks?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `Foo` class in `tests/functional/ext/docparams/return/missing_return_doc_Google.py` contains a `@property`-decorated method `foo_method` (first definition). The stripped code shows it raises `RuntimeError()` and then has a `return 10` statement.\n\nProperty return documentation handling involves the `GoogleDocstring` class in `pylint/extensions/_check_docs_utils.py`. The key methods include:\n\n1. **`has_property_type()`**: Defined in the base `Docstring` class and overridden in `GoogleDocstring`, uses `re_property_returns_line` regex pattern to match property-style type declarations.\n\n2. **`has_property_returns()`**: Defined in the base `Docstring` class and overridden in `GoogleDocstring`, checks the first line of the docstring.\n\n3. **`matching_sections()`**: Defined in the base `Docstring` class (returning 0) and overridden in subclasses like `SphinxDocstring` to count matching documentation patterns.\n\nThe `DocstringParameterChecker` (in `pylint/extensions/docparams.py`) uses the `docstringify()` factory function to auto-detect the docstring type by comparing `matching_sections()` counts across SphinxDocstring, EpytextDocstring, GoogleDocstring, and NumpyDocstring classes, choosing the one with the highest score.\n\nThe `check_functiondef_returns` method only checks `has_returns()` and `has_rtype()` for the `redundant-returns-doc` message \u2014 NOT `has_property_returns()`/`has_property_type()`. This means property shorthand documentation doesn't trigger redundant-returns warnings even when the return is unreachable.\n\nThe `unreachable` warning (W0101) is defined in `BasicChecker` in `pylint/checkers/base/basic_checker.py`. In the test file, the `raise RuntimeError()` statement is followed by `return 10`, which would trigger the unreachable code detection for both Foo class property methods.\n\nThe `visit_return` method in `DocstringParameterChecker` checks for `has_property_returns()` and `has_property_type()` when the function is decorated with `@property`, using `checker_utils.decorated_with_property(func_node)`.",
    "rubric": [
      "Mentions that `has_property_returns()` and `has_property_type()` are defined in the base `Docstring` class (returning False) and overridden in subclasses like `SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, and `NumpyDocstring`",
      "Explains that `GoogleDocstring.has_property_returns()` checks the first line of the docstring to ensure it's not a known section directive (param, raise, returns, yields)",
      "Explains that `GoogleDocstring.has_property_type()` uses `re_property_returns_line` regex to match a type declaration on the first line",
      "Explains that `visit_return` checks both `has_property_returns()` and `has_property_type()` when the function is decorated with `@property` (using `decorated_with_property`)",
      "Explains that `check_functiondef_returns` only checks `has_returns()` and `has_rtype()` for the `redundant-returns-doc` message and does NOT check `has_property_returns()`/`has_property_type()`",
      "Mentions the `docstringify()` factory function that auto-detects docstring type by comparing `matching_sections()` counts across all docstring subclasses"
    ],
    "key_files": [
      "tests/functional/ext/docparams/return/missing_return_doc_Google.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/extensions/docparams.py",
      "pylint/checkers/base/basic_checker.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_Google.py] tests.functional.ext.docparams.return.missing_return_doc_Google.Foo\ntests.functional.ext.docparams.return.missing_return_doc_Google.Foo:\n  test_finds_property_return_type_google\n    Example of a property having return documentation in\n    a Google style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_Google.Foo.foo_method:\n  int: docstring ...\n\n        Raises:\n            RuntimeError: Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation is very minimal - it only shows that Foo class exists with a test_finds_property_return_type_google description, and that foo_method has 'int: docstring ...' format with a Raises: RuntimeError section. Most of the answer's detailed claims about line numbers, specific method implementations, regex patterns, and expected test messages are extrapolations beyond what the documentation provides.",
      "The documentation does support that foo_method uses 'int: docstring ...' format and has a 'Raises: RuntimeError: Always' section, which aligns with the answer's claims about the shorthand property return documentation format.",
      "The claim about 7 specific expected messages (3 redundant-returns-doc, 2 unreachable, 1 useless-param-doc, 1 useless-type-doc) cannot be verified from the documentation provided.",
      "The specific line numbers (81, 94, 95, 253, 617, 668, 679) are not in the documentation and cannot be verified."
    ],
    "strip_verify_leakage": 0.23,
    "strip_verify_summary": "Several claims about docstring content and .txt test expectations rely on stripped documentation; most code-structural claims are verifiable.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_54",
    "repo": "pylint",
    "question": "Why does the docstring parameter checker exempt abstract methods from return documentation requirements, and how does it handle NotImplementedError differently?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `missing_return_doc_Numpy.py` test file verifies that pylint's `DocstringParameterChecker` (in `pylint/extensions/docparams.py`) correctly handles several edge cases for Numpy-style docstrings related to return documentation, including testing that abstract methods\u2014whether declared with `@abc.abstractmethod` or by raising `NotImplementedError()`\u2014are properly exempted from return documentation checks.\n\n**How abstract methods are skipped for return checks:**\nIn `check_functiondef_returns()` of `docparams.py`, the checker explicitly returns early if `node.is_abstract()` is True. The `is_abstract()` method is called on astroid's `FunctionDef` node. The test file includes classes using both `@abc.abstractmethod` decorators and methods raising `NotImplementedError`, both of which are treated as abstract.\n\n**Why `NotImplementedError` is also tolerated for raises documentation:**\nIn `_add_raise_message()`, the checker removes `NotImplementedError` from the set of missing exceptions when `node.is_abstract()` returns True. This means methods that raise `NotImplementedError` as a stand-in for abstract behavior don't need to document it in their docstring.\n\n**Why the test has multiple `Foo` classes with the same name:**\nThe file contains multiple redefinitions of `Foo` classes and `my_func` functions, allowing multiple test scenarios in a single file.\n\n**How Numpy docstrings are detected:**\n`NumpyDocstring` extends `GoogleDocstring` and uses a distinct `_re_section_template` that requires underlines (`[-=]+`) beneath section headers (unlike Google style which uses colons). The `docstringify()` function (in `_check_docs_utils.py`) auto-detects the docstring type by trying all four formats (Sphinx, Epytext, Google, Numpy) and picking the one with the most `matching_sections()`. `NumpyDocstring` inherits `matching_sections()` from `GoogleDocstring`, which counts matches across param, raise, returns, yields, and property returns sections.\n\n**Why ignored argument names produce warnings for documentation:**\nThe test file includes parameters `_` and `_ignored` which match pylint's ignored-argument-names pattern. The test at line 164 verifies that documenting these ignored parameters (like `_` with type and `_ignored` with description) triggers `useless-type-doc` (W9020) and `useless-param-doc` (W9019) warnings, enforcing that deliberately ignored parameters should not be documented.",
    "rubric": [
      "Explains that check_functiondef_returns() returns early if node.is_abstract() is True, skipping redundant-returns-doc checks for abstract methods",
      "Explains that in _add_raise_message(), NotImplementedError is specifically removed from missing_exceptions when node.is_abstract() is True, so abstract methods don't need to document raising NotImplementedError",
      "Mentions that both @abc.abstractmethod decorated methods and methods raising NotImplementedError are treated as abstract (via astroid's is_abstract() method)",
      "Notes that check_functiondef_yields() also exempts abstract methods with the same is_abstract() check"
    ],
    "key_files": [
      "tests/functional/ext/docparams/return/missing_return_doc_Numpy.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_Numpy.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_Numpy.py] tests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo\ntests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo:\n  test_ignores_return_in_abstract_method_numpy_2\n    Example of a method documenting the return type that an\n    implementation should return.\n\ntests.functional.ext.docparams.return.missing_return_doc_Numpy.Foo.foo:\n  docstring ...\n\n        Parameters\n        ----------\n        arg : int\n            An argument.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is extremely minimal - just a docstring showing a class Foo with a method test_ignores_return_in_abstract_method_numpy_2 and a method foo with a Parameters section. The generated answer makes extensive claims about implementation details (line numbers, specific method names, regex patterns, etc.) that cannot be verified from the provided documentation.",
      "The answer claims specific line numbers (line 256, line 666, line 164) and specific implementation details that are not present in the documentation chunk provided.",
      "The answer discusses multiple Foo classes and function-redefined disables, which are not mentioned in the provided documentation.",
      "The answer discusses NumpyDocstring class inheritance and detection mechanisms not present in the documentation.",
      "However, the basic claim about abstract methods being exempted from return documentation checks is consistent with the method name 'test_ignores_return_in_abstract_method_numpy_2' shown in the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed design rationale justification and adjusted claims about default config patterns and comment-based pragmas.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_55",
    "repo": "pylint",
    "question": "Where is the `missing-return-type-doc` check triggered for `@property` decorated methods, and how does the logic differ for non-property methods?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `missing-return-type-doc` check for `Foo.foo_method` (the `@property` decorated method) is triggered in the `visit_return` method of `DocstringParameterChecker` class located in `pylint/extensions/docparams.py`. The check flow works as follows:\n\n1. The `visit_return` method is called when encountering a `return` statement (line 57: `return 10`)\n2. It checks `accept_no_return_doc` config (set to `no` in the `.rc` file)\n3. It calls `checker_utils.decorated_with_property(func_node)` (from `pylint/checkers/utils.py`) which detects the `@property` decorator\n4. For the return doc check: `doc.has_property_returns()` is called, and since `is_property` is True and the method returns True for the property's docstring, no `missing-return-doc` is emitted\n5. For the return type check: `doc.has_property_type()` is called \u2014 since it returns False for the given docstring, and the condition `not (doc.has_rtype() or (doc.has_property_type() and is_property))` is True, the `missing-return-type-doc` message (W9012) is triggered\n\nThe `docstringify` function (in `_check_docs_utils.py`) selects the best matching docstring parser by instantiating all four docstring types (Sphinx, Epytext, Google, Numpy) and choosing the one with the most `matching_sections()`. The Google style is detected because the `Raises:` section matches `re_raise_section`.\n\nThe key logic for the second `Foo` class's non-property `foo_method` at line 66 is that since it's not decorated with `@property`, the `is_property` flag is False in `visit_return`. This means that even if `doc.has_property_type()` would return True, the condition `doc.has_property_type() and is_property` evaluates to False, so property-style return type documentation is not recognized for non-property methods.",
    "rubric": [
      "Identifies that the check is triggered in the `visit_return` method of `DocstringParameterChecker` in `pylint/extensions/docparams.py`",
      "Explains that `checker_utils.decorated_with_property(func_node)` is used to detect the `@property` decorator",
      "Describes that `doc.has_property_type()` is called and its return value combined with `is_property` flag determines whether the message is emitted",
      "Explains the condition: `not (doc.has_rtype() or (doc.has_property_type() and is_property))` triggers the W9012 message",
      "Notes that for non-property methods, `is_property` is False so `doc.has_property_type() and is_property` always evaluates to False, meaning property-style return type docs are not recognized"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "pylint/checkers/utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Google.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Google.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_required_Google.py] tests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo\ntests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo:\n  test_finds_missing_property_return_type_google\n    Example of a property having return documentation in\n    a Google style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo.foo_method:\n  docstring ...\n\n        Raises:\n            RuntimeError: Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer provides extensive implementation details (specific line numbers, regex patterns, method names) that cannot be verified from the provided documentation chunk alone",
      "The documentation only shows that Foo has a method test_finds_missing_property_return_type_google with a Google style docstring example, and foo_method has a docstring with a Raises section - the detailed implementation flow is extrapolated from assumed knowledge of the codebase",
      "The claim about 'return 10' at line 57 and second Foo class's foo_method at line 66 cannot be verified from the documentation",
      "The documentation does show that foo_method has a 'Raises: RuntimeError: Always' section, which is consistent with the answer's claim about Google-style detection via the Raises section"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for implementation details of truncated GoogleDocstring methods and decorated_with_property internals.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_56",
    "repo": "pylint",
    "question": "How does the missing-return-doc check work end-to-end, from visiting a return node through docstring type detection to deciding whether to emit a warning?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The missing return documentation check for Google-style docstrings works through a multi-step algorithm in the `DocstringParameterChecker` and `GoogleDocstring` classes:\n\n## Algorithm Flow\n\n### 1. Entry Point: `visit_return`\nWhen a `return` statement is visited:\n- First checks if the return actually returns something (not `None`) via `returns_something()` \u2014 which checks if `return_node.value` is not `None` and not a `Const(None)`.\n- If `accept-no-return-doc` config is `True` (default), it returns early and skips the check. The test file uses `.rc` config setting `accept-no-return-doc=no` to activate the check.\n- Calls `utils.docstringify()` to auto-detect the docstring type.\n\n### 2. Docstring Type Detection: `docstringify()`\nIterates through `SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`, calls `matching_sections()` on each, and picks the one with the highest count. `GoogleDocstring.matching_sections()` checks 5 regex patterns: param section, raise section, returns section, yields section, and `re_property_returns_line` against the first line.\n\n### 3. Property vs Non-Property Distinction\nThe key logic is:\n```python\nis_property = checker_utils.decorated_with_property(func_node)\nif not (doc.has_returns() or (doc.has_property_returns() and is_property)):\n    self.add_message(\"missing-return-doc\", ...)\nif not (doc.has_rtype() or (doc.has_property_type() and is_property)):\n    self.add_message(\"missing-return-type-doc\", ...)\n```\nThe `has_property_returns()` and `has_property_type()` results are only considered if the function is actually decorated with `@property`. This is why the second `Foo` class's `foo_method` (non-property, line 66) gets both `missing-return-doc` and `missing-return-type-doc` \u2014 since the method isn't a property, property-style matches are ignored.\n\n### 4. Google-style Return Parsing\n- **`has_returns()`**: Parses the `Returns:` section using `_parse_section()`, then checks each entry against `re_returns_line` for a non-empty description (group 2).\n- **`has_rtype()`**: Same parsing, but checks for a non-empty type (group 1, the `type:` prefix).\n- **`has_property_returns()`**: Checks that the first line of the docstring is NOT a known section header (not Args/Raises/Returns/Yields). If it's not, the summary line itself is treated as return documentation.\n- **`has_property_type()`**: Checks if the first line matches `re_property_returns_line` \u2014 a regex that expects `type: description` format.\n\n### 5. Section Parsing: `_parse_section()`\nUses indentation-based parsing: finds the section header via regex, then reads lines with indentation >= the minimum section indent (header indent + 1). Groups consecutive lines into entries, splitting at minimum indentation level. Empty lines are skipped.\n\n### 6. The `func_node.returns` short-circuit\nBefore checking for missing return *type* documentation, the code checks if the function has a return type annotation (`func_node.returns`) or type comment (`func_node.type_comment_returns`). If either exists, the `missing-return-type-doc` message is suppressed.\n\n### Test Cases in the File (from expected output .txt)\n- Line 7: `missing-return-type-doc`\n- Line 16: `missing-return-doc`\n- Line 25: Both `missing-return-doc` and `missing-return-type-doc`\n- Line 50: `missing-return-type-doc` (property \u2014 property returns check passes but type check fails)\n- Line 66: Both `missing-return-doc` and `missing-return-type-doc` (non-property, so property-style type is ignored)",
    "rubric": [
      "Explains that visit_return first checks returns_something() to verify the return isn't None/empty",
      "Mentions the accept-no-return-doc config option that can short-circuit the check",
      "Describes docstringify() iterating through SphinxDocstring, EpytextDocstring, GoogleDocstring, NumpyDocstring and selecting the one with highest matching_sections() count",
      "Explains how GoogleDocstring.matching_sections() checks regex patterns (param, raise, returns, yields sections and property_returns_line on first line)",
      "Describes the property vs non-property distinction: has_property_returns()/has_property_type() results are only considered when the function is decorated with @property",
      "Explains how has_returns() parses the Returns section via _parse_section() and checks re_returns_line for a non-empty description (group 2)",
      "Explains how has_rtype() checks re_returns_line for a non-empty type (group 1)",
      "Explains has_property_returns() checks that the first line is not a known section header, and has_property_type() checks if first line matches re_property_returns_line regex",
      "Describes _parse_section() using indentation-based parsing to group lines into entries",
      "Mentions the func_node.returns/type_comment_returns check that suppresses missing-return-type-doc when a type annotation exists"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Google.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Google.rc",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Google.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_required_Google.py] tests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo\ntests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo:\n  test_ignores_non_property_return_type_google\n    Example of a class function trying to use `type` as return\n    documentation in a Google style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_required_Google.Foo.foo_method:\n  int: docstring ...\n\n        Raises:\n            RuntimeError: Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is extremely minimal \u2014 it only shows the class `Foo`, a method `test_ignores_non_property_return_type_google` with a brief description about using `type` as return documentation in Google style, and `foo_method` with docstring `int: docstring ...` and a `Raises: RuntimeError: Always` section. Most of the detailed algorithmic claims (line numbers, regex patterns, method names, config settings) cannot be verified from this documentation alone.",
      "The answer claims specific line numbers (335, 353-362, 636, 652, 668, 679, 772, 7, 16, 25, 50, 66) that cannot be verified from the provided documentation.",
      "The answer's description of `foo_method` having `int: docstring ...` as its docstring and getting both missing-return-doc and missing-return-type-doc because it's not a property is consistent with the documentation's description of `test_ignores_non_property_return_type_google` being about 'a class function trying to use `type` as return documentation in a Google style docstring'.",
      "The documentation shows `foo_method` has `Raises: RuntimeError: Always` which the answer doesn't explicitly mention but is consistent with the test scenario."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed where docstring content or PEP references are cited that aren't visible in stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_57",
    "repo": "pylint",
    "question": "What chain of checks causes a Numpy-style property docstring to trigger `missing-return-type-doc` but not `missing-return-doc` in the docparams extension?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The test file `missing_return_doc_required_Numpy.py` tests the `DocstringParameterChecker` (in `pylint/extensions/docparams.py`) for detecting missing return documentation in Numpy-style docstrings when `accept-no-return-doc=no` is configured.\n\nThe `Foo.foo_prop` property triggers `missing-return-type-doc` (W9012) because:\n\n1. The `.rc` file sets `accept-no-return-doc=no` and loads `pylint.extensions.docparams`.\n2. When `visit_return` is called in docparams.py, it calls `docstringify()` from `_check_docs_utils.py` which auto-detects the docstring type by trying SphinxDocstring, EpytextDocstring, GoogleDocstring, and NumpyDocstring (in that order) and picking the one with the highest `matching_sections()` count.\n3. For `foo_prop`, if the docstring contains a Numpy-style \"Raises\" section (with underline), `NumpyDocstring` is selected (its `re_raise_section` matches, giving `matching_sections() >= 1`).\n4. `NumpyDocstring` inherits from `GoogleDocstring`, which provides `has_property_returns()` (checks if the first line is NOT a known section directive) and `has_property_type()` (checks if the first line matches `re_property_returns_line`, a pattern requiring `type: description` format).\n5. If the first line of `foo_prop`'s docstring is not a section directive, `has_property_returns()` returns `True` (no `missing-return-doc` warning), but if it doesn't match the `re_property_returns_line` regex pattern `^{re_multiple_type}:`, then `has_property_type()` returns `False`.\n6. Since `foo_prop` has no return type annotation (`func_node.returns` is None) and `has_property_type()` is False, the checker emits `missing-return-type-doc`.\n\nThe `_first_line()` helper method extracts the first line by doing `self.doc.lstrip().split(\"\\n\", 1)[0]`. The `decorated_with_property()` function in `pylint/checkers/utils.py` is used to detect whether a function is a property.\n\nThe `NumpyDocstring` class uses a different `_re_section_template` than `GoogleDocstring`: it requires the section header to be on its own line followed by a line of dashes/equals (`\\s*[-=]+\\s*?$`), while Google style requires the header followed by a colon (`{0} \\s*:\\s*$`).",
    "rubric": [
      "Explains that `docstringify()` auto-detects docstring type by trying SphinxDocstring, EpytextDocstring, GoogleDocstring, and NumpyDocstring and selecting the one with the highest `matching_sections()` count",
      "Identifies that NumpyDocstring is selected because its `re_raise_section` (or other section regex) matches, giving `matching_sections() >= 1`",
      "Explains that NumpyDocstring inherits `has_property_returns()` from GoogleDocstring, which checks if the first line is NOT a known section directive (param/raise/returns/yields)",
      "Explains that `has_property_returns()` returns True (so no `missing-return-doc` warning) because the first line (e.g. 'docstring ...') is not a section header",
      "Explains that `has_property_type()` checks if the first line matches `re_property_returns_line` (pattern `^{re_multiple_type}:`) and returns False when the first line doesn't have the `type: description` format",
      "Notes that `visit_return` in docparams.py checks `decorated_with_property()` and uses the is_property flag to gate the `has_property_returns()` and `has_property_type()` calls",
      "Mentions that `accept-no-return-doc=no` configuration must be set (otherwise `visit_return` exits early)",
      "Notes that if `func_node.returns` (return type annotation) is present, the `missing-return-type-doc` check is skipped"
    ],
    "key_files": [
      "tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.py] tests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo\ntests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo:\n  test_finds_missing_property_return_type_numpy\n    Example of a property having return documentation in\n    a numpy style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo.foo_prop:\n  docstring ...\n\n        Raises\n        ------\n        RuntimeError\n            Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most of the detailed claims about line numbers, internal method implementations, regex patterns, and specific code flow (e.g., line 335 of docparams.py, line 759 for _first_line(), line 805 for decorated_with_property(), line 812 for NumpyDocstring) are not verifiable from the provided documentation and are extrapolations or knowledge from the broader codebase.",
      "The core claim that foo_prop triggers missing-return-type-doc (W9012) is plausible but not explicitly stated in the documentation snippet.",
      "The documentation does confirm: (1) foo_prop has a Numpy-style docstring with a Raises section and 'docstring ...' as content, (2) the test is about finding missing property return type in numpy style, (3) the class is in the missing_return_doc_required_Numpy.py test file.",
      "The claim about .rc file settings (accept-no-return-doc=no) is not mentioned in the documentation but is a plausible extrapolation given the file name contains 'required'.",
      "The description of the docstring matching/selection process and the has_property_returns/has_property_type logic is plausible code-level detail but entirely unsupported by the provided documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor rewrites needed for line number references and docstring content assumptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_58",
    "repo": "pylint",
    "question": "Why does `visit_return` in the docparams checker gate `has_property_returns()` and `has_property_type()` behind `is_property`, and how do the early-exit checks avoid unnecessary docstring parsing?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `visit_return` method in `DocstringParameterChecker` (pylint/extensions/docparams.py) uses multiple early-exit checks that avoid docstring parsing when it's unnecessary:\n\n1. **`accept-no-return-doc` early exit**: When this option is `True` (the default), the entire method exits immediately without calling `docstringify()` or `decorated_with_property()`. This avoids the `docstringify()` function which iterates through all 4 docstring type classes (Sphinx, Epytext, Google, Numpy), instantiates each, and calls `matching_sections()` with multiple regex searches per instance to determine the best match.\n\n2. **`is_property` gate**: The property-style return documentation (`type: description` on the first line) is ONLY accepted when the method is decorated with `@property`. This is why `foo_method` at line 68 (which uses `int: docstring ...` format) gets both `missing-return-doc` and `missing-return-type-doc` \u2014 even though `has_property_type()` would return True for that docstring, the checker gates it behind `is_property`.\n\n3. **`func_node.returns` short-circuit**: If a function has a return type annotation (e.g., `-> int`), the checker skips the `missing-return-type-doc` message entirely (via `if func_node.returns or func_node.type_comment_returns: return`). This is why the third `Foo.foo_method` (line 87, with `-> int`) only gets `missing-return-doc` but not `missing-return-type-doc`.\n\nThe split between `check_functiondef_returns` (checking for *redundant* docs, called once per function via `visit_functiondef`) and `visit_return` (checking for *missing* docs, called per return statement) means the property checks only happen when there's an actual return value, not on every function definition.",
    "rubric": [
      "Explains that `accept-no-return-doc` being True causes immediate return before `docstringify()` is ever called, avoiding the expensive iteration over all 4 docstring type classes (Sphinx, Epytext, Google, Numpy) and their regex-based `matching_sections()` calls",
      "Explains that `has_property_returns()` and `has_property_type()` results are only used when `is_property` is True (i.e., `decorated_with_property`), so non-property methods using `type: description` format will get missing-return-doc and missing-return-type-doc warnings",
      "Explains that `func_node.returns` (return type annotation) short-circuits the `missing-return-type-doc` check, which is why an annotated non-property method only gets `missing-return-doc` but not `missing-return-type-doc`",
      "Notes the distinction between `check_functiondef_returns` (checks for redundant docs per function definition) and `visit_return` (checks for missing docs per return statement)"
    ],
    "key_files": [
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.txt"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_required_Numpy.py] tests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo\ntests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo:\n  test_ignores_non_property_return_type_numpy\n    Example of a class function trying to use `type` as return\n    documentation in a numpy style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_required_Numpy.Foo.foo_method:\n  int: docstring ...\n\n        Raises\n        ------\n        RuntimeError\n            Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, method names, regex patterns, internal logic) that are not present in or verifiable from the provided documentation chunk.",
      "The documentation only shows a test file's docstring structure with a class `Foo`, a method `test_ignores_non_property_return_type_numpy`, and `foo_method` with `int: docstring ...` format and a `Raises` section.",
      "The claim about `foo_method` getting both `missing-return-doc` and `missing-return-type-doc` is not directly contradicted but also not verifiable from the provided doc chunk.",
      "The claim about a third `Foo.foo_method` at line 87 with `-> int` annotation is not mentioned in the documentation.",
      "Most of the answer appears to be based on knowledge of the pylint source code rather than the provided documentation chunk, making it largely unverifiable against the given doc."
    ],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from code structure and test outputs; a few design rationale statements ('performance optimization', 'to avoid false negatives') rely on reasoning not present in stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_60",
    "repo": "pylint",
    "question": "How does the TypingChecker suppress `deprecated-typing-alias` messages for `typing.Callable` when a broken callable location is detected?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `typing_broken_callable_deprecated_alias` test verifies that pylint's `TypingChecker` (in `pylint/extensions/typing.py`) suppresses the `deprecated-typing-alias` message for ALL uses of `typing.Callable` in a module when at least one usage would create a broken `collections.abc.Callable` instance inside `Optional` or `Union` (Python bug https://bugs.python.org/issue42965).\n\nHere's how the mechanism works:\n\n1. **Detection phase** (`_check_for_typing_alias` method): When the checker encounters a `typing.Callable` usage on Python 3.9+, it calls `_broken_callable_location(node)`. If that returns `True`, it sets the module-level flag `self._found_broken_callable_location = True`. The message is still appended to `_deprecated_typing_alias_msgs` regardless.\n\n2. **Location check** (`_broken_callable_location` method): This method determines if a Callable node would be broken if replaced with `collections.abc.Callable`. It returns `True` only when ALL of these conditions are met:\n   - The node is NOT inside a `TYPE_CHECKING` block (checked via `in_type_checking_block`)\n   - Postponed evaluation is NOT enabled OR the node is not in a type annotation context\n   - The Callable is subscripted with a list of arguments as the first parameter (pattern: `nodes.Subscript(slice=nodes.Tuple(elts=[nodes.List(), _]))`), e.g., `Callable[[int], None]` (not `Callable[..., None]`)\n   - The Callable is nested inside an `Optional` or `Union` type (verified by inferring the parent subscript's value to check if its qname is `typing.Optional` or `typing.Union`, or if it's an instance of `typing._SpecialForm`)\n\n3. **Emission phase** (`leave_module` method): After the entire module is processed, the checker iterates through all collected `_deprecated_typing_alias_msgs`. If `_found_broken_callable_location` is `True`, it skips ALL messages where `msg.qname == \"typing.Callable\"` \u2014 not just the broken one. This is a module-wide suppression: if any single Callable usage would be broken, no Callable deprecation warnings are emitted for the entire module.\n\n4. **Cleanup**: After processing, the flag and message lists are cleared (`_found_broken_callable_location = False`, lists are cleared).\n\nThe test file has no corresponding `.txt` expected output file, confirming that no `deprecated-typing-alias` messages should be emitted. The test uses `py-version=3.9` (from the `.rc` file). This means usages like `Alias5 = list[Callable[[int], None]]` (not inside Optional/Union) and `Alias6 = Callable[[int], None]` (not subscripted in a broken way) are still suppressed because the module-wide flag is set.\n\nThe `_should_check_callable` flag (for the `broken-collections-callable` message) is set when `py_version < (3, 9, 2)`, while `_should_check_typing_alias` is enabled when `py_version >= (3, 9)`. So for Python 3.9.0 and 3.9.1, both checks are active simultaneously.",
    "rubric": [
      "Explains that `_broken_callable_location` checks if Callable is subscripted with a list as first arg (e.g., Callable[[int], None]) AND nested inside Optional or Union",
      "Mentions that `_broken_callable_location` also verifies the node is NOT in a TYPE_CHECKING block and that postponed evaluation doesn't apply",
      "Explains that when a broken location is found, the module-level flag `_found_broken_callable_location` is set to True",
      "Explains that in `leave_module`, ALL `deprecated-typing-alias` messages with qname `typing.Callable` are skipped (not just the broken one) \u2014 module-wide suppression",
      "Notes that messages are collected in `_deprecated_typing_alias_msgs` during traversal and only emitted at module exit",
      "Mentions that the flag and message lists are cleared after processing each module"
    ],
    "key_files": [
      "pylint/extensions/typing.py",
      "tests/functional/ext/typing/typing_broken_callable_deprecated_alias.py",
      "tests/functional/ext/typing/typing_broken_callable_deprecated_alias.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/typing/typing_broken_callable_deprecated_alias.py] tests.functional.ext.typing.typing_broken_callable_deprecated_alias\n'collections.abc.Callable' is broken inside Optional and Union types for Python 3.9.0\nhttps://bugs.python.org/issue42965\n\nUse 'typing.Callable' instead.\n\nDon't emit 'deprecated-typing-alias' for 'Callable' if at least one replacement\nwould create broken instances.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Minor changes to remove line number references; all substantive claims are verifiable from stripped code logic.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_59",
    "repo": "pylint",
    "question": "Where is the logic for checking missing return documentation in properties implemented, and how does it differ from regular function return checks?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `DocstringParameterChecker` class that checks for missing return documentation in Sphinx-style docstrings is located in `pylint/extensions/docparams.py`. It extends `BaseChecker` and is registered with the checker name `\"parameter_documentation\"`. The Sphinx docstring parsing logic used by this checker is in the `SphinxDocstring` class, defined in `pylint/extensions/_check_docs_utils.py`.\n\nSpecifically for the `Foo.foo` property test case in `missing_return_doc_required_Sphinx.py`:\n\n1. The test file is configured via `.rc` file with `load-plugins = pylint.extensions.docparams` and `accept-no-return-doc=no` (the key setting that makes missing return docs an error).\n\n2. When a `return` node is visited (`visit_return` in `docparams.py`), the checker calls `checker_utils.decorated_with_property(func_node)` (from `pylint/checkers/utils.py`) to determine if the function is a property.\n\n3. For properties, the checker uses `doc.has_property_returns()` and `doc.has_property_type()` instead of the standard `doc.has_returns()` and `doc.has_rtype()`. In `SphinxDocstring.has_property_returns()`, it checks whether the docstring's first line does NOT start with `:` (i.e., `return not self.doc.lstrip().startswith(':')`). For the `Foo.foo` property, the docstring starts with `\"docstring ...\"` (not `:`), so `has_property_returns()` returns `True`, and only `missing-return-type-doc` (W9012) is emitted, not `missing-return-doc`.\n\n4. `SphinxDocstring.has_property_type()` checks for the `:type:` Sphinx directive pattern (`re_property_type_in_docstring` which matches `:type:` followed by a type). The `Foo.foo` docstring lacks this, so `missing-return-type-doc` is emitted.\n\n5. The `docstringify()` function in `_check_docs_utils.py` auto-detects the docstring format by instantiating all four docstring types (`SphinxDocstring`, `EpytextDocstring`, `GoogleDocstring`, `NumpyDocstring`) and selecting the one with the highest `matching_sections()` count. For Sphinx, `matching_sections()` counts matches across 5 regex patterns: `re_param_in_docstring`, `re_raise_in_docstring`, `re_rtype_in_docstring`, `re_returns_in_docstring`, and `re_property_type_in_docstring`.\n\n6. The second `Foo` class's method `test_ignores_non_property_return_type_sphinx` uses `:type: int` (a property-type directive) but is NOT decorated with `@property`. Since `is_property` is `False`, the property-specific checks are skipped, and both `missing-return-doc` and `missing-return-type-doc` are emitted.",
    "rubric": [
      "Identifies DocstringParameterChecker in pylint/extensions/docparams.py as the checker class",
      "Identifies SphinxDocstring in pylint/extensions/_check_docs_utils.py as the Sphinx docstring parsing class",
      "Explains that visit_return uses checker_utils.decorated_with_property to determine if a function is a property",
      "Explains that for properties, has_property_returns() and has_property_type() are used instead of has_returns() and has_rtype()",
      "Explains that SphinxDocstring.has_property_returns() checks whether the docstring's first line does NOT start with ':' (meaning the summary line serves as the return description)",
      "Explains that SphinxDocstring.has_property_type() checks for the ':type:' directive pattern in the docstring"
    ],
    "key_files": [
      "tests/functional/ext/docparams/return/missing_return_doc_required_Sphinx.py",
      "pylint/extensions/docparams.py",
      "pylint/extensions/_check_docs_utils.py",
      "tests/functional/ext/docparams/return/missing_return_doc_required_Sphinx.rc"
    ],
    "source_doc": "[docstring: tests/functional/ext/docparams/return/missing_return_doc_required_Sphinx.py] tests.functional.ext.docparams.return.missing_return_doc_required_Sphinx.Foo\ntests.functional.ext.docparams.return.missing_return_doc_required_Sphinx.Foo:\n  test_finds_missing_property_return_type_sphinx\n    Example of a property having missing return documentation in\n    a Sphinx style docstring\n\ntests.functional.ext.docparams.return.missing_return_doc_required_Sphinx.Foo.foo:\n  docstring ...\n\n        :raises RuntimeError: Always",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation chunk is extremely minimal - it only shows class/method names and brief docstrings. The generated answer contains extensive implementation details (line numbers, specific regex patterns, method names, file paths) that cannot be verified from the provided documentation.",
      "The answer claims a second Foo class with method 'test_ignores_non_property_return_type_sphinx' which is not mentioned in the documentation chunk.",
      "The answer provides specific line numbers (e.g., line 334, 335, 450, 458, 805, 259) that cannot be verified from the documentation.",
      "The documentation mentions 'Foo.foo' with docstring containing ':raises RuntimeError: Always', but the answer claims the docstring starts with 'docstring ...' - while the doc does show 'docstring ...' it also shows ':raises RuntimeError: Always' as part of it, which the answer doesn't fully address.",
      "Most of the answer's content is extrapolation about pylint internals that goes far beyond what's in the documentation chunk, making verification impossible."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only specific line numbers and one minor design rationale phrase needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_61",
    "repo": "pylint",
    "question": "What is the architecture of the `broken-noreturn` check, and how does `from __future__ import annotations` change its behavior?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `typing_broken_noreturn_future_import.py` test file validates the behavior of the `broken-noreturn` (E6004) message from the `TypingChecker` class in `pylint/extensions/typing.py`. The key architectural insight is how `from __future__ import annotations` changes the behavior of the checker.\n\n**Architecture:**\n\n1. **`TypingChecker`** (in `pylint/extensions/typing.py`) extends `BaseChecker` (from `pylint/checkers/base_checker.py`), which itself extends `_ArgumentsProvider`. It's registered as a plugin via the `register()` function.\n\n2. **Version gating**: The `_should_check_noreturn` flag is set in the `open()` method to `py_version < (3, 7, 2)`. This means the broken-noreturn check only fires for Python versions below 3.7.2 (i.e., 3.7.0 and 3.7.1).\n\n3. **The `_check_broken_noreturn` method** is the core logic:\n   - First checks if the node's parent is a `nodes.BaseContainer` (the NoReturn is inside a compound type)\n   - Then checks two exemptions: `in_type_checking_block(node)` OR (`_postponed_evaluation_enabled` AND `is_node_in_type_annotation_context(node)`)\n   - When `from __future__ import annotations` is present, `_postponed_evaluation_enabled` is True, so nodes in a type annotation context (function return annotations, AnnAssign annotations, argument annotations) are exempt\n   - For nodes NOT in a type annotation context (like module-level alias assignments like `Alias2 = Callable[..., NoReturn]`), the error is still emitted\n\n4. **`visit_module`** sets `_postponed_evaluation_enabled` to `True` if either `_py314_plus` (Python 3.14+) or `is_postponed_evaluation_enabled(node)` which checks `\"annotations\" in module.future_imports`.\n\n5. **`is_node_in_type_annotation_context`** (in `pylint/checkers/utils.py`) checks if the node is part of an `AnnAssign` annotation, function `Arguments` annotations, or `FunctionDef` return type annotation.\n\n6. **`TYPING_NORETURN`** (in `pylint/constants.py`) is a `frozenset` containing `\"typing.NoReturn\"` and `\"typing_extensions.NoReturn\"`.\n\n7. The checker also handles the case where in Python 3.7-3.8, `NoReturn` is an alias of `typing._SpecialForm` (detected via `bases.BaseInstance` with `_proxied.qname() == \"typing._SpecialForm\"`).\n\n**Key behavioral difference with `from __future__ import annotations`:**\n- Without the future import: 4 errors are emitted (func2, func4, func5, Alias2)\n- With the future import: Only 1 error is emitted (Alias2), because func2/func4/func5 are in type annotation contexts (function return annotations) which are exempt when postponed evaluation is enabled",
    "rubric": [
      "Identifies that TypingChecker extends BaseChecker and is registered via a register() function",
      "Explains that `_should_check_noreturn` is gated on py_version < (3, 7, 2), meaning the check only fires for Python 3.7.0/3.7.1",
      "Describes that `_check_broken_noreturn` first checks if the node's parent is a BaseContainer (compound type)",
      "Explains the two exemptions: `in_type_checking_block(node)` or (`_postponed_evaluation_enabled` AND `is_node_in_type_annotation_context(node)`)",
      "Explains that `visit_module` sets `_postponed_evaluation_enabled` to True if `_py314_plus` or `is_postponed_evaluation_enabled(node)` (which checks for 'annotations' in future_imports)",
      "Describes what `is_node_in_type_annotation_context` checks: AnnAssign annotations, function Arguments annotations, or FunctionDef return annotations",
      "Notes that TYPING_NORETURN is a frozenset containing 'typing.NoReturn' and 'typing_extensions.NoReturn'",
      "Explains the behavioral difference: without future import 4 errors are emitted (func2, func4, func5, Alias2), with the future import only 1 error (Alias2) because function return annotations are exempt when postponed evaluation is enabled"
    ],
    "key_files": [
      "pylint/extensions/typing.py",
      "tests/functional/ext/typing/typing_broken_noreturn_future_import.py",
      "pylint/checkers/utils.py",
      "pylint/constants.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/typing/typing_broken_noreturn_future_import.py] tests.functional.ext.typing.typing_broken_noreturn_future_import\n'typing.NoReturn' is broken inside compound types for Python 3.7.0\nhttps://bugs.python.org/issue34921\n\nIf no runtime introspection is required, use string annotations instead.\n\nWith 'from __future__ import annotations', only emit errors for nodes\nnot in a type annotation context.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in PEP reference and line numbers; almost all claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_62",
    "repo": "pylint",
    "question": "Why does the `broken-noreturn` check only activate when `py-version` is below 3.7.2, and how is it suppressed?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `typing_broken_noreturn_py372.py` test file verifies that the `broken-noreturn` message (E6004) is NOT emitted when `py-version` is set to `3.7.2` or higher. The design rationale is based on a specific Python bug (https://bugs.python.org/issue34921) where `typing.NoReturn` inside compound types (like `Union` or `Callable`) was broken only in Python 3.7.0 and 3.7.1, as indicated by the E6004 message definition in the code.\n\nThe version-gating logic is implemented in `pylint/extensions/typing.py` in the `TypingChecker.open()` method:\n```python\nself._should_check_noreturn = py_version < (3, 7, 2)\n```\n\nThis means the check is only active when the configured `py-version` is strictly less than `(3, 7, 2)`. When `py-version=3.7.2` (as in the `.rc` file for this test), the condition is `False`, so `_check_broken_noreturn` is never invoked, and no `broken-noreturn` messages are emitted. The test file has no corresponding `.txt` file with expected messages, confirming zero errors are expected.\n\nThe `_check_broken_noreturn` method has additional suppression logic. The code imports `in_type_checking_block`, `is_node_in_type_annotation_context`, and `is_postponed_evaluation_enabled`, suggesting these are used for suppression conditions. The `TYPING_NORETURN` frozenset constant (defined in `pylint/constants.py`) contains both `typing.NoReturn` and `typing_extensions.NoReturn`, and is imported in the typing checker module for use in the broken-noreturn check.",
    "rubric": [
      "Explains the check targets a Python bug (https://bugs.python.org/issue34921) where typing.NoReturn in compound types was broken in Python 3.7.0 and 3.7.1 only",
      "Identifies the version-gating logic: `self._should_check_noreturn = py_version < (3, 7, 2)` in the `open()` method",
      "Mentions that the check is suppressed inside TYPE_CHECKING blocks (via `in_type_checking_block`)",
      "Mentions that the check is suppressed when postponed evaluation is enabled and the node is in a type annotation context",
      "Notes that `TYPING_NORETURN` includes both `typing.NoReturn` and `typing_extensions.NoReturn`"
    ],
    "key_files": [
      "pylint/extensions/typing.py",
      "tests/functional/ext/typing/typing_broken_noreturn_py372.py",
      "tests/functional/ext/typing/typing_broken_noreturn_py372.rc",
      "pylint/constants.py"
    ],
    "source_doc": "[docstring: tests/functional/ext/typing/typing_broken_noreturn_py372.py] tests.functional.ext.typing.typing_broken_noreturn_py372\n'typing.NoReturn' is broken inside compound types for Python 3.7.0\nhttps://bugs.python.org/issue34921\n\nIf no runtime introspection is required, use string annotations instead.\n\nDon't emit errors if py-version set to >= 3.7.2.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (line numbers, method names, specific logic) that cannot be verified from the documentation alone - these are plausible but unverifiable extrapolations",
      "The answer claims the error code is E6004 - this cannot be verified from the documentation",
      "The answer claims specific implementation details about `_check_broken_noreturn` method, `in_type_checking_block()`, `TYPING_NORETURN` frozenset, etc. - none of these are in the documentation",
      "The answer states 'The test file has no corresponding .txt file with expected messages' - plausible but unverifiable from the doc"
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; one claim about a code comment is doc-leakage, and some claims about the truncated method body need slight rewording.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_63",
    "repo": "pylint",
    "question": "Where does the unused-argument detection flow through for class methods, and what causes certain arguments to be skipped or reported?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The data flow for unused-argument detection in the `AAAA` class methods in `tests/functional/u/unused/unused_argument.py` goes through the `VariablesChecker` class in `pylint/checkers/variables.py`. Here's the detailed flow:\n\n1. **Entry point**: `leave_functiondef()` (line 1546) is called when exiting each method. It pops `not_consumed` locals and iterates over them, calling `_check_is_unused()` for each unused name.\n\n2. **Name filtering**: `_check_is_unused()` (line 2768) first calls `_is_name_ignored()` (line 2868), which matches argument names against `self.linter.config.ignored_argument_names`. The default regex `IGNORED_ARGUMENT_NAMES` at line 50 is `re.compile(\"_.*|^ignored_|^unused_\")`. This is why:\n   - `_not_used` in `test_unused` is silently ignored (matches `_.*`)\n   - `*unused_args` and `**unused_kwargs` in `AAAA.__init__` are silently ignored (match `^unused_`)\n   - `ignored_second` in `test_prefixed_with_ignored` is silently ignored (matches `^ignored_`)\n\n3. **Abstract method skip**: In `leave_functiondef()` (line 1570), `Base.inherited` is skipped entirely because `node.is_abstract()` returns True \u2014 the method body consists only of `raise NotImplementedError`, and `utils.is_error()` (line 1565, defined at `utils.py:277`) returns True for functions whose body is a single `Raise` statement.\n\n4. **Overridden method skip**: For `Sub.inherited`, `_check_unused_arguments()` (line 2884) calls `overridden_method(klass, node.name)` (defined at `utils.py:2316`) which finds `Base.inherited` as an ancestor method. Since `aab` and `aac` appear in `overridden.argnames()`, no warning is emitted for those unused arguments.\n\n5. **Confidence assignment**: In `_check_unused_arguments()` (line 2892-2899), methods inside classes get `INFERENCE` confidence (when `utils.has_known_bases(klass)` is True), while standalone functions get `HIGH` confidence. This explains why `test_unused` shows `HIGH` while `AAAA.method`, `AAAA.selected`, and `Sub.newmethod` show `INFERENCE`.\n\n6. **`AAAA.selected` classmethod**: `cls` is the first argument and is skipped (line 2903: `name == argnames[0]` check for non-static methods). But `*args` and `**kwargs` are unused and not matched by the ignored_argument_names regex, so both trigger unused-argument with `INFERENCE` confidence. Their column info in the .txt file is `0:None` (unlike regular args which have specific column ranges).\n\n7. **`AAAA.using_inner_function`**: The `etype` and `self` arguments are consumed because they're used as default parameter values in the inner function definition (`etype=etype`, `req=self`, `rset=rset`). The `size` parameter is used on line 80 in `[('A',)]*size`. Thus no warnings are emitted.\n\n8. **`__new__` special handling** (line 2804-2813): When checking `__new__`, the code iterates through sibling methods looking for `__init__`. If `__init__` exists (as in `TestClassWithInitAndNew`), unused arguments in `__new__` are ignored. If only `__new__` exists (as in `TestClassWithOnlyNew`), warnings are emitted.\n\nThe checker class is `VariablesChecker(BaseChecker)` with `name = \"variables\"`, and the message code for unused-argument is `W0613`.",
    "rubric": [
      "Identifies `leave_functiondef()` in `VariablesChecker` as the entry point that pops `not_consumed` locals and iterates over them calling `_check_is_unused()`",
      "Explains that `_is_name_ignored()` matches argument names against `ignored_argument_names` regex (default `_.*|^ignored_|^unused_`), which silently skips names like `_not_used`, `unused_args`, `unused_kwargs`, and `ignored_second`",
      "Explains that abstract methods are skipped via `node.is_abstract()` check, and `utils.is_error()` returns True for functions whose body is only a Raise statement",
      "Explains that overridden methods are handled in `_check_unused_arguments()` via `overridden_method()` which finds ancestor methods, so arguments present in the ancestor's argnames are not warned about",
      "Explains the confidence level logic: methods in classes get `INFERENCE` confidence (when bases are known), standalone functions get `HIGH` confidence",
      "Explains that the first argument of non-static methods (e.g., `self`, `cls`) is skipped via the `name == argnames[0]` check in `_check_unused_arguments()`",
      "Explains the `__new__` special handling: if `__init__` exists as a sibling method, unused arguments in `__new__` are ignored; otherwise warnings are emitted"
    ],
    "key_files": [
      "tests/functional/u/unused/unused_argument.py",
      "pylint/checkers/variables.py",
      "pylint/checkers/utils.py",
      "tests/functional/u/unused/unused_argument.txt"
    ],
    "source_doc": "[docstring: tests/functional/u/unused/unused_argument.py] tests.functional.u.unused.unused_argument.AAAA\ntests.functional.u.unused.unused_argument.AAAA:\n  dummy class\n\ntests.functional.u.unused.unused_argument.AAAA.method:\n  dummy method\n\ntests.functional.u.unused.unused_argument.AAAA.selected:\n  called by the registry when the vobject has been selected.\n\ntests.functional.u.unused.unused_argument.AAAA.using_inner_function:\n  return a fake result set for a particular entity type",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.0,
    "strip_verify_summary": "All claims in the gold answer are verifiable from the stripped source code, with only a minor rewrite needed for clarity on the abstract method skip mechanism.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_64",
    "repo": "pylint",
    "question": "How does the pylint docstring checker handle property setters and deleters to avoid emitting missing-docstring warnings?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `DocStringChecker` in pylint handles property setters and deleters by **early-returning from `visit_functiondef`** before any docstring check occurs. Here's the detailed mechanism:\n\n1. **Detection Flow**: In `visit_functiondef` of `docstring_checker.py`, after verifying the function name doesn't match `no_docstring_rgx`, the checker calls three guard functions: `is_property_setter(node)`, `is_property_deleter(node)`, and `is_overload_stub(node)`. If any returns `True`, the method returns immediately \u2014 no `_check_docstring()` is ever called, so no warning is emitted.\n\n2. **How setter/deleter detection works**: Both `is_property_setter` and `is_property_deleter` (in `pylint/checkers/utils.py`) delegate to `_is_property_kind(node, *kinds)`. This function checks if the node is a `FunctionDef` or `UnboundMethod`, then iterates over its `node.decorators.nodes`. For each decorator, it checks if the decorator is a `nodes.Attribute` and if `decorator.attrname` is in the specified `kinds` (e.g., `\"setter\"` or `\"deleter\"`). This means it matches decorators like `@test.setter` and `@test.deleter` by checking that the attribute name portion (after the dot) equals \"setter\" or \"deleter\" \u2014 it does NOT verify that the left-hand side actually refers to a property object.\n\n3. **Test verification**: The test file `missing_docstring.py` defines a `Property` class with a `@property`-decorated `test` getter, a `@test.setter` `test` method (no docstring), and a `@test.deleter` `test` method (no docstring). The expected output file (`missing_docstring.txt`) only contains 3 messages: missing-module-docstring (line 1), missing-class-docstring for `ClassUndocumented` (line 21), and missing-function-docstring for `public_undocumented` (line 25). No messages are generated for the setter or deleter methods.\n\n4. **Additional skip conditions**: The same early-return also skips `typing.overload` stubs (via `is_overload_stub`). Additionally, nested functions (parent frame is neither `ClassDef` nor `Module`) are silently skipped. Overridden methods in class hierarchies are checked but with `report_missing=False`, suppressing missing-docstring warnings while still checking for empty docstrings.\n\n5. **`_BasicChecker` base class**: `DocStringChecker` extends `_BasicChecker`, which itself extends `BaseChecker` and sets `name = \"basic\"`. This allows the docstring checker to share the \"basic\" checker name with other basic checkers.\n\n6. **Async function handling**: The line `visit_asyncfunctiondef = visit_functiondef` means async functions receive identical treatment \u2014 property setters/deleters on async functions would also be skipped.\n\n7. **`__doc__` fallback**: The `_check_docstring` method has a fallback via `_infer_dunder_doc_attribute()` \u2014 if no `doc_node` exists, it tries to infer a `__doc__` class attribute (as used by `DocumentedViaDunderDoc` in the test file), using `utils.safe_infer()` on the node's `__doc__` key.",
    "rubric": [
      "Explains that visit_functiondef early-returns before calling _check_docstring when is_property_setter or is_property_deleter returns True",
      "Describes how _is_property_kind checks if a decorator is a nodes.Attribute whose attrname matches 'setter' or 'deleter'",
      "Mentions that is_overload_stub is also checked as a skip condition in the same guard",
      "Notes that the detection does NOT verify the left-hand side of the decorator actually refers to a property object",
      "Mentions that visit_asyncfunctiondef is aliased to visit_functiondef so async functions get the same treatment",
      "Mentions the _infer_dunder_doc_attribute fallback in _check_docstring for inferring __doc__ class attributes",
      "Notes that overridden methods are checked with report_missing=False, suppressing missing-docstring but still flagging empty docstrings"
    ],
    "key_files": [
      "pylint/checkers/base/docstring_checker.py",
      "pylint/checkers/utils.py",
      "tests/functional/m/missing/missing_docstring.py",
      "tests/functional/m/missing/missing_docstring.txt"
    ],
    "source_doc": "[docstring: tests/functional/m/missing/missing_docstring.py] tests.functional.m.missing.missing_docstring.Property\ntests.functional.m.missing.missing_docstring.Property:\n  Don't warn about setters and deleters.\n\ntests.functional.m.missing.missing_docstring.Property.test:\n  Default docstring for setters and deleters.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation provided is extremely minimal \u2014 just docstrings for the Property class and its test method \u2014 while the answer contains extensive implementation details (line numbers, file paths, method names, class hierarchies) that cannot be verified from the given documentation chunk alone.",
      "The answer's claims about the Property class are consistent with the documentation: the class docstring says 'Don't warn about setters and deleters' and the test method docstring says 'Default docstring for setters and deleters', which aligns with the answer's explanation.",
      "Most of the detailed implementation claims (specific line numbers, method signatures, utility function internals) go far beyond what the documentation supports but don't directly contradict it."
    ],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; only minor inaccuracy about the getter having a docstring was corrected.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_65",
    "repo": "pylint",
    "question": "What does the `ModifiedIterationChecker` do when it encounters a `del` statement inside a for loop iterating over an instance attribute like `self.attribute`?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `MyClass` in `tests/functional/m/modified_iterating.py` tests whether pylint correctly detects modification of iteration targets when iterating over instance attributes (specifically `self.attribute`). The class defines `self.attribute = [1, 2, 3]` in `__init__` and then in `my_method` iterates over `self.attribute` while using `del var` to delete the loop variable, which triggers the `modified-iterating-list` (W4701) warning.\n\nThe detection is handled by the `ModifiedIterationChecker` class in `pylint/checkers/modified_iterating_checker.py`, which inherits from `checkers.BaseChecker`. The key detection path for this case goes through `_modified_iterating_check`, which checks if the node is a `nodes.Delete` statement, then calls `_deleted_iteration_target_cond` to verify the deleted variable matches the for-loop's iteration target. It then calls `utils.safe_infer(iter_obj)` on the `self.attribute` Attribute node to determine its type (resolving to `nodes.List()`), and uses a `match` statement to select the appropriate message ID.\n\nThe `_deleted_iteration_target_cond` method performs three checks: (1) the node is a `DelName`, (2) `iter_obj.parent` is a `For` node, and (3) the for-loop's target is an `AssignName` or `BaseContainer`. It then uses `utils.find_assigned_names_recursive` to find all assigned names from the target, supporting nested unpacking (e.g., `for inner_first, inner_second in ...`). The message is emitted with `confidence=interfaces.INFERENCE` (defined as `Confidence('INFERENCE', 'Warning based on inference result.')`).\n\nNotably, `modified-iterating-list` is a Warning (W4701), while `modified-iterating-dict` (E4702) and `modified-iterating-set` (E4703) are Errors\u2014the message descriptions in the code state that modifying a dict/set during iteration \"raises a RuntimeError,\" while modifying a list \"can result in unexpected behaviour.\" The expected output shows the message for `MyClass.my_method` as: `Iterated list 'attribute' is being modified inside for loop body` where 'attribute' comes from `iter_obj.repr_name()` on the `nodes.Attribute` node.",
    "rubric": [
      "Explains that `_modified_iterating_check` checks if the node is a `nodes.Delete` statement and calls `_deleted_iteration_target_cond` on each target",
      "Describes that `_deleted_iteration_target_cond` verifies the deleted node is a `DelName`, that `iter_obj.parent` is a `For` node, and that the for-loop target is an `AssignName` or `BaseContainer`",
      "Mentions that `find_assigned_names_recursive` is used to match the deleted variable name against the loop's assigned target names (supporting nested unpacking)",
      "Explains that `utils.safe_infer` is called on the iter_obj (the Attribute node) to determine its type (List, Dict, or Set) using a match statement to select the message ID",
      "Notes the distinction in severity: `modified-iterating-list` is a Warning (W4701), while dict (E4702) and set (E4703) are Errors because they raise RuntimeError",
      "Mentions that the message is emitted with `confidence=interfaces.INFERENCE` and uses `iter_obj.repr_name()` as the argument"
    ],
    "key_files": [
      "tests/functional/m/modified_iterating.py",
      "pylint/checkers/modified_iterating_checker.py",
      "pylint/checkers/utils.py",
      "pylint/interfaces.py"
    ],
    "source_doc": "[docstring: tests/functional/m/modified_iterating.py] tests.functional.m.modified_iterating.MyClass\ntests.functional.m.modified_iterating.MyClass:\n  Regression test for https://github.com/pylint-dev/pylint/issues/7380\n\ntests.functional.m.modified_iterating.MyClass.my_method:\n  This should raise as we are deleting.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation says 'we are deleting' in my_method, but the answer claims 'del var' is used to delete the loop variable. The doc doesn't specify the exact syntax used, though 'deleting' is consistent with a del statement.",
      "Most of the technical details about ModifiedIterationChecker, _modified_iterating_check, _deleted_iteration_target_cond, message IDs (W4701, E4702, E4703), confidence levels, and internal implementation are not supported by the provided documentation - they are plausible extrapolations from the codebase but not verifiable from the doc chunk alone.",
      "The claim about 'self.attribute = [1, 2, 3]' in __init__ is not stated in the documentation.",
      "The expected output message format 'Iterated list 'attribute' is being modified inside for loop body' is not stated in the documentation."
    ],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Only the GitHub issue reference (#7380) is pure doc-leakage; all other claims are verifiable from the stripped source code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_66",
    "repo": "pylint",
    "question": "Why doesn't the modified-iterating-dict checker fire a false positive when iterating a dict attribute but only modifying a copy of it?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `MyClass2` test class in `tests/functional/m/modified_iterating.py` tests the scenario where a dictionary is being iterated but only a *copy* of it is being modified. In `MyClass2.my_method`, the loop iterates over `self.attribute` (a dict), makes a copy (`tmp = self.attribute.copy()`), and modifies the copy (`tmp[key] = None`). This should NOT trigger a warning because the original iterated dict is not modified.\n\nThe checker avoids the false positive through the `_modified_iterating_dict_cond` method in `ModifiedIterationChecker`. Specifically, this method uses `_is_node_assigns_subscript_name` to check if the node is an assignment to a subscript with a `Name` value. For `tmp[key] = None`, this matches. However, the method then performs a **name comparison**: `node.targets[0].value.name == iter_obj_name`. Since `iter_obj` is `self.attribute` (a `nodes.Attribute`), `iter_obj_name` is set to `iter_obj.attrname` which is `\"attribute\"`. The assignment target's value name is `\"tmp\"`. Since `\"tmp\" != \"attribute\"`, the method returns `False` and no warning is emitted.\n\nAdditionally, even before the name comparison, `safe_infer(node.targets[0].value)` on `tmp` (which was assigned from `.copy()`) would likely not infer to a `nodes.Dict`, causing an early return at the `if not isinstance(infer_val, nodes.Dict): return False` check.",
    "rubric": [
      "Mentions that `_modified_iterating_dict_cond` (or the dict condition check) performs a name comparison between the assignment target's variable name and the iterated object's name",
      "Explains that for `tmp[key] = None`, the assignment target's value name is 'tmp' while the iterated object's attrname is 'attribute', so they don't match",
      "Mentions that `safe_infer` on `tmp` (which was assigned from `.copy()`) would likely not infer to a `nodes.Dict`, causing an early return from the condition",
      "References the `_is_node_assigns_subscript_name` check that verifies the node is an assignment to a subscript with a Name value",
      "Notes the test class (MyClass2 or regression test for issue 7461) that specifically tests this copy-then-modify pattern"
    ],
    "key_files": [
      "pylint/checkers/modified_iterating_checker.py",
      "tests/functional/m/modified_iterating.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/m/modified_iterating.py] tests.functional.m.modified_iterating.MyClass2\ntests.functional.m.modified_iterating.MyClass2:\n  Regression test for https://github.com/pylint-dev/pylint/issues/7461\n\ntests.functional.m.modified_iterating.MyClass2.my_method:\n  This should not raise, as a copy was made.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (specific method names like `_modified_iterating_dict_cond`, `_is_node_assigns_subscript_name`, line numbers like 165, variable name comparisons, `safe_infer` behavior) that are not present in the provided documentation and cannot be verified from it. These are plausible extrapolations/code-level details but are not directly supported.",
      "The core claims that MyClass2 is a regression test for issue #7461 and that my_method should not raise because a copy was made are directly supported by the documentation."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage from GitHub issue reference; core technical claims are all verifiable from stripped source code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_67",
    "repo": "pylint",
    "question": "Where does pylint determine that a @property-decorated method should be name-checked as an attribute rather than a method?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The property name checking for `FooClass` in `name_styles.py` is located in the `NameChecker` class in `pylint/checkers/base/name_checker/checker.py`. The key function that determines whether a property-decorated method should be checked as an attribute (rather than a method) is `_determine_function_name_type()`.\n\nHere's the data flow:\n\n1. When pylint visits a `FunctionDef` node (like `PROPERTY_NAME`), `visit_functiondef()` is called.\n2. Since the function is a method, it calls `_determine_function_name_type(node, config)` to determine whether to check it as 'function', 'method', or 'attr'.\n3. `_determine_function_name_type()` first checks `is_property_setter()` and `is_property_deleter()` (from `pylint/checkers/utils.py`). This is how `PROPERTY_NAME_SETTER` (decorated with `@PROPERTY_NAME.setter`) is classified as 'attr'.\n4. For `@property` and `@abc.abstractproperty` decorated methods, the function iterates over decorators, infers them, and checks if their `qname()` is in `property_classes` (which defaults to `{'builtins.property'}` plus any from `config.property_classes`, which defaults to `('abc.abstractproperty',)`).\n5. When the name type is 'attr', the `_check_name()` method uses the regex from `NamingStyle.get_regex('attr')`, which maps to `DEFAULT_NAME_RGX` \u2014 the same regex used for 'method', 'function', 'argument', and 'variable' types.\n6. However, when the warning message is generated in `_raise_name_warning()`, it uses `constants.HUMAN_READABLE_TYPES[node_type]` to produce the label. For 'attr', this maps to 'attribute' (defined in `HUMAN_READABLE_TYPES` in `pylint/constants.py`), which is why the warning says 'Attribute name' rather than 'Method name'.\n7. The confidence for all three property-related warnings in FooClass is `INFERENCE` (not `HIGH`), because `visit_functiondef` sets confidence to `interfaces.INFERENCE` when `utils.has_known_bases(node.parent.frame())` returns True for the class.\n\nThe default naming style for 'attr' is 'snake_case' (defined in `DEFAULT_NAMING_STYLES` in `naming_style.py`), so `PROPERTY_NAME`, `ABSTRACT_PROPERTY_NAME`, and `PROPERTY_NAME_SETTER` all fail the snake_case regex check.",
    "rubric": [
      "Identifies `_determine_function_name_type()` in the NameChecker class (in the base name checker) as the key function",
      "Explains that property setters/deleters are detected via `is_property_setter()` and `is_property_deleter()` from checkers/utils",
      "Explains that @property decorators are matched by inferring decorator nodes and checking qname() against `property_classes`",
      "Mentions that the function returns 'attr' as the name type for property-decorated methods",
      "Notes that `visit_functiondef` calls `_determine_function_name_type` when the function is a method",
      "Mentions that the human-readable label 'Attribute name' comes from HUMAN_READABLE_TYPES mapping 'attr' to 'attribute'",
      "Notes that confidence is set to INFERENCE (not HIGH) based on `has_known_bases()` check"
    ],
    "key_files": [
      "pylint/checkers/base/name_checker/checker.py",
      "pylint/checkers/base/name_checker/naming_style.py",
      "pylint/constants.py",
      "pylint/checkers/utils.py",
      "tests/functional/n/name/name_styles.py",
      "tests/functional/n/name/name_styles.txt"
    ],
    "source_doc": "[docstring: tests/functional/n/name/name_styles.py] tests.functional.n.name.name_styles.FooClass\ntests.functional.n.name.name_styles.FooClass:\n  A test case for property names.\n\n    Since by default, the regex for attributes is the same as the one\n    for method names, we check the warning messages to contain the\n    string 'attribute'.\n\ntests.functional.n.name.name_styles.FooClass.PROPERTY_NAME:\n  Ignored.\n\ntests.functional.n.name.name_styles.FooClass.ABSTRACT_PROPERTY_NAME:\n  Ignored.\n\ntests.functional.n.name.name_styles.FooClass.PROPERTY_NAME_SETTER:\n  Ignored.\n\ntests.functional.n.name.name_styles.FooClass._nice_and_long_descriptive_private_method_name:\n  private method with long name",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only specific line numbers and one reference to an unseen internal function (_is_property_kind) needed minor adjustments.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_68",
    "repo": "pylint",
    "question": "How does pylint's `no-self-argument` check determine whether a method's first argument is valid, including its handling of staticmethods, metaclasses, and special methods?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `no-self-argument` check (message ID `E0213`) is implemented in the `_check_first_arg_for_type` method of the `ClassChecker` class (which extends `BaseChecker`) in `pylint/checkers/classes/class_checker.py`. Here's how the algorithm works:\n\n**Entry point:** The check is triggered by `visit_functiondef` (line 1268), which first verifies `node.is_method()` returns True (i.e., the function is inside a class), then calls `self._check_first_arg_for_type(node, klass.type == \"metaclass\")` at line 1280.\n\n**Algorithm in `_check_first_arg_for_type` (lines 2081-2157):**\n\n1. **Guard clause:** If `node.args.args is None`, return immediately.\n\n2. **Determine `first_arg`:** Check `posonlyargs` first (positional-only args), then `node.args.args` (using `node.argnames()[0]`). If neither has entries, `first_arg` is set to `None`. This `None` case covers methods that only have `*args` and/or `**kwargs` but no positional parameters.\n\n3. **Track first attribute:** Append `first_arg` to `self._first_attrs` list (initialized as empty in `__init__`, popped in `leave_functiondef`).\n\n4. **Static method check (two-tier):**\n   - If `node.type == \"staticmethod\"` (direct `@staticmethod` decorator or alias like `MYSTATICMETHOD = staticmethod`): checks if the first arg looks like it should be an instance/class arg, emits `bad-staticmethod-argument` if so, otherwise sets `_first_attrs[-1] = None` and returns.\n   - Elif `\"builtins.staticmethod\" in node.decoratornames()`: catches decorators that *return* `staticmethod()` (like `returns_staticmethod` in the test), and simply returns without emitting any message.\n\n5. **No-argument method check:** If the method has no `args`, `posonlyargs`, `vararg`, or `kwarg` at all, emit `no-method-argument` (E0211).\n\n6. **Metaclass handling:** For metaclass methods, delegates to `_check_first_arg_config` with either `valid_metaclass_classmethod_first_arg` or `valid_classmethod_first_arg` config options.\n\n7. **`__class_getitem__` special case:** Treated as a classmethod regardless of its actual decorator (line 2147: `node.type == \"classmethod\" or node.name == \"__class_getitem__\"`), validated against `valid_classmethod_first_arg`.\n\n8. **Final fallback \u2014 `no-self-argument`:** If none of the above matched and `first != \"self\"`, emit `no-self-argument`. This is the key line (2156-2157).\n\n**Edge cases from the test file:**\n- Methods with `*args` only, `**kwargs` only, or both: `first_arg` is `None` (because `node.args.args` is empty), but they pass the no-args check (because `vararg`/`kwarg` is truthy), so they fall through to the `first != \"self\"` check, and `None != \"self\"` triggers the warning.\n- `__class_other__` on the `Toto` class: unlike `__class_getitem__`, it has no special exemption and triggers `no-self-argument` because its first arg is `cls`, not `self`.\n- `say_goodbye` using `MYSTATICMETHOD` (which is `staticmethod` assigned to a variable): `node.type` resolves to `\"staticmethod\"` so it's handled in the first branch.\n- `concatenate_strings` using `returns_staticmethod` (a function returning `staticmethod(...)`): `node.type` won't be `\"staticmethod\"`, but `\"builtins.staticmethod\"` appears in `node.decoratornames()` via astroid inference, so the second branch catches it.\n\nThe message format is `'Method %r should have \"self\" as first argument'` with the method name substituted.",
    "rubric": [
      "Explains that the check is triggered from visit_functiondef after confirming the function is a method (node.is_method()) and determining if the parent class is a metaclass",
      "Describes the guard clause: returns immediately if node.args.args is None (unknown/builtin arguments)",
      "Explains how first_arg is determined: checks posonlyargs first, then node.args.args, otherwise sets first_arg to None (for methods with only *args/**kwargs)",
      "Describes the staticmethod handling: if node.type == 'staticmethod', checks if first arg looks like self/cls and emits bad-staticmethod-argument; if 'builtins.staticmethod' in decoratornames(), returns without emitting any message",
      "Explains the no-method-argument case: emits E0211 if the method has no args, posonlyargs, vararg, or kwarg at all",
      "Describes metaclass handling: delegates to _check_first_arg_config with valid_metaclass_classmethod_first_arg or valid_classmethod_first_arg depending on whether it's a classmethod",
      "Explains that __class_getitem__ is treated as a classmethod regardless of its actual decorator type",
      "Describes the final fallback: if first != 'self', emits no-self-argument (E0213)"
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "tests/functional/n/no/no_self_argument.py",
      "tests/functional/n/no/no_self_argument.txt"
    ],
    "source_doc": "[docstring: tests/functional/n/no/no_self_argument.py] tests.functional.n.no.no_self_argument.NoSelfArgument\ntests.functional.n.no.no_self_argument.NoSelfArgument:\n  dummy class\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.__init__:\n  method without self\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.abdc:\n  another test\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.edf:\n  just another method\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.say_hello:\n  A standard staticmethod\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.say_goodbye:\n  A staticmethod but using a different name\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.concatenate_strings:\n  A staticmethod created by `returns_staticmethod` function\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.varargs:\n  A method without a self argument but with *args.\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.kwargs:\n  A method without a self argument but with **kwargs.\n\ntests.functional.n.no.no_self_argument.NoSelfArgument.varargs_and_kwargs:\n  A method without a self argument but with *args and **kwargs.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.02,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code, MSGS dict, and test files; only a minor parenthetical explanation was slightly doc-dependent.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_69",
    "repo": "pylint",
    "question": "What conditions prevent `MyClassBase` in the property-with-parameters test from triggering the `property-with-parameters` warning, and how are those conditions checked?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `property_with_parameters.py` functional test file tests the `property-with-parameters` message defined in `pylint/checkers/classes/class_checker.py`. The `MyClassBase` class serves as a **negative test case** \u2014 it does NOT trigger the warning, as confirmed by the expected output file.\n\nThe check is implemented in the `_check_property_with_parameters` method of `ClassChecker` (which inherits from `BaseChecker`). The method checks three conditions conjunctively:\n1. `len(node.args.arguments) > 1` \u2014 the function has more than just `self`\n2. `decorated_with_property(node)` \u2014 the function is decorated with `@property` or `@functools.cached_property`\n3. `not is_property_setter(node)` \u2014 the function is NOT a property setter\n\nFor `MyClassBase.example` getter: it has only `self`, so condition 1 fails (len == 1).\nFor `MyClassBase.example` setter: it has `(self, value)` so condition 1 is True, but `is_property_setter` returns True because the `@example.setter` decorator is an `Attribute` node with `attrname == \"setter\"`, checked by `_is_property_kind` in `pylint/checkers/utils.py`. So condition 3 fails.\n\nThe `_is_property_decorator` helper function (in `pylint/checkers/utils.py`) recognizes both `builtins.property` and `functools.cached_property` as valid property decorators, and also handles ancestor classes that inherit from `builtins.property`. It additionally handles function decorators that return a property-decorated function.\n\nThe test's expected output (`.txt` file) only contains 5 warnings, all for class `Cls` (methods a-e), confirming `MyClassBase` emits no warnings. The `_check_property_with_parameters` is called from `visit_functiondef`, which first checks `node.is_method()` to ensure only methods are checked. The `ClassChecker` also aliases `visit_asyncfunctiondef = visit_functiondef`.",
    "rubric": [
      "Identifies the three conjunctive conditions in `_check_property_with_parameters`: (1) more than one argument, (2) decorated with property, (3) not a property setter",
      "Explains that the getter `example(self)` fails condition 1 because it only has `self` (len(node.args.arguments) == 1, not > 1)",
      "Explains that the setter `example(self, value)` passes condition 1 but fails condition 3 because `is_property_setter` returns True",
      "Describes how `is_property_setter` works via `_is_property_kind`, checking if any decorator is an `Attribute` node with `attrname == 'setter'`",
      "Notes that `decorated_with_property` uses `_is_property_decorator` which recognizes `builtins.property` and `functools.cached_property`",
      "Mentions that the expected output file only contains 5 warnings for class `Cls` (methods a through e), confirming `MyClassBase` emits no warnings"
    ],
    "key_files": [
      "tests/functional/p/property_with_parameters.py",
      "pylint/checkers/classes/class_checker.py",
      "pylint/checkers/utils.py",
      "tests/functional/p/property_with_parameters.txt"
    ],
    "source_doc": "[docstring: tests/functional/p/property_with_parameters.py] tests.functional.p.property_with_parameters.MyClassBase\ntests.functional.p.property_with_parameters.MyClassBase:\n  MyClassBase.\n\ntests.functional.p.property_with_parameters.MyClassBase.example:\n  Getter.\n\ntests.functional.p.property_with_parameters.MyClassBase.example:\n  Setter.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only provides docstrings for MyClassBase, its getter, and its setter. The vast majority of the answer's claims (about ClassChecker, _check_property_with_parameters, condition checking logic, line numbers, is_property_setter, _is_property_decorator, expected output details, visit_functiondef, etc.) are not supported by the provided documentation and cannot be verified from it.",
      "The claim that MyClassBase is a 'negative test case' is a plausible extrapolation given the docstrings show a simple getter/setter pattern, but is not explicitly stated in the documentation.",
      "The answer's claims about the getter having docstring 'Getter.' and the setter having docstring 'Setter.' are consistent with the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from code structure, imports, test files, and logic; only minor documentation-dependent details like specific line numbers needed adjustment.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_70",
    "repo": "pylint",
    "question": "Why does the `@only_required_for_messages` decorator on `visit_attribute` help with performance, and how does it interact with the ASTWalker?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The regression test `regression_property_no_member_3269.py` exists to prevent a false-positive `no-member` error when accessing `.fget` on a property descriptor accessed via the class (e.g., `A.test.fget(self)`). The design that prevents this false positive operates at multiple levels:\n\n1. **Early-exit in `visit_attribute`:** When `A.test` is accessed from the class (not an instance), astroid infers it as a property object. Since property objects have `fget` as a valid attribute, `owner.getattr(node.attrname)` succeeds, hitting a `break` and skipping all remaining inferred values.\n\n2. **`@only_required_for_messages` decorator:** The `visit_attribute` method is decorated with `@only_required_for_messages(\"no-member\", \"c-extension-no-member\")`. In `ASTWalker.add_checker()` (ast_walker.py), methods are only added to the walk events if at least one of their handled messages is enabled (`_is_method_enabled`). This means the entire attribute checking logic is completely skipped when these messages are disabled, and `visit_attribute` will not be called on attribute nodes in the AST.\n\n3. **`ignore_on_opaque_inference` (default: True):** If any inference result is an opaque/uninferrable object and this setting is True, the method returns immediately without checking any attributes.\n\nThe test itself has no `.txt` file (meaning zero expected lint messages), and class `B` does not inherit from class `A`\u2014it accesses `A.test.fget(self)` directly, a valid Python pattern for manually invoking another class's property getter without inheritance.",
    "rubric": [
      "Explains that `only_required_for_messages` stores the message names (e.g., 'no-member', 'c-extension-no-member') as a `checks_msgs` attribute on the method",
      "Explains that ASTWalker's `_is_method_enabled` checks whether any of the decorator's messages are enabled before registering the method",
      "Explains that in `add_checker`, visit/leave methods are only appended to the walk events if `_is_method_enabled` returns True, so the method is never called if none of its messages are enabled",
      "Explains that this avoids running expensive inference and attribute-checking logic in `visit_attribute` when `no-member` and `c-extension-no-member` are both disabled"
    ],
    "key_files": [
      "tests/functional/r/regression/regression_property_no_member_3269.py",
      "pylint/checkers/typecheck.py",
      "pylint/utils/ast_walker.py",
      "pylint/checkers/utils.py"
    ],
    "source_doc": "[docstring: tests/functional/r/regression/regression_property_no_member_3269.py] tests.functional.r.regression.regression_property_no_member_3269.B\ntests.functional.r.regression.regression_property_no_member_3269.B:\n  A child class\n\ntests.functional.r.regression.regression_property_no_member_3269.B.test:\n  Overriding implementation of prop which calls the parent",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims 'class B does not inherit from class A' but the documentation explicitly states B is 'A child class', which strongly implies B inherits from A or is at least described as a child class.",
      "The documentation says B.test is an 'Overriding implementation of prop which calls the parent', suggesting B does inherit from A and overrides the test property, contradicting the claim that B 'accesses A.test.fget(self) directly without inheritance'.",
      "Most of the detailed claims about typecheck.py internals (line numbers, decorator behavior, ignore_on_opaque_inference) are not supported by the documentation and cannot be verified from it, though they may be plausible extrapolations from the broader codebase."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "One claim directly quoted a docstring (removed), and two claims included design rationale language that was softened; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_71",
    "repo": "pylint",
    "question": "Where is `_called_in_methods` defined and how does it guard against non-NodeNG objects returned during metaclass lookup?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `_called_in_methods` function that was fixed for regression #10892 is located in `pylint/checkers/classes/class_checker.py`. It is a module-level helper function (not a method of any class). The fix is the guard: `if not isinstance(infer_method, nodes.NodeNG): continue`, which prevents calling `.nodes_of_class()` on non-NodeNG objects returned by `klass.getattr()`. This function is called from within the `_check_attribute_defined_outside_init` method of the `ClassChecker` class, which itself is invoked from the `leave_classdef` visitor method. The test case triggers the bug by defining a metaclass `Meta` with `__init__ = some_descriptor` (an undefined variable), causing astroid's `getattr('__init__')` on `MyClass` to return a non-NodeNG object, which the isinstance guard now handles.",
    "rubric": [
      "Identifies that `_called_in_methods` is defined as a module-level helper function in the classes checker module (class_checker.py)",
      "Explains the isinstance guard: `if not isinstance(infer_method, nodes.NodeNG): continue` prevents calling `.nodes_of_class()` on non-NodeNG objects",
      "Mentions that this function is called from `_check_attribute_defined_outside_init` within the `ClassChecker` class",
      "Notes the connection to `leave_classdef` as the visitor method that triggers the check",
      "Describes how the regression test triggers the bug: a metaclass with `__init__ = some_descriptor` (undefined variable) causes `klass.getattr()` to return non-NodeNG objects like UninferableBase"
    ],
    "key_files": [
      "pylint/checkers/classes/class_checker.py",
      "tests/functional/r/regression_02/regression_10892.py",
      "tests/functional/r/regression_02/regression_10892.txt"
    ],
    "source_doc": "[docstring: tests/functional/r/regression_02/regression_10892.py] tests.functional.r.regression_02.regression_10892\nRegression test for https://github.com/pylint-dev/pylint/issues/10892\n\nCrash in ``_called_in_methods`` when ``klass.getattr()`` returns non-NodeNG\nobjects (e.g. ``UninferableBase``) from the metaclass lookup path.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Nearly all claims are verifiable from stripped code; minor rewrites needed for runtime behavior details about UninferableBase through metaclass lookup.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "pylint_gen_72",
    "repo": "pylint",
    "question": "How does the `bad-super-call` checker avoid false positives when a super() call occurs inside a nested class defined within a method?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `bad-super-call` check for `SuperDifferentScope` is handled by the `NewStyleConflictChecker` class in `pylint/checkers/newstyle.py`. The key mechanism that prevents false positives for nested class scopes works as follows:\n\n1. **Scope filtering via `node_frame_class`**: In `visit_functiondef`, the checker iterates over all `nodes.Call` instances within the method using `node.nodes_of_class(nodes.Call)`. For each call statement, it compares `node_frame_class(stmt) != node_frame_class(node)`. If they differ, it skips the statement with `continue`.\n\n2. **How `node_frame_class` works** (in `pylint/checkers/utils.py`): It walks up the node tree by repeatedly calling `klass.parent.frame()` until it finds a `ClassDef` node. For the nested `FalsePositive.__init__` super call at line 61 of the test, `node_frame_class(stmt)` returns the `FalsePositive` ClassDef, while `node_frame_class(node)` (where `node` is the `test` method) returns `SuperDifferentScope`. Since they differ, the nested super call is skipped.\n\n3. **The `super(object, 1).__init__()` call at line 62**: This IS in the same scope as `SuperDifferentScope.test`, but no error is emitted because the checker verifies `klass is not supcls and all(i != supcls for i in klass.ancestors())`. Since `object` is in the ancestors of `SuperDifferentScope` (all classes implicitly inherit from `object`), the `all()` condition is False, so no message is emitted.\n\n4. **Pattern matching**: The checker uses Python 3.10+ structural pattern matching (`match`/`case`) to identify super calls. It first uses `match call := expr.expr` with `case nodes.Call(func=nodes.Name(name=\"super\"), args=[arg0, *_])` to detect calls to `super()` with arguments.\n\n5. **The checker also handles `visit_asyncfunctiondef`** via the alias `visit_asyncfunctiondef = visit_functiondef`.\n\n6. **The checker class** inherits from `BaseChecker`, is named \"newstyle\", has message ID `E1003`, and is registered via the `register()` function that creates a `NewStyleConflictChecker(linter)` instance.",
    "rubric": [
      "Explains that `node_frame_class` is used to compare the scope of each call statement against the scope of the enclosing method/function node",
      "Describes how `node_frame_class` walks up the node tree via `klass.parent.frame()` until it finds a ClassDef",
      "States that if `node_frame_class(stmt) != node_frame_class(node)`, the statement is skipped (continue)",
      "Mentions that the checker iterates over all `nodes.Call` instances within the method via `node.nodes_of_class(nodes.Call)`",
      "Explains the additional ancestor check (`klass is not supcls and all(i != supcls for i in klass.ancestors())`) that prevents false positives for valid super calls like `super(object, 1)`",
      "Notes that structural pattern matching (match/case) is used to identify super calls with arguments"
    ],
    "key_files": [
      "pylint/checkers/newstyle.py",
      "pylint/checkers/utils.py",
      "tests/functional/s/super/super_checks.py",
      "tests/functional/s/super/super_checks.txt"
    ],
    "source_doc": "[docstring: tests/functional/s/super/super_checks.py] tests.functional.s.super.super_checks.SuperDifferentScope\ntests.functional.s.super.super_checks.SuperDifferentScope:\n  Don'emit bad-super-call when the super call is in another scope.\n    For reference, see https://bitbucket.org/logilab/pylint/issue/403.\n\ntests.functional.s.super.super_checks.SuperDifferentScope.test:\n  Test that a bad-super-call is not emitted for this case.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that bad-super-call should not be emitted when the super call is in another scope, referencing a specific issue. The answer provides extensive implementation details (line numbers, class names, method names, pattern matching details) that are not verifiable from the provided documentation alone.",
      "The answer claims specific implementation details like 'NewStyleConflictChecker' class, 'node_frame_class' utility, Python 3.10+ pattern matching, specific line numbers, etc. \u2014 none of which can be verified from the documentation provided, making them potential hallucinations.",
      "The documentation mentions the class is 'SuperDifferentScope' and its purpose is to not emit bad-super-call when super is in another scope. The answer's general claim about scope-based filtering is consistent with this, but the specific implementation details are unverifiable."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage from a stripped comment ('Don't look down in other scopes') and an unverifiable claim about BaseChecker's parent class.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_04",
    "repo": "seaborn",
    "question": "How does the Jitter move generate and apply random displacement to data points, and how does it decide on default width?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `Jitter` class in seaborn implements random displacement of data points. Here's how it works in detail:\n\n**Implementation Algorithm (seaborn/_core/moves.py):**\n\n1. **Class Structure**: `Jitter` is a `@dataclass` that inherits from `Move` base class. It has `group_by_orient: ClassVar[bool] = True` (inherited from `Move`).\n\n2. **Parameters**: It has 4 fields: `width` (typed as `float | Default`, defaulting to the `default` sentinel), `x` (float, default 0), `y` (float, default 0), and `seed` (int | None, default None).\n\n3. **Noise Generation**: Uses `np.random.default_rng(self.seed)` to create a random number generator. The noise is generated from a uniform distribution in the range `[-0.5, +0.5]` via `rng.uniform(-.5, +.5, len(data))`, then scaled by the relevant scale factor.\n\n4. **Default Width Logic**: When `self.width is default` (checked via identity comparison with the `Default` sentinel instance), the local `width` variable is set to `0.0` if either `self.x` or `self.y` is truthy (non-zero), otherwise it defaults to `0.2`. This means by default there's a small jitter of 20% of mark width along the orient axis.\n\n5. **Critical Conditional Logic (line 70)**: After computing the local `width`, the code checks `if self.width:` to decide whether to apply orient-axis jitter. Since `Default.__bool__` is not defined, the `Default` instance is always truthy. This means when no explicit `width` is provided, the orientation jitter is *always* applied (even when the computed local width is 0.0, but 0.0 * anything = 0, so it effectively does nothing in that case).\n\n6. **Width-based jitter**: For the orientation axis, the jitter is `noise * width * data[\"width\"]` \u2014 it's relative to the mark's width column.\n\n7. **Absolute x/y jitter**: For `self.x` and `self.y`, the jitter is in data units: `noise * self.x` or `noise * self.y` \u2014 these are added directly without referencing the width column.\n\n**Integration with Plot Pipeline:**\n\nThe `Jitter` move's `__call__` method accepts a DataFrame, GroupBy, orientation string ('x' or 'y'), and scales dict.\n\n**Data Copying**: `Jitter.__call__` uses `data.copy()` (deep copy), while `Shift.__call__` uses `data.copy(deep=False)` (shallow copy).",
    "rubric": [
      "Uses np.random.default_rng(self.seed) to create a random number generator",
      "Generates noise from a uniform distribution in [-0.5, +0.5] via rng.uniform(-.5, +.5, len(data))",
      "Default width is 0.0 if self.x or self.y is non-zero, otherwise 0.2",
      "Width-based jitter along orient axis is scaled by data['width'] column (noise * width * data['width'])",
      "x and y jitter are in data units, added directly without referencing the width column",
      "The condition 'if self.width:' uses the truthiness of the Default sentinel (which is always truthy) to gate orient-axis jitter application",
      "Jitter makes a deep copy of data (data.copy()), unlike Shift which uses shallow copy"
    ],
    "key_files": [
      "seaborn/_core/moves.py",
      "seaborn/_core/typing.py",
      "seaborn/_core/plot.py",
      "seaborn/_core/groupby.py"
    ],
    "source_doc": "[docstring: seaborn/_core/moves.py] seaborn._core.moves.Jitter\nseaborn._core.moves.Jitter:\n  Random displacement along one or both axes to reduce overplotting.\n\n    Parameters\n    ----------\n    width : float\n        Magnitude of jitter, relative to mark width, along the orientation axis.\n        If not provided, the default value will be 0 when `x` or `y` are set, otherwise\n        there will be a small amount of jitter applied by default.\n    x : float\n        Magnitude of jitter, in data units, along the x axis.\n    y : float\n        Magnitude of jitter, in data units, along the y axis.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Jitter.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed claims about upstream pipeline details not visible in truncated code and docstring-only references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_01",
    "repo": "seaborn",
    "question": "What is the architecture of VectorPlotter, including its subclass hierarchy, semantic mapping system, and how subclasses customize behavior?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `VectorPlotter` class in `seaborn/_base.py` is the base class that the main seaborn plotter classes inherit from. Here's its detailed architecture:\n\n**Class Hierarchy:**\n- `VectorPlotter` is the base class (line 616)\n- Direct subclasses: `_RelationalPlotter` (relational.py:188), `_CategoricalPlotter` (categorical.py:49), `_DistributionPlotter` (distributions.py:99)\n- `_RelationalPlotter` is further subclassed by `_LinePlotter` (relational.py:198) and `_ScatterPlotter` (relational.py:383)\n\n**Wide-form Data Structure Overrides:**\nEach subclass overrides `wide_structure` and `flat_structure` class attributes:\n- `VectorPlotter`: `{\"x\": \"@index\", \"y\": \"@values\", \"hue\": \"@columns\", \"style\": \"@columns\"}` with `flat_structure = {\"x\": \"@index\", \"y\": \"@values\"}`\n- `_CategoricalPlotter`: `{\"x\": \"@columns\", \"y\": \"@values\", \"hue\": \"@columns\"}` with `flat_structure = {\"y\": \"@values\"}`\n- `_DistributionPlotter`: `{\"x\": \"@values\", \"hue\": \"@columns\"}` with `flat_structure = {\"x\": \"@values\"}`\n\n**Semantic Mapping System:**\nThree `SemanticMapping` subclasses exist: `HueMapping`, `SizeMapping`, and `StyleMapping`.\n- `HueMapping` (line 88): Contains `categorical_mapping` and `numeric_mapping` methods for producing color lookup tables from categorical palettes or continuous colormaps. Has `infer_map_type` method that checks `QUAL_PALETTES`, `norm`, dict/list palette types, input format, and var_type to determine mapping approach.\n- `SizeMapping` (line 298): Contains `categorical_mapping` and `numeric_mapping` methods for producing size lookup tables. Falls back to `self.plotter._default_size_range` when no sizes specified.\n- `StyleMapping` (line 519): Always categorical (`map_type = \"categorical\"`). Maps to markers and dashes. Validates that markers can't mix filled and unfilled types.\n\n**Key Implementation Details:**\n1. `_default_size_range = 1, 2` on the base class. `_LinePlotter` overrides it to `np.r_[.5, 2] * mpl.rcParams[\"lines.linewidth\"]` and `_ScatterPlotter` to `np.r_[.5, 2] * np.square(mpl.rcParams[\"lines.markersize\"])`.\n\n2. During `__init__`, after `assign_variables`, the plotter auto-calls `map_hue()`, `map_size()`, or `map_style()` for any semantic variable present in the `variables` dict (lines 639-641).\n\n3. The `assign_variables` method uses `PlotData` from `seaborn._core.data` for long-form input (when x or y is specified), but uses its own `_assign_variables_wideform` method otherwise.\n\n4. `PlotData._assign_variables` supports looking up variables from both DataFrame columns and index levels (line 182 in _core/data.py).\n\n5. `comp_data` property replaces inf values with NaN, filters data to values in `var_levels`, converts units, applies axis transforms, and caches the result in `_comp_data`.\n\n6. `_attach` imports `FacetGrid` from `.axisgrid` (deferred import) and handles axis sharing by mapping each data point to its converter axis object via a `pd.Series` stored in `self.converters`. It uses `facet_dim = {\"x\": \"col\", \"y\": \"row\"}` to determine which facet dimension corresponds to which axis.\n\n7. `iter_data` handles a pandas version compatibility issue: for pandas < 2.2.0, single-element group keys are passed as scalars to `get_group`, otherwise as tuples (line 944-945).\n\n8. `VariableType` (line 1451) extends `UserString` and restricts equality comparisons to only `(\"numeric\", \"datetime\", \"categorical\")`, asserting on invalid comparisons.\n\n9. `handle_data_source` in `_core/data.py` supports objects with a `to_pandas()` method, as seen in the code that checks `hasattr(data, \"to_pandas\")` and attempts to call it.\n\n10. The `_legend_attributes` class variable differs per subclass: `_CategoricalPlotter` uses `[\"color\"]`, `_LinePlotter` uses `[\"color\", \"linewidth\", \"marker\", \"dashes\"]`, and `_ScatterPlotter` uses `[\"color\", \"s\", \"marker\"]`.",
    "rubric": [
      "Identifies VectorPlotter as the base class in seaborn/_base.py with direct subclasses _RelationalPlotter, _CategoricalPlotter, and _DistributionPlotter",
      "Identifies _LinePlotter and _ScatterPlotter as subclasses of _RelationalPlotter",
      "Describes how wide_structure and flat_structure class attributes differ across subclasses (e.g., _CategoricalPlotter uses @columns for x, _DistributionPlotter uses @values for x)",
      "Describes the three SemanticMapping subclasses: HueMapping, SizeMapping, and StyleMapping, and notes that StyleMapping is always categorical",
      "Explains that __init__ auto-calls map_hue/map_size/map_style for semantic variables present in the variables dict",
      "Mentions _default_size_range differs per subclass: base=(1,2), _LinePlotter uses lines.linewidth, _ScatterPlotter uses lines.markersize squared",
      "Describes _legend_attributes differences: _CategoricalPlotter=['color'], _LinePlotter=['color','linewidth','marker','dashes'], _ScatterPlotter=['color','s','marker']",
      "Mentions VariableType extending UserString with restricted equality comparisons to 'numeric', 'datetime', 'categorical'",
      "Mentions the comp_data property that replaces inf with NaN, converts units, applies axis transforms, and caches results",
      "Notes the pandas version compatibility handling in iter_data for group keys (single-element keys as scalars for pandas < 2.2.0)"
    ],
    "key_files": [
      "seaborn/_base.py",
      "seaborn/_core/data.py",
      "seaborn/relational.py",
      "seaborn/categorical.py",
      "seaborn/distributions.py"
    ],
    "source_doc": "[docstring: seaborn/_base.py] seaborn._base.VectorPlotter\nseaborn._base.VectorPlotter:\n  Base class for objects underlying *plot functions.\n\nseaborn._base.VectorPlotter.has_xy_data:\n  Return True at least one of x or y is defined.\n\nseaborn._base.VectorPlotter.var_levels:\n  Property interface to ordered list of variables levels.\n\n        Each time it's accessed, it updates the var_levels dictionary with the\n        list of levels in the current semantic mappers. But it also allows the\n        dictionary to persist, so it can be used to set levels by a key. This is\n        used to track the list of col/row levels using an attached FacetGrid\n        object, but it's kind of messy and ideally fixed by improving the\n        faceting logic so it interfaces better with the modern approach to\n        tracking plot variables.\n\nseaborn._base.VectorPlotter.assign_variables:\n  Define plot variables, optionally using lookup from `data`.\n\nseaborn._base.VectorPlotter._assign_variables_wideform:\n  Define plot variables given wide-form data.\n\n        Parameters\n        ----------\n        data : flat vector or collection of vectors\n            Data can be a vector or mapping that is coerceable to a Series\n            or a sequence- or mapping-based collection of such vectors, or a\n            rectangular numpy array, or a Pandas DataFrame.\n        kwargs : variable -> data mappings\n            Behavior with keyword arguments is currently undefined.\n\n        Returns\n        -------\n        plot_data : :class:`pandas.DataFrame`\n            Long-form data object mapping seaborn variables (x, y, hue, ...)\n            to data vectors.\n        variables : dict\n            Keys are defined seaborn variables; values are names inferred from\n            the inputs (or None when no name can be determined).\n\nseaborn._base.VectorPlotter.iter_data:\n  Generator for getting subsets of data defined by semantic variables.\n\n        Also injects \"col\" and \"row\" into grouping semantics.\n\n        Parameters\n        ----------\n        grouping_vars : string or list of strings\n            Semantic variables that define the subsets of data.\n        reverse : bool\n            If True, reverse the order of iteration.\n        from_comp_data : bool\n            If True, use self.comp_data rather than self.plot_data\n        by_facet : bool\n            If True, add faceting variables to the set of grouping variables.\n        allow_empty : bool\n            If True, yield an empty dataframe when no observations exist for\n            combinations of grouping variables.\n        dropna : bool\n            If True, remove rows with missing data.\n\n        Yields\n        ------\n        sub_vars : dict\n            Keys are semantic names, values are the level of that semantic.\n        sub_data : :class:`pandas.DataFrame`\n            Subset of ``plot_data`` for this combination of semantic values.\n\nseaborn._base.VectorPlotter.comp_data:\n  Dataframe with numeric x and y, after unit conversion and log scaling.\n\nseaborn._base.VectorPlotter._get_axes:\n  Return an Axes object based on existence of row/col variables.\n\nseaborn._base.VectorPlotter._attach:\n  Associate the plotter with an Axes manager and initialize its units.\n\n        Parameters\n        ----------\n        obj : :class:`matplotlib.axes.Axes` or :class:'FacetGrid`\n            Structural object that we will eventually plot onto.\n        allowed_types : str or list of str\n            If provided, raise when either the x or y variable does not have\n            one of the declared seaborn types.\n        log_scale : bool, number, or pair of bools or numbers\n            If not False, set the axes to use log scaling, with the given\n            base or defaulting to 10. If a tuple, interpreted as separate\n            arguments for the x and y axes.\n\nseaborn._base.VectorPlotter._get_scale_transforms:\n  Return a function implementing the scale transform (or its inverse).\n\nseaborn._base.VectorPlotter._add_axis_labels:\n  Add axis labels if not present, set visibility to match ticklabels.\n\nseaborn._base.VectorPlotter.add_legend_data:\n  Add labeled artists to represent the different plot semantics.\n\nseaborn._base.VectorPlotter._update_legend_data:\n  Generate legend tick values and formatted labels.\n\nseaborn._base.VectorPlotter.scale_categorical:\n  Enforce categorical (fixed-scale) rules for the data on given axis.\n\n        Parameters\n        ----------\n        axis : \"x\" or \"y\"\n            Axis of the plot to operate on.\n        order : list\n            Order that unique values should appear in.\n        formatter : callable\n            Function mapping values to a string representation.\n\n        Returns\n        -------\n        self",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer contains extensive implementation details (line numbers, specific class attributes, subclass hierarchies, internal variable names) that are not present in the provided documentation chunk. These could be accurate based on the actual source code, but cannot be verified from the documentation alone.",
      "The documentation describes `_assign_variables_wideform` as returning 'plot_data' and 'variables', while the answer claims `assign_variables` uses `PlotData` from `seaborn._core.data` for long-form input - this is plausible but not verifiable from the doc.",
      "The answer claims `_default_size_range = 1, 2` is annotated as 'Unused but needed in tests, ugh' - this cannot be verified from the documentation.",
      "The answer claims specific line numbers (e.g., line 616, 88, 298, 519, 624, 639-641, 944-945, 1451) which cannot be verified from the documentation.",
      "The answer's description of `comp_data` as replacing inf values with NaN and filtering to var_levels goes beyond the documentation which simply states it is a 'Dataframe with numeric x and y, after unit conversion and log scaling.'",
      "The answer's description of `iter_data` handling pandas version compatibility is not mentioned in the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor leakage from a comment annotation and some design rationale descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_02",
    "repo": "seaborn",
    "question": "Why does `variable_type` go beyond pandas' typing API instead of just relying on `pd.api.types.is_numeric_dtype` and friends?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `variable_type` function exists in `seaborn/_core/rules.py` (and is referenced/used in `seaborn/_base.py`). The function's approach goes beyond simply using pandas' typing API, as can be seen from its code logic:\n\n1. **Handling object-dtype data containing numbers**: The function uses an inner helper `all_numeric()` that iterates through each element checking `isinstance(x_i, Number)` (from Python's `numbers` module) as a fallback when `pd.api.types.is_numeric_dtype` returns False. This allows detection of numeric data in object-dtype containers.\n\n2. **Boolean/binary data handling**: The `boolean_type` parameter controls how binary data (only 0s and 1s) is classified. The `_core/rules.py` version adds a third option `boolean_type='boolean'` and a `strict_boolean` parameter \u2014 when `strict_boolean=True`, only actual bool/Boolean dtypes are treated as boolean (not integer data containing only 0/1). This is used in `seaborn/_core/properties.py` where `default_scale()` and `infer_scale()` call `variable_type(data, boolean_type='boolean', strict_boolean=True)` to decide whether to return a `Boolean()`, `Continuous()`, `Temporal()`, or `Nominal()` scale.\n\n3. **Safety through return type**: The `_core/rules.py` implementation wraps return values in a `UserString` subclass (`VarType`) whose `__eq__` method asserts that the comparison operand is in a set of allowed values. The `VarType` allows 5 values ('numeric', 'datetime', 'categorical', 'boolean', 'unknown').\n\n4. **Fallback to categorical**: The function's ultimate fallback is 'categorical' \u2014 if data doesn't match any recognized type, it defaults to categorical (the last `return VarType(\"categorical\")` statement).\n\n5. **All-NA special case**: All-NA data is treated as 'numeric' early in the function (before NaN dropping via `vector.dropna()`).\n\n6. **Warning suppression and TypeError handling for boolean check**: The `np.isin(vector, [0, 1])` check is wrapped in `warnings.catch_warnings()` with `warnings.simplefilter(action='ignore', category=(FutureWarning, DeprecationWarning))`, and a `try/except TypeError` block catches cases where `isin` doesn't work for certain dtypes.",
    "rubric": [
      "Explains that object-dtype containers with numeric entries are detected via an element-wise `isinstance(x_i, Number)` check (the `all_numeric` helper), which pandas' dtype API would miss",
      "Mentions that boolean/binary data (0s and 1s) receives special treatment via the `boolean_type` parameter, and `strict_boolean` controls whether only actual bool dtypes count",
      "Describes the `VarType` (UserString subclass) whose `__eq__` asserts comparisons use only allowed type names ('numeric', 'datetime', 'categorical', 'boolean', 'unknown')",
      "Notes the fallback to 'categorical' when no other type matches",
      "Mentions that all-NA data is treated as 'numeric' before dropping nulls",
      "Notes the warning suppression (FutureWarning/DeprecationWarning) and TypeError handling around the `np.isin` boolean check"
    ],
    "key_files": [
      "seaborn/_base.py",
      "seaborn/_core/rules.py",
      "seaborn/_core/properties.py"
    ],
    "source_doc": "[docstring: seaborn/_base.py] seaborn._base.variable_type\nDetermine whether a vector contains numeric, categorical, or datetime data.\n\n    This function differs from the pandas typing API in two ways:\n\n    - Python sequences or object-typed PyData objects are considered numeric if\n      all of their entries are numeric.\n    - String or mixed-type data are considered categorical even if not\n      explicitly represented as a :class:`pandas.api.types.CategoricalDtype`.\n\n    Parameters\n    ----------\n    vector : :func:`pandas.Series`, :func:`numpy.ndarray`, or Python sequence\n        Input data to test.\n    boolean_type : 'numeric' or 'categorical'\n        Type to use for vectors containing only 0s and 1s (and NAs).\n\n    Returns\n    -------\n    var_type : 'numeric', 'categorical', or 'datetime'\n        Name identifying the type of data in the vector.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes extensive implementation details (inner helper `all_numeric()`, `UserString` subclass `VariableType`/`VarType`, `strict_boolean` parameter, `np.isin` warning suppression, line numbers like 686-689) that are not present in the provided documentation chunk. These could be accurate code-level details but cannot be verified from the given doc alone.",
      "The documentation only mentions `boolean_type` accepting 'numeric' or 'categorical', but the answer claims a third option 'boolean' exists in a newer version - this is not contradicted by the doc (which describes the _base.py version) but extends beyond what's documented.",
      "The answer describes two parallel implementations (_base.py and _core/rules.py) - only the _base.py version is documented in the provided chunk.",
      "Claims about all-NA special case behavior, fallback to categorical, and warning suppression are not verifiable from the provided documentation"
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about design rationale and the truncated _base.py code needed removal or rewriting, but most functional claims are verifiable from the stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_03",
    "repo": "seaborn",
    "question": "Where does PlotData flow through the seaborn plotting system, from creation to consumption in rendering?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `PlotData` class in `seaborn/_core/data.py` serves as a central data management component that flows through the seaborn plotting system in the following way:\n\n**Initialization entry points:**\n1. In the new objects interface: `Plot.__init__()` in `plot.py` creates `self._data = PlotData(data, variables)` after resolving positional arguments and validating variable names against `PROPERTIES`.\n2. In the legacy interface: `VectorPlotter.assign_variables()` in `_base.py` imports and uses `PlotData` for data assignment when processing long-form data.\n\n**Data flow through the compilation pipeline:**\n1. `Plotter._extract_data()` takes the Plot-level `PlotData` and chains `.join()` calls: first joining facet variables (`p._facet_spec.get('variables')`), then pair variables (`p._pair_spec.get('variables')`) to form `common_data`.\n2. For each layer, `common_data.join(layer.get('source'), layer.get('vars'))` merges layer-specific data/variables with the common data. The result is stored as `spec['data']` in the `Layer` TypedDict.\n3. In `_setup_scales()`, `layer['data'].frame` and `layer['data'].frames` are used to collect variable columns across all layers for scale setup.\n4. In `_compute_stats()`, `layer['data'].frame` is directly modified (or `data.frames` is populated for paired plots).\n5. In `_plot_layer()`, `_generate_pairings()` reads from `data.frame` or `data.frames` depending on whether the frame is empty and frames exist.\n6. In `_update_legend_contents()`, `data.ids` is used to group legend entries that share the same underlying data variable (matching by `data.ids[var] == part_id`).\n\n**Key implementation details of `join()`:**\n- When `data` is None, it inherits `self.source_data` (the original data source).\n- When `variables` is falsy (None or empty dict), it inherits `self.source_vars`.\n- Variables set to `None` in the variables dict create a `disinherit` list, used to remove those columns from the resulting frame.\n- It uses `pd.concat(parts, axis=1, sort=False)` to combine the old frame (with replaced/dropped columns removed) and the new frame.\n- After joining, `new.source_data` and `new.source_vars` are overwritten with `self.source_data` and `self.source_vars`, so chained `.join()` calls always reference the original data.\n\n**`handle_data_source()` function:**\n- Accepts DataFrame, Mapping, or None directly.\n- Supports objects with a `to_pandas()` method (DataFrameProtocol) \u2014 tested via `MockConvertibleDataFrame` in conftest.\n- Wraps conversion errors in a RuntimeError with an explanatory message.\n- Raises TypeError for unsupported types (e.g., plain lists).\n\n**`PlotData` attributes beyond the documented ones:**\n- `frames: dict[tuple, DataFrame]` \u2014 initialized as empty `{}`, populated for paired plots where each variable pair gets its own DataFrame.\n- `source_data: DataSource` \u2014 stores the original data source for inheritance in `join()`.\n- `source_vars: dict[str, VariableSpec]` \u2014 stores original variable specs for inheritance in `join()`.",
    "rubric": [
      "Explains that PlotData is created in Plot.__init__() and/or VectorPlotter.assign_variables() as entry points",
      "Describes how Plotter._extract_data() chains .join() calls with facet and pair variables to form common_data",
      "Explains that each layer's data is produced by joining common_data with layer-specific source/vars",
      "Mentions that PlotData.frame is used in _setup_scales() for collecting variable columns across layers",
      "Mentions that PlotData.frame is modified in _compute_stats() or that frames dict is populated for paired plots",
      "Explains that _generate_pairings() or _plot_layer() reads from data.frame or data.frames for rendering",
      "Mentions that data.ids is used in _update_legend_contents() for grouping legend entries",
      "Describes key behavior of join(): inheriting source_data/source_vars, disinheriting None variables, using pd.concat",
      "Mentions the frames attribute (dict keyed by tuple) that is initialized empty and populated for paired plots"
    ],
    "key_files": [
      "seaborn/_core/data.py",
      "seaborn/_core/plot.py",
      "seaborn/_base.py",
      "seaborn/_core/typing.py",
      "tests/conftest.py"
    ],
    "source_doc": "[docstring: seaborn/_core/data.py] seaborn._core.data.PlotData\nseaborn._core.data.PlotData:\n  Data table with plot variable schema and mapping to original names.\n\n    Contains logic for parsing variable specification arguments and updating\n    the table with layer-specific data and/or mappings.\n\n    Parameters\n    ----------\n    data\n        Input data where variable names map to vector values.\n    variables\n        Keys are names of plot variables (x, y, ...) each value is one of:\n\n        - name of a column (or index level, or dictionary entry) in `data`\n        - vector in any format that can construct a :class:`pandas.DataFrame`\n\n    Attributes\n    ----------\n    frame\n        Data table with column names having defined plot variables.\n    names\n        Dictionary mapping plot variable names to names in source data structure(s).\n    ids\n        Dictionary mapping plot variable names to unique data source identifiers.\n\nseaborn._core.data.PlotData.__contains__:\n  Boolean check on whether a variable is defined in this dataset.\n\nseaborn._core.data.PlotData.join:\n  Add, replace, or drop variables and return as a new dataset.\n\nseaborn._core.data.PlotData._assign_variables:\n  Assign values for plot variables given long-form data and/or vector inputs.\n\n        Parameters\n        ----------\n        data\n            Input data where variable names map to vector values.\n        variables\n            Keys are names of plot variables (x, y, ...) each value is one of:\n\n            - name of a column (or index level, or dictionary entry) in `data`\n            - vector in any format that can construct a :class:`pandas.DataFrame`\n\n        Returns\n        -------\n        frame\n            Table mapping seaborn variables (x, y, color, ...) to data vectors.\n        names\n            Keys are defined seaborn variables; values are names inferred from\n            the inputs (or None when no name can be determined).\n        ids\n            Like the `names` dict, but `None` values are replaced by the `id()`\n            of the data object that defined the variable.\n\n        Raises\n        ------\n        TypeError\n            When data source is not a DataFrame or Mapping.\n        ValueError\n            When variables are strings that don't appear in `data`, or when they are\n            non-indexed vector datatypes that have a different length from `data`.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Minor doc leakage in a performance rationale comment and line number references; most claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_05",
    "repo": "seaborn",
    "question": "What is the structure and algorithm of the Dodge class used for positioning in seaborn's core moves?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `Dodge` class in `seaborn/_core/moves.py` is a `@dataclass` that inherits from the `Move` base class. Here are the key implementation details:\n\n**Class Structure:**\n- `Dodge` is a dataclass with three fields: `empty` (str, default `\"keep\"`), `gap` (float, default `0`), and `by` (Optional[list[str]], default `None`).\n- It inherits `group_by_orient: ClassVar[bool] = True` from the `Move` base class.\n\n**Algorithm in `__call__`:**\n1. It first aggregates the data using `groupby.agg(data, {\"width\": \"max\"})` to get max widths per group.\n2. If `empty=\"fill\"`, it drops NA rows from the aggregated groups.\n3. The `groupby_pos` inner function groups by the orient axis plus any `col`/`row` faceting variables that exist in the data.\n4. `scale_widths` normalizes widths within each positional group: it divides each width by the sum of all widths in that group, then multiplies by the maximum width. When `empty=\"fill\"`, missing widths are filled with 0; when `empty=\"keep\"`, they're filled with the mean width.\n5. `widths_to_offsets` computes cumulative offsets by shifting widths and centering them: `w.shift(1).fillna(0).cumsum() + (w - w.sum()) / 2`.\n6. The gap is applied *after* offset computation by multiplying new widths by `(1 - self.gap)`.\n7. The final output merges the dodged positions back to the original data via the grouping variables, renaming a temporary `_dodged` column back to the orient axis name.\n\n**Integration with Plot:**\n- In `Plotter._plot_layer()`, the `by` attribute is accessed via `getattr(move_step, \"by\", None)`. If `by` is None, it defaults to grouping properties derived from PROPERTIES.\n- The GroupBy used by Dodge is constructed with an order dict based on available scales.\n\n**GroupBy dependency:**\n- The `GroupBy` class's `agg` method uses `observed=False` in pandas groupby, producing rows for all combinations of grouping variable levels.\n- `GroupBy._get_groups` uses `categorical_order` from `seaborn/_core/rules` to determine level ordering when no explicit order is provided.",
    "rubric": [
      "Identifies Dodge as a dataclass inheriting from the Move base class",
      "Lists the three fields: empty (str, default 'keep'), gap (float, default 0), and by (Optional[list[str]], default None)",
      "Explains that it aggregates data using groupby.agg with max width",
      "Describes the empty='fill' behavior (dropping NA rows and filling missing widths with 0 vs mean for 'keep')",
      "Explains that groupby_pos groups by orient axis plus col/row faceting variables",
      "Describes scale_widths: normalizes widths by dividing by sum and multiplying by max width in each group",
      "Describes widths_to_offsets: uses shifted cumsum and centering formula (w.shift(1).fillna(0).cumsum() + (w - w.sum()) / 2)",
      "Notes that gap is applied after offset computation by multiplying widths by (1 - self.gap)",
      "Mentions that the final output merges dodged positions back via grouping variables with a temporary _dodged column"
    ],
    "key_files": [
      "seaborn/_core/moves.py",
      "seaborn/_core/plot.py",
      "seaborn/_core/groupby.py"
    ],
    "source_doc": "[docstring: seaborn/_core/moves.py] seaborn._core.moves.Dodge\nseaborn._core.moves.Dodge:\n  Displacement and narrowing of overlapping marks along orientation axis.\n\n    Parameters\n    ----------\n    empty : {'keep', 'drop', 'fill'}\n    gap : float\n        Size of gap between dodged marks.\n    by : list of variable names\n        Variables to apply the movement to, otherwise use all.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Dodge.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; a few design rationale statements and references to truncated code sections needed removal or rewriting.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_06",
    "repo": "seaborn",
    "question": "Why does the Marker class generate its default values using a two-phase strategy with hardcoded markers followed by polygon generation?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `Marker._default_values` method provides an arbitrarily long sequence of filled marker styles for use in nominal/categorical data mappings, ensuring that even with many categories, each gets a unique marker.\n\n**How the implementation works:**\n\n1. **Two-phase generation strategy**: The method uses a hardcoded list of 9 markers first (`\"o\"`, `\"X\"`, `(4, 0, 45)`, `\"P\"`, `(4, 0, 0)`, `(4, 1, 0)`, `\"^\"`, `(4, 1, 45)`, `\"v\"`), then programmatically generates additional markers from regular polygons of increasing order.\n\n2. **Polygon generation algorithm**: Starting with `s=5`, each iteration generates 4 new markers: `(s+1, 1, a)`, `(s+1, 0, a)`, `(s, 1, 0)`, `(s, 0, 0)` where `a = 360 / (s + 1) / 2`. This produces star polygons and regular polygons for side counts `s+1` (rotated) and `s` (unrotated), then increments `s`.\n\n3. **All markers are filled**: The initial list uses only filled markers. In the legacy `_base.py` code, `StyleMapping.__init__` explicitly rejects mixing filled and unfilled markers with the error `\"Filled and line art markers cannot be mixed\"`.\n\n4. **Double wrapping through `_get_values`**: `_default_values` converts specs to `MarkerStyle` objects, but the parent class `ObjectProperty._get_values` also calls `self.standardize(x)` on every returned value (where `standardize` calls `MarkerStyle(val)`), resulting in `MarkerStyle(MarkerStyle(...))`. This ensures standardization is applied uniformly to values whether they come from defaults, user-provided lists, or dicts.\n\n5. **Null value for missing data**: `Marker.null_value` is set to `MarkerStyle(\"\")` (an empty/invisible marker), used in `ObjectProperty.get_mapping()` when data values are NaN.\n\n6. **Integration with the property/scale system**: `Marker._default_values` wraps markers in `MarkerStyle` to integrate with the property/scale system where `ObjectProperty.get_mapping` returns `MarkerStyle` objects directly for use by marks like `DotBase._resolve_paths`.",
    "rubric": [
      "Explains that the initial hardcoded list of 9 markers provides well-distinguishable shapes first",
      "Describes the polygon generation algorithm (using side counts s and rotation angle a = 360/(s+1)/2) to extend beyond the initial set",
      "Notes that all generated markers are filled (no mixing of filled and unfilled markers)",
      "Explains that the method supports an arbitrarily long sequence for nominal/categorical data with many categories",
      "Mentions the double standardization: _default_values wraps in MarkerStyle, and _get_values also calls standardize(x) on each value"
    ],
    "key_files": [
      "seaborn/_core/properties.py",
      "seaborn/_base.py",
      "seaborn/_marks/dot.py",
      "tests/_core/test_properties.py"
    ],
    "source_doc": "[docstring: seaborn/_core/properties.py] seaborn._core.properties.Marker\nseaborn._core.properties.Marker:\n  Shape of points in scatter-type marks or lines with data points marked.\n\nseaborn._core.properties.Marker._default_values:\n  Build an arbitrarily long list of unique marker styles.\n\n        Parameters\n        ----------\n        n : int\n            Number of unique marker specs to generate.\n\n        Returns\n        -------\n        markers : list of string or tuples\n            Values for defining :class:`matplotlib.markers.MarkerStyle` objects.\n            All markers will be filled.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only states that _default_values builds 'an arbitrarily long list of unique marker styles' with parameter n and returns 'markers: list of string or tuples' where 'All markers will be filled.' Most of the detailed claims in the answer (specific marker lists, polygon generation algorithm, double wrapping, null_value details, legacy code references, line numbers) go far beyond what the documentation provides and cannot be verified from it.",
      "The answer claims specific implementation details like a hardcoded list of 9 markers ('o', 'X', (4, 0, 45), etc.), rotation angle calculations, and references to specific line numbers in _base.py - none of which are mentioned in the documentation.",
      "The claim about 'Double wrapping through _get_values' and MarkerStyle(MarkerStyle(...)) is entirely unsupported by the documentation.",
      "The claim about Marker.null_value being MarkerStyle('') is not mentioned in the documentation.",
      "The documentation says the return type is 'list of string or tuples' for 'defining MarkerStyle objects', but the answer claims _default_values already converts specs to MarkerStyle objects (line 433), which would contradict the documented return type."
    ],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Several claims about design rationale, comments, and a commented-out line in unseen code rely on documentation rather than stripped source.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_07",
    "repo": "seaborn",
    "question": "Where does the `LineStyle` class get registered and how does it flow through scale setup, mapping, and mark rendering?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `LineStyle` class is located in `seaborn/_core/properties.py` and is used in two entries in the `PROPERTY_CLASSES` dictionary (line 813 for 'linestyle' and line 817 for 'edgestyle'). It inherits from `ObjectProperty`, which in turn inherits from `Property`. The `LineStyle` class is instantiated as part of the `PROPERTIES` dictionary (line 834: `PROPERTIES = {var: cls(var) for var, cls in PROPERTY_CLASSES.items()}`).\n\nThe `LineStyle` class flows through the system as follows:\n\n1. **Registration**: Both 'linestyle' and 'edgestyle' properties map to `LineStyle` in the `PROPERTY_CLASSES` dict.\n\n2. **Scale setup**: In `seaborn/_core/plot.py` (line 1353), `prop = PROPERTIES[prop_key]` retrieves the LineStyle instance, which is then passed to `scale._setup(var_df[var], prop)`. For nominal/categorical data, `Nominal._setup()` in `seaborn/_core/scales.py` (line 316) calls `prop.get_mapping(new, data)` which invokes `ObjectProperty.get_mapping()`.\n\n3. **Mapping creation**: `ObjectProperty.get_mapping()` (line 354) calls `self._get_values(scale, levels)` which calls `self._default_values(n)` when `scale.values is None`, generating the dash patterns.\n\n4. **Resolution in marks**: In `seaborn/_marks/base.py`, `Mark._resolve()` handles linestyle specially \u2014 `return_array = return_multiple and not name.endswith(\"style\")` ensures that properties ending in 'style' (linestyle, edgestyle) are returned as lists rather than numpy arrays.\n\n5. **Usage in marks**: The `Path` mark (in `seaborn/_marks/line.py`) passes the resolved linestyle value directly to `mpl.lines.Line2D`'s `linestyle` parameter. The `Path._handle_capstyle()` method (line 102-108) checks if `vals[\"linestyle\"][1] is None` (meaning a solid line) and setting `dash_capstyle` to the `solid_capstyle` rc parameter.\n\n6. **Default values**: `_default_values()` starts with 5 predefined dash specs: `[\"-\", (4, 1.5), (1, 1), (3, 1.25, 1.5, 1.25), (5, 1, 1, 1)]`, then generates more using `itertools.combinations_with_replacement` with values `[3, 1.25]` and `[4, 1]`, starting with `p=3` and incrementing.\n\n7. **`_get_dash_pattern` static method**: Converts string shortcuts (using `ls_mapper = {\"-\": \"solid\", \"--\": \"dashed\", \"-.\": \"dashdot\", \":\": \"dotted\"}`) or tuples to a `(offset, dashes)` tuple format. For named styles like 'dashed', it reads from `mpl.rcParams[f\"lines.{style}_pattern\"]`. It normalizes the offset using modulo of the dash sum.",
    "rubric": [
      "Identifies that LineStyle is defined in the properties module and inherits from ObjectProperty",
      "Notes that LineStyle maps to both 'linestyle' and 'edgestyle' keys in the PROPERTY_CLASSES dictionary",
      "Explains that PROPERTIES dict instantiates LineStyle via `cls(var)` for each key",
      "Describes scale setup: the LineStyle instance is retrieved from PROPERTIES in the plot module and passed to `scale._setup()`",
      "Explains that ObjectProperty.get_mapping() calls _get_values/_default_values to generate dash patterns",
      "Mentions the 5 predefined dash specs and the programmatic generation using itertools.combinations_with_replacement",
      "Describes _get_dash_pattern converting string shortcuts or tuples to (offset, dashes) format using ls_mapper and rcParams",
      "Notes the special handling in Mark._resolve() where properties ending in 'style' return lists instead of numpy arrays",
      "Explains how Path mark passes resolved linestyle to mpl.lines.Line2D and the _handle_capstyle workaround for solid lines"
    ],
    "key_files": [
      "seaborn/_core/properties.py",
      "seaborn/_marks/line.py",
      "seaborn/_marks/base.py",
      "seaborn/_core/scales.py"
    ],
    "source_doc": "[docstring: seaborn/_core/properties.py] seaborn._core.properties.LineStyle\nseaborn._core.properties.LineStyle:\n  Dash pattern for line-type marks.\n\nseaborn._core.properties.LineStyle._default_values:\n  Build an arbitrarily long list of unique dash styles for lines.\n\n        Parameters\n        ----------\n        n : int\n            Number of unique dash specs to generate.\n\n        Returns\n        -------\n        dashes : list of strings or tuples\n            Valid arguments for the ``dashes`` parameter on\n            :class:`matplotlib.lines.Line2D`. The first spec is a solid\n            line (``\"\"``), the remainder are sequences of long and short\n            dashes.\n\nseaborn._core.properties.LineStyle._get_dash_pattern:\n  Convert linestyle arguments to dash pattern with offset.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer includes many specific code-level details (line numbers, method implementations, dictionary entries, etc.) that go far beyond what the documentation provides. These cannot be verified from the documentation alone.",
      "The documentation says _default_values returns 'list of strings or tuples' with the first spec being a solid line ('\"\"'), but the answer claims the first predefined dash spec is '-' (a dash character), not '\"\"' (empty string). This is a potential contradiction \u2014 the doc says the first spec is '\"\"' (empty string), while the answer says it starts with '-'.",
      "The detailed description of _get_dash_pattern's internals (ls_mapper dictionary, rcParams reading, offset normalization) cannot be verified from the documentation, which only says it converts linestyle arguments to dash pattern with offset.",
      "The claim about 5 predefined dash specs and the itertools.combinations_with_replacement generation logic cannot be verified from the documentation."
    ],
    "strip_verify_leakage": 0.06,
    "strip_verify_summary": "Almost all claims are verifiable from stripped code; only minor rationale explanations ('works around a matplotlib bug', 'can't be uniformly represented as arrays') relied on documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_08",
    "repo": "seaborn",
    "question": "How does the `variable_type` function determine whether a data vector is numeric, categorical, datetime, or boolean?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `variable_type` function in `seaborn/_core/rules.py` implements a multi-step type inference algorithm with the following detailed steps:\n\n**Step 1: Categorical dtype check** (line 70)\nChecks if `vector.dtype` is an instance of `pd.CategoricalDtype`. If so, immediately returns `VarType(\"categorical\")`. This takes highest priority \u2014 even if the underlying data is boolean (0s and 1s), the categorical dtype wins.\n\n**Step 2: All-NA special case** (line 74)\nIf all values in the vector are NA (using `pd.isna(vector).all()`), returns `VarType(\"numeric\")`. All-NA data defaults to numeric.\n\n**Step 3: Drop nulls** (line 78)\nCalls `vector.dropna()` to simplify subsequent checks.\n\n**Step 4: Boolean/binary detection** (lines 88-107)\nWrapped in `warnings.catch_warnings()` that suppresses `FutureWarning` and `DeprecationWarning`. Two modes:\n- **`strict_boolean=True`**: Checks the dtype directly. For `ExtensionDtype`, allows `\"bool\"` or `\"boolean\"` dtype names. For standard dtypes, only allows `\"bool\"`. This means `pd.Series([1, 0, 0])` with int dtype will NOT be treated as boolean in strict mode.\n- **`strict_boolean=False`** (default): Uses `np.isin(vector, [0, 1]).all()` wrapped in a try/except for `TypeError`. Any vector containing only 0s, 1s, and NAs is treated as boolean.\nIf boolean is detected, returns `VarType(boolean_type)` where `boolean_type` defaults to `\"numeric\"`.\n\n**Step 5: Pandas type API checks** (lines 110-114)\nUses `pd.api.types.is_numeric_dtype()` and `pd.api.types.is_datetime64_dtype()` for standard typed data.\n\n**Step 6: Element-by-element inspection** (lines 120-138)\nFor object-typed data that passed through pandas checks, defines two inner functions:\n- `all_numeric(x)`: Iterates through all elements, checks each with `isinstance(x_i, Number)` (from Python's `numbers` module).\n- `all_datetime(x)`: Iterates through all elements, checks each with `isinstance(x_i, (datetime, np.datetime64))` where `datetime` is from Python's `datetime` module.\n\n**Step 7: Final fallback** (line 142)\nIf no other condition matched (e.g., strings, mixed types), returns `VarType(\"categorical\")`.\n\n**Return type**: The function returns `VarType`, a subclass of `collections.UserString` that only allows values `\"numeric\"`, `\"datetime\"`, `\"categorical\"`, `\"boolean\"`, and `\"unknown\"`. Its `__eq__` method asserts that the comparison value is also in the allowed set, providing a safety check against typos in comparisons.\n\n**Key differences from the legacy version** in `seaborn/_base.py`:\n- The legacy `VariableType` class only allows 3 values: `\"numeric\"`, `\"datetime\"`, `\"categorical\"` (no `\"boolean\"` or `\"unknown\"`).\n- The legacy version wraps the input in `pd.Series(vector)` at the start; the new version does not.\n- The legacy version has no `strict_boolean` parameter.\n- The new version's `boolean_type` parameter accepts `\"boolean\"` as a valid option in addition to `\"numeric\"` and `\"categorical\"`.\n\n**Usage in the new API (properties.py)**: All calls in `seaborn/_core/properties.py` use `boolean_type=\"boolean\", strict_boolean=True`, which means the `Boolean()` scale is only used when data has an actual bool/Boolean dtype, not just when it contains 0s and 1s.\n\n**Usage in the legacy API (_base.py)**: The `var_types` dict in `_assign_variables_longform` uses `boolean_type=\"numeric\"` for x/y variables and `boolean_type=\"categorical\"` for other variables (like hue, size, etc.).\n\n**Internal consumer**: `categorical_order` in the same module calls `variable_type` to determine if unique categorical values should be sorted numerically (line 170-171).",
    "rubric": [
      "Mentions that CategoricalDtype is checked first and immediately returns 'categorical' regardless of underlying values",
      "Mentions the all-NA special case that defaults to 'numeric'",
      "Explains that nulls are dropped before subsequent checks",
      "Describes the strict_boolean mode checking the dtype name directly (bool/boolean) vs the non-strict mode using np.isin(vector, [0, 1]).all()",
      "Mentions the warnings.catch_warnings context suppressing FutureWarning and DeprecationWarning around the boolean check",
      "Notes that boolean_type parameter controls what type is returned for boolean data (defaults to 'numeric')",
      "Mentions the use of pd.api.types.is_numeric_dtype and is_datetime64_dtype for standard typed data",
      "Describes the element-by-element fallback using isinstance checks against Number and datetime/np.datetime64 for object-typed data",
      "States that the final fallback returns 'categorical' if nothing else matched",
      "Mentions VarType is a UserString subclass that constrains allowed values and asserts valid comparisons"
    ],
    "key_files": [
      "seaborn/_core/rules.py",
      "seaborn/_core/properties.py",
      "seaborn/_base.py",
      "tests/_core/test_rules.py"
    ],
    "source_doc": "[docstring: seaborn/_core/rules.py] seaborn._core.rules.variable_type\nDetermine whether a vector contains numeric, categorical, or datetime data.\n\n    This function differs from the pandas typing API in a few ways:\n\n    - Python sequences or object-typed PyData objects are considered numeric if\n      all of their entries are numeric.\n    - String or mixed-type data are considered categorical even if not\n      explicitly represented as a :class:`pandas.api.types.CategoricalDtype`.\n    - There is some flexibility about how to treat binary / boolean data.\n\n    Parameters\n    ----------\n    vector : :func:`pandas.Series`, :func:`numpy.ndarray`, or Python sequence\n        Input data to test.\n    boolean_type : 'numeric', 'categorical', or 'boolean'\n        Type to use for vectors containing only 0s and 1s (and NAs).\n    strict_boolean : bool\n        If True, only consider data to be boolean when the dtype is bool or Boolean.\n\n    Returns\n    -------\n    var_type : 'numeric', 'categorical', or 'datetime'\n        Name identifying the type of data in the vector.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer claims specific line numbers, internal implementation details, helper functions, class hierarchies (VarType as UserString subclass), and usage patterns in other files \u2014 none of which are verifiable from the provided documentation alone. These could be accurate code-level details or could be hallucinated.",
      "The documentation states boolean_type accepts 'numeric', 'categorical', or 'boolean' \u2014 the answer claims it accepts 'boolean' as a valid option which aligns with the doc.",
      "The documentation says the function returns 'numeric', 'categorical', or 'datetime' \u2014 the answer claims VarType also allows 'boolean' and 'unknown' values, which partially contradicts the documented return values (the doc says the return is one of the three types, not five).",
      "The answer states the return type is VarType with 5 allowed values including 'boolean' and 'unknown', but the documentation explicitly states Returns var_type as one of 'numeric', 'categorical', or 'datetime'. The 'boolean' and 'unknown' return values are not documented.",
      "The answer claims strict_boolean defaults to False and boolean_type defaults to 'numeric' \u2014 these defaults are not specified in the documentation."
    ],
    "strip_verify_leakage": 0.05,
    "strip_verify_summary": "Only one claim about numpy issue numbers was documentation-dependent; all other claims are verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_09",
    "repo": "seaborn",
    "question": "What does `categorical_order` do internally to determine value ordering, and where is it used across the seaborn codebase?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `categorical_order` function in `seaborn/_core/rules.py` determines the ordering of values. Its implementation has three branches:\n\n1. If `order` is explicitly provided (not None), it is returned directly without any processing.\n2. If the vector's dtype name is \"category\" (pandas Categorical), it extracts categories from `vector.cat.categories`.\n3. Otherwise, it gets unique values via `vector.unique()`, filters out null values using `pd.notnull`, and then checks if those values are numeric by calling `variable_type(pd.Series(order))` \u2014 if they are, it sorts them with `order.sort()`.\n\nThe function depends on `variable_type()` from the same module, which returns a `VarType` instance (a `UserString` subclass). `VarType` only allows comparisons against its 5 allowed values: \"numeric\", \"datetime\", \"categorical\", \"boolean\", \"unknown\" \u2014 using an assertion check in `__eq__`.\n\nThe function is used extensively across seaborn's architecture:\n- In `seaborn/_core/scales.py`, the `Nominal` scale's `_setup()` method calls `categorical_order(data, new.order)` to create a `units_seed` list which is then used to set up axis units via matplotlib's unit conversion system.\n- In `seaborn/_core/groupby.py`, `GroupBy._get_groups()` calls `categorical_order(data[var])` when no explicit order is given, to determine group levels.\n- In `seaborn/_core/plot.py`, `categorical_order` is imported and used for determining facet structure and generating split keys for grouping variables.\n- In `seaborn/_core/properties.py`, it is imported and used for determining levels for mapping data to visual properties.\n\nThere is also a usage of `categorical_order` in `seaborn/_base.py` where it is imported from the module and called in `HueMapping.categorical_mapping`, `SizeMapping.categorical_mapping`, and `StyleMapping.__init__` to determine category levels.",
    "rubric": [
      "Explains that if `order` is explicitly provided (not None), it is returned directly",
      "Explains that for pandas Categorical dtypes, categories are extracted from `vector.cat.categories`",
      "Explains the fallback: unique values are obtained, nulls filtered with pd.notnull, and numeric values are sorted",
      "Mentions that numeric detection uses `variable_type()` from the same module",
      "Mentions usage in scales.py (Nominal scale's _setup method)",
      "Mentions usage in groupby.py (GroupBy._get_groups for determining group levels)",
      "Mentions usage in plot.py (for facet structure or split keys)",
      "Mentions usage in properties.py (for mapping data to visual properties)",
      "Mentions usage in _base.py (HueMapping, SizeMapping, or StyleMapping)"
    ],
    "key_files": [
      "seaborn/_core/rules.py",
      "seaborn/_core/scales.py",
      "seaborn/_core/groupby.py",
      "seaborn/_core/plot.py",
      "seaborn/_base.py"
    ],
    "source_doc": "[docstring: seaborn/_core/rules.py] seaborn._core.rules.categorical_order\nReturn a list of unique data values using seaborn's ordering rules.\n\n    Parameters\n    ----------\n    vector : Series\n        Vector of \"categorical\" values\n    order : list\n        Desired order of category levels to override the order determined\n        from the `data` object.\n\n    Returns\n    -------\n    order : list\n        Ordered list of category levels not including null values.",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor issues with truncated files, line numbers, and design rationale descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_10",
    "repo": "seaborn",
    "question": "Why does the Nominal scale use a simple constant spacer and a list comprehension for membership checks instead of the dynamic spacing and pandas built-ins used elsewhere?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The Nominal scale in seaborn's `_core/scales.py` uses a Python list comprehension (`np.array([x_ in units_seed for x_ in x], bool)`) instead of pandas' `x.isin(units_seed)` or `np.isin` in its `convert_units` pipeline function.\n\nThe Nominal scale uses `_default_spacer` which unconditionally returns `1`, whereas the Continuous scale computes spacing dynamically by calling `x.dropna().unique()`, sorting, computing diffs, and finding the minimum. Since categorical data maps to consecutive integers, the Nominal scale uses the simpler constant spacer rather than the dynamic computation.\n\nAdditionally, the Nominal `CatScale` is defined as a subclass of `mpl.scale.LinearScale` with a no-op `set_default_locators_and_formatters` (body is just `...`), rather than using the base class's `_get_scale` method which creates an `InternalScale(mpl.scale.FuncScale)`. The locators/formatters are instead set via matplotlib's unit conversion system (`axis.update_units(stringify(np.array(units_seed)))`) which handles category-to-position mapping natively.",
    "rubric": [
      "Explains that the Nominal scale uses `_default_spacer` which unconditionally returns 1, because categorical data maps to consecutive integers making dynamic minimum-diff computation unnecessary",
      "Explains that the list comprehension (`[x_ in units_seed for x_ in x]`) is used instead of `x.isin()` or `np.isin` because isin fails when units_seed mixes numbers and strings (as noted in the code's TODO comment)",
      "Mentions that the Continuous scale's spacer dynamically computes minimum spacing via dropna/unique/sort/diff/min, which is needed for non-uniform numeric data but overkill for evenly-spaced categories",
      "Notes that the Nominal scale defines a CatScale subclass of LinearScale with a no-op set_default_locators_and_formatters, relying on matplotlib's unit conversion system (axis.update_units with stringified categories) for positioning instead of using FuncScale"
    ],
    "key_files": [
      "seaborn/_core/scales.py",
      "seaborn/_core/properties.py",
      "seaborn/_marks/base.py"
    ],
    "source_doc": "[docstring: seaborn/_core/scales.py] seaborn._core.scales.Nominal\nseaborn._core.scales.Nominal:\n  A categorical scale without relative importance / magnitude.\n\nseaborn._core.scales.Nominal.tick:\n  Configure the selection of ticks for the scale's axis or legend.\n\n        .. note::\n            This API is under construction and will be enhanced over time.\n            At the moment, it is probably not very useful.\n\n        Parameters\n        ----------\n        locator : :class:`matplotlib.ticker.Locator` subclass\n            Pre-configured matplotlib locator; other parameters will not be used.\n\n        Returns\n        -------\n        Copy of self with new tick configuration.\n\nseaborn._core.scales.Nominal.label:\n  Configure the selection of labels for the scale's axis or legend.\n\n        .. note::\n            This API is under construction and will be enhanced over time.\n            At the moment, it is probably not very useful.\n\n        Parameters\n        ----------\n        formatter : :class:`matplotlib.ticker.Formatter` subclass\n            Pre-configured matplotlib formatter; other parameters will not be used.\n\n        Returns\n        -------\n        scale\n            Copy of self with new tick configuration.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The generated answer discusses detailed implementation internals (list comprehensions, CatScale subclass, _default_spacer, etc.) that are not present in the provided documentation. These claims cannot be verified against the given doc chunk.",
      "The documentation only describes the Nominal scale's high-level API (tick and label methods with their parameters and return values), not any of the internal implementation details discussed in the answer.",
      "No claims in the answer directly contradict the documentation, but virtually the entire answer goes far beyond what the documentation covers, making it impossible to verify from this source alone."
    ],
    "strip_verify_leakage": 0.38,
    "strip_verify_summary": "Several claims about design rationale (why list comprehension was chosen, why it's not a performance optimization, why CatScale avoids transform overhead) rely on stripped comments and cannot be verified from code alone.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_11",
    "repo": "seaborn",
    "question": "Where is the Continuous scale class defined, and how does its class hierarchy and priority value influence orientation inference?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `Continuous` class is defined in `seaborn/_core/scales.py` and it inherits from `ContinuousBase`, which in turn inherits from `Scale`. The class hierarchy is: `Scale` \u2192 `ContinuousBase` \u2192 `Continuous`. `ContinuousBase` is also the parent of `Temporal`.\n\nThe `Continuous` class is located in `seaborn/_core/scales.py`. It has a `_priority` class variable of `1`, which is the lowest among the visible scale types (Boolean=3, Nominal=4). This priority is used in `seaborn/_marks/base.py` in `Mark._infer_orient()` to determine plot orientation \u2014 the logic `if y > x: return 'y' else: return 'x'` means higher priority makes the variable more likely to determine orientation as the dependent axis.\n\nThe `_setup` method (inherited from `ContinuousBase`) constructs a 4-stage pipeline: `[axis.convert_units, forward, normalize, prop.get_mapping(new, data)]`. The `_get_scale` method on the base `Scale` class creates an `InternalScale` class inheriting from `mpl.scale.FuncScale`. The `ContinuousBase._setup` defines a custom spacer function that returns the minimum gap between consecutive sorted unique non-NaN values.\n\nThe `Continuous.tick()` method performs input validation: with log/symlog transforms, it raises a `RuntimeError` if `count` is used without `between`, and also raises a `RuntimeError` if `every` is used. When `upto` is specified without a log transform, it uses `MaxNLocator` with custom steps `[1, 1.5, 2, 2.5, 3, 5, 10]`.\n\nThe `Continuous.label()` method uses a `Default` sentinel class (from `seaborn/_core/typing`) for the `base` parameter's default value \u2014 the use of a `Default` sentinel distinct from `None` allows the code to distinguish between the user not passing `base` and explicitly passing `None`. The `like` parameter logic checks for `\"{x\"` or `\"{pos\"` in the string to decide whether to wrap it in `{{x:{like}}}` or use it directly as a `StrMethodFormatter` format string.\n\nThe `PseudoAxis` class is used when `axis is None` (i.e., for non-coordinate properties). Its `get_tick_space()` method always returns 5.",
    "rubric": [
      "Continuous is defined in seaborn/_core/scales.py and inherits from ContinuousBase, which inherits from Scale (Scale \u2192 ContinuousBase \u2192 Continuous)",
      "ContinuousBase is also the parent of Temporal",
      "Continuous has _priority = 1, which is the lowest among visible scale types (Temporal=2, Boolean=3, Nominal=4)",
      "The _priority values are used in Mark._infer_orient() in seaborn/_marks/base.py, where higher priority makes the variable more likely to determine orientation (if y > x: return 'y', else: return 'x')",
      "ContinuousBase._setup builds a 4-stage pipeline: [axis.convert_units, forward, normalize, prop.get_mapping(new, data)]",
      "When axis is None (non-coordinate properties), a PseudoAxis is used whose get_tick_space() always returns 5",
      "Continuous.tick() raises RuntimeError if count is used without between or if every is used with log/symlog transforms",
      "Continuous.label() uses a Default sentinel (from seaborn/_core/typing) for the base parameter's default to distinguish 'not passed' from explicitly passing None",
      "The label method's like parameter checks for '{x' or '{pos' in the string to decide formatting approach"
    ],
    "key_files": [
      "seaborn/_core/scales.py",
      "seaborn/_core/properties.py",
      "seaborn/_marks/base.py",
      "seaborn/_core/plot.py",
      "seaborn/_core/typing.py"
    ],
    "source_doc": "[docstring: seaborn/_core/scales.py] seaborn._core.scales.Continuous\nseaborn._core.scales.Continuous:\n  A numeric scale supporting norms and functional transforms.\n\nseaborn._core.scales.Continuous.tick:\n  Configure the selection of ticks for the scale's axis or legend.\n\n        Parameters\n        ----------\n        locator : :class:`matplotlib.ticker.Locator` subclass\n            Pre-configured matplotlib locator; other parameters will not be used.\n        at : sequence of floats\n            Place ticks at these specific locations (in data units).\n        upto : int\n            Choose \"nice\" locations for ticks, but do not exceed this number.\n        count : int\n            Choose exactly this number of ticks, bounded by `between` or axis limits.\n        every : float\n            Choose locations at this interval of separation (in data units).\n        between : pair of floats\n            Bound upper / lower ticks when using `every` or `count`.\n        minor : int\n            Number of unlabeled ticks to draw between labeled \"major\" ticks.\n\n        Returns\n        -------\n        scale\n            Copy of self with new tick configuration.\n\nseaborn._core.scales.Continuous.label:\n  Configure the appearance of tick labels for the scale's axis or legend.\n\n        Parameters\n        ----------\n        formatter : :class:`matplotlib.ticker.Formatter` subclass\n            Pre-configured formatter to use; other parameters will be ignored.\n        like : str or callable\n            Either a format pattern (e.g., `\".2f\"`), a format string with fields named\n            `x` and/or `pos` (e.g., `\"${x:.2f}\"`), or a callable with a signature like\n            `f(x: float, pos: int) -> str`. In the latter variants, `x` is passed as the\n            tick value and `pos` is passed as the tick index.\n        base : number\n            Use log formatter (with scientific notation) having this value as the base.\n            Set to `None` to override the default formatter with a log transform.\n        unit : str or (str, str) tuple\n            Use  SI prefixes with these units (e.g., with `unit=\"g\"`, a tick value\n            of 5000 will appear as `5 kg`). When a tuple, the first element gives the\n            separator between the number and unit.\n\n        Returns\n        -------\n        scale\n            Copy of self with new label configuration.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains many specific implementation details (line numbers, class hierarchies, internal method details, priority values, pipeline stages, etc.) that are not present in the provided documentation. These cannot be verified from the given doc chunk alone.",
      "The claims about `tick()` method behavior (RuntimeError with log/symlog transforms when using count without between, RuntimeError when using every) are not contradicted by the documentation but go far beyond what the documentation states.",
      "The claims about `label()` method's `Default` sentinel class and the logic for the `like` parameter are plausible implementation details but cannot be verified from the documentation.",
      "The documentation states `like` can be 'a format pattern (e.g., \".2f\")', 'a format string with fields named x and/or pos', or 'a callable'. The answer's description of checking for '{x' or '{pos' in the string is consistent with this but adds implementation detail.",
      "The answer's description of tick() parameters and label() parameters is consistent with the documentation where they overlap."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in design rationale explanations and references to unseen files.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_12",
    "repo": "seaborn",
    "question": "How does seaborn's Mark base class resolve visual properties from data, defaults, and scales in the objects interface?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `Mark` base class in seaborn's objects interface (`seaborn/_marks/base.py`) provides a resolution mechanism for mapping data values to visual properties. Here's how it works:\n\n**`_resolve` method implementation:**\nThe `_resolve` method follows a 4-step resolution chain:\n1. **Directly specified** \u2014 If the Mark attribute was set to a concrete value (not a `Mappable`), it calls `prop.standardize(feature)` to coerce it (e.g., `Color.standardize` converts to RGB/RGBA tuples, `Marker.standardize` wraps in `MarkerStyle`). For DataFrames, the scalar is replicated to match data length and wrapped in `np.array` (unless the name ends with \"style\").\n2. **Data-mapped** \u2014 If `name` is present in `data`, it looks up the scale in the `scales` dict and calls `scale(value)` to transform it. If that fails, it raises `PlotSpecError._during(\"Scaling operation\", name)`, which generates a message like `\"Scaling operation failed for the `{name}` variable.\"`\n3. **Dependency resolution** \u2014 If the `Mappable` has a `depend` attribute set, `_resolve` calls itself recursively with the dependency's name (e.g., `edgecolor` depending on `color`).\n4. **Default fallback** \u2014 Falls back to `prop.standardize(feature.default)`, where `feature.default` either returns the `_val` or reads from `mpl.rcParams` if `_rc` is set.\n\nA special case exists for `width`: when `name == \"width\"`, even directly-specified values are treated as not directly specified if `\"width\"` is already in the data.\n\n**`Mappable` class:**\nThe `Mappable` class supports 4 initialization modes: `val` (direct value), `depend` (references another property name, validated against `PROPERTIES` dict), `rc` (rcParam name, validated against `mpl.rcParams`), and `auto` (a boolean flag). It also has a `grouping` boolean that controls whether the property contributes to data grouping via `_grouping_props`.\n\n**`_infer_orient` method:**\nOrientation is inferred by comparing `_priority` values of x and y scales. Priority values are: `Continuous=1`, `Temporal=2`, `Boolean=3`, `Nominal=4`. If y's priority exceeds x's, orient is `\"y\"`; otherwise `\"x\"`.\n\n**`_plot` method:**\nThe base class `_plot` raises `NotImplementedError`; subclasses must implement it. It receives a `split_generator` callable that yields `(keys_dict, DataFrame_subset, axes)` tuples (as seen from subclass usage patterns like `for keys, data, ax in split_gen()`). The `keep_na` parameter controls whether NaN coordinate values are preserved (used by `Path`/`Line` marks with `keep_na=not self._sort`).\n\n**Helper functions:**\n- `resolve_properties` resolves all mappable properties at once by calling `mark._resolve` for each property in `_mappable_props`.\n- `resolve_color` resolves color+alpha together \u2014 it respects alpha from RGBA values but also merges a separate `alpha` (or `{prefix}alpha`) property. It uses a `visible` helper function to detect invisible colors and set alpha to `np.nan` for them.\n- `document_properties` is a class decorator that injects a list of mappable property names into the Mark subclass's docstring.\n\n**Concrete subclass patterns:**\n- `DotBase._plot` iterates split_gen and creates `mpl.collections.PathCollection` artists.\n- `Path._plot` uses `resolve_properties` with the `keys` dict (scalars) rather than the `data` DataFrame, creating `mpl.lines.Line2D` artists.\n- `Paths._plot` aggregates segments across groups and creates `mpl.collections.LineCollection` per axis.\n- `Line` extends `Path` only by setting `_sort: ClassVar[bool] = True`.\n- The `Dot` class sets all its `Mappable` properties with `grouping=False`, while `Path`'s color/alpha/linewidth/linestyle use the default `grouping=True`.",
    "rubric": [
      "Explains the 4-step resolution chain in _resolve: directly specified, data-mapped, dependency resolution, and default fallback",
      "Mentions that directly specified values are standardized via prop.standardize() and replicated to match DataFrame length",
      "Describes that data-mapped values use the scale object and raise PlotSpecError on failure",
      "Explains Mappable's depend attribute triggers recursive _resolve call",
      "Mentions the default fallback uses feature.default which reads _val or mpl.rcParams via _rc",
      "Notes the special case for 'width' where directly_specified is overridden if width is in data",
      "Describes Mappable initialization modes: val, depend, rc, auto, and the grouping flag",
      "Explains _infer_orient compares scale _priority values (Continuous=1, Temporal=2, Boolean=3, Nominal=4)",
      "Mentions resolve_color handles color+alpha together, detecting invisible colors",
      "Describes how subclasses use _plot differently (Path uses keys dict for resolve_properties, DotBase uses data DataFrame)"
    ],
    "key_files": [
      "seaborn/_marks/base.py",
      "seaborn/_core/properties.py",
      "seaborn/_core/scales.py",
      "seaborn/_core/plot.py",
      "seaborn/_marks/dot.py",
      "seaborn/_marks/line.py",
      "seaborn/_core/exceptions.py"
    ],
    "source_doc": "[docstring: seaborn/_marks/base.py] seaborn._marks.base.Mark\nseaborn._marks.base.Mark:\n  Base class for objects that visually represent data.\n\nseaborn._marks.base.Mark._resolve:\n  Obtain default, specified, or mapped value for a named feature.\n\n        Parameters\n        ----------\n        data : DataFrame or dict with scalar values\n            Container with data values for features that will be semantically mapped.\n        name : string\n            Identity of the feature / semantic.\n        scales: dict\n            Mapping from variable to corresponding scale object.\n\n        Returns\n        -------\n        value or array of values\n            Outer return type depends on whether `data` is a dict (implying that\n            we want a single value) or DataFrame (implying that we want an array\n            of values with matching length).\n\nseaborn._marks.base.Mark._plot:\n  Main interface for creating a plot.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation only describes Mark as a base class for visually representing data, _resolve's parameters/return types, and _plot as the main interface for creating a plot. The vast majority of the generated answer contains detailed implementation claims (4-step resolution chain, Mappable class details, _infer_orient method, priority values, helper functions, concrete subclass patterns) that are not present in or supported by the provided documentation.",
      "The 4-step resolution chain (directly specified, data-mapped, dependency resolution, default fallback) is plausible but entirely unsupported by the documentation provided.",
      "Claims about Mappable class initialization modes (val, depend, rc, auto) are not in the documentation.",
      "Claims about _infer_orient and priority values (Continuous=1, Temporal=2, Boolean=3, Nominal=4) are not in the documentation.",
      "Claims about specific helper functions (resolve_properties, resolve_color, document_properties) are not in the documentation.",
      "Claims about concrete subclass patterns (DotBase, Path, Paths, Line, Dot) are not in the documentation.",
      "The special case for 'width' is not mentioned in the documentation.",
      "The PlotSpecError._during claim is not in the documentation.",
      "None of these additional claims directly contradict what the documentation does say \u2014 the doc is simply very sparse, and the answer goes far beyond it with claims that appear to be based on source code knowledge but cannot be verified from the provided doc chunk alone."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; minor doc-leakage in design rationale descriptions and references to truncated implementation details.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_13",
    "repo": "seaborn",
    "question": "What is the architecture of the Agg stat class and how does it relate to sibling classes in the aggregation module?",
    "category": "what",
    "sub_type": "architecture_exploration",
    "gold_answer": "The `Agg` class in `seaborn/_stats/aggregation.py` is a `@dataclass` that inherits from `Stat` (defined in `seaborn/_stats/base.py`). It performs aggregation on grouped data.\n\n**Key architectural details:**\n\n1. **Class hierarchy**: `Agg` extends `Stat`, which is itself a `@dataclass`. `Agg`, `Est`, and `Rolling` all inherit from `Stat`. The `Stat` base class provides a default `__call__` that simply returns data unchanged, along with helper methods `_check_param_one_of` and `_check_grouping_vars`.\n\n2. **Default function**: `Agg.func` defaults to `\"mean\"` and is typed as `str | Callable[[Vector], float]`, where `Vector` is imported from `seaborn._core.typing`.\n\n3. **group_by_orient**: `Agg` sets `group_by_orient: ClassVar[bool] = True`, overriding the base class default of `False`. This attribute is used in `seaborn/_core/plot.py` to influence grouping behavior when the stat is applied.\n\n4. **Aggregation mechanism**: `Agg.__call__` determines the value axis variable using `var = {\"x\": \"y\", \"y\": \"x\"}.get(orient)` and delegates to `GroupBy.agg()`. The `GroupBy.agg` method (in `seaborn/_core/groupby.py`) uses pandas `groupby(grouper, sort=False, observed=False).agg()` followed by `.reindex(groups)` to maintain the Cartesian product of all group level combinations.\n\n5. **Result cleaning**: After aggregation, `Agg` calls `.dropna(subset=[var])` to remove rows where the aggregated variable is null (these arise from the reindex step for group combinations not present in data), then `.reset_index(drop=True)`.\n\n6. **Contrast with Est**: The sibling class `Est` in the same module uses `GroupBy.apply` instead of `GroupBy.agg`, and delegates to either `EstimateAggregator` or `WeightedAggregator` (from `seaborn/_statistics.py`) depending on whether a \"weight\" column is present in the data. `Est` also fills NaN in min/max columns with the estimate value.\n\n7. **Rolling placeholder**: The file also contains a `Rolling(Stat)` class that is just a stub (`...` in both the class body and `__call__`).\n\n8. **Public API exposure**: `Agg` and `Est` are expected to be importable from `seaborn.objects` based on the module structure.",
    "rubric": [
      "Identifies that Agg is a @dataclass inheriting from Stat (which is also a @dataclass)",
      "Mentions that Agg.func defaults to 'mean' and accepts str or Callable[[Vector], float]",
      "Notes that Agg sets group_by_orient as a ClassVar[bool] = True, overriding the base class default of False",
      "Explains the aggregation mechanism: determines value axis via orient mapping and delegates to GroupBy.agg()",
      "Describes the result cleaning: dropna on the aggregated variable column followed by reset_index(drop=True)",
      "Contrasts with the Est sibling class which uses GroupBy.apply and delegates to EstimateAggregator or WeightedAggregator",
      "Mentions the Rolling class exists as a stub/placeholder that also inherits from Stat",
      "Notes that the Stat base class provides helper methods like _check_param_one_of and _check_grouping_vars"
    ],
    "key_files": [
      "seaborn/_stats/aggregation.py",
      "seaborn/_stats/base.py",
      "seaborn/_core/groupby.py",
      "seaborn/_core/plot.py",
      "seaborn/_statistics.py"
    ],
    "source_doc": "[docstring: seaborn/_stats/aggregation.py] seaborn._stats.aggregation.Agg\nseaborn._stats.aggregation.Agg:\n  Aggregate data along the value axis using given method.\n\n    Parameters\n    ----------\n    func : str or callable\n        Name of a :class:`pandas.Series` method or a vector -> scalar function.\n\n    See Also\n    --------\n    objects.Est : Aggregation with error bars.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Agg.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; minor rewrites needed for design rationale language and unverifiable file references.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_14",
    "repo": "seaborn",
    "question": "Why does the Est stat switch between two different aggregator classes depending on whether weight data is present?",
    "category": "why",
    "sub_type": "design_rationale",
    "gold_answer": "The `Est` stat class uses a design pattern that selects between two different aggregation engines (`EstimateAggregator` and `WeightedAggregator`) based on whether a `\"weight\"` column is present in the data. `WeightedAggregator` has more limited functionality compared to `EstimateAggregator`.\n\nSpecifically, `WeightedAggregator` explicitly restricts the estimator to only `\"mean\"` (raising a `ValueError` if anything else is passed) and the error bar method to only `\"ci\"` (raising a `ValueError` for other methods like \"sd\", \"pi\", or \"se\").\n\nThe `WeightedAggregator` computes its point estimate using `np.average(vals, weights=weights)` and for CI error bars, it passes both the values and weights as positional arguments to the `bootstrap()` function along with a closure `error_func(x, w)` that calls `np.average(x, weights=w)`. This works because `bootstrap()` resamples all positional argument arrays with the same indices, ensuring value-weight correspondence is maintained during resampling.\n\nIn contrast, `EstimateAggregator` supports callable estimators (using `self.estimator(vals)` directly), string method names via `vals.agg()`, and all four error bar methods (\"ci\", \"pi\", \"se\", \"sd\") plus callable error bar functions and `None`.\n\nThe `Est._process` method wraps the aggregator result in `pd.DataFrame([res])`. After applying `_process` through `groupby.apply`, `Est.__call__` fills NaN values in the min/max columns with the point estimate using `res.fillna({f\"{var}min\": res[var], f\"{var}max\": res[var]})`, ensuring that when no error bar is computed, the interval collapses to the point estimate rather than being missing.\n\nThe `group_by_orient` ClassVar is set to `True` for `Est`, which causes the plotting system in `_core/plot.py` to include the orient dimension in the grouping variables passed to `GroupBy`, meaning aggregation happens within each level of the orient axis.",
    "rubric": [
      "Explains that Est selects EstimateAggregator (no weights) vs WeightedAggregator (weights present) based on checking for a 'weight' column in the data",
      "Notes that WeightedAggregator restricts the estimator to only 'mean' and error bar method to only 'ci', raising ValueError otherwise, while EstimateAggregator supports multiple estimators and all error bar methods (ci, pi, se, sd, callable, None)",
      "Explains that WeightedAggregator computes its estimate with np.average(vals, weights=weights) and passes both vals and weights as positional args to bootstrap(), which resamples them with the same indices to maintain value-weight correspondence",
      "Mentions that Est.__call__ fills NaN min/max values with the point estimate so the error interval collapses to the point rather than being missing when no error bar is computed",
      "Mentions that group_by_orient is True for Est, causing the orient dimension to be included in the grouping so aggregation happens within each level of the orient axis"
    ],
    "key_files": [
      "seaborn/_stats/aggregation.py",
      "seaborn/_statistics.py",
      "seaborn/algorithms.py",
      "seaborn/_stats/base.py",
      "seaborn/_core/groupby.py"
    ],
    "source_doc": "[docstring: seaborn/_stats/aggregation.py] seaborn._stats.aggregation.Est\nseaborn._stats.aggregation.Est:\n  Calculate a point estimate and error bar interval.\n\n    For more information about the various `errorbar` choices, see the\n    :doc:`errorbar tutorial </tutorial/error_bars>`.\n\n    Additional variables:\n\n    - **weight**: When passed to a layer that uses this stat, a weighted estimate\n      will be computed. Note that use of weights currently limits the choice of\n      function and error bar method  to `\"mean\"` and `\"ci\"`, respectively.\n\n    Parameters\n    ----------\n    func : str or callable\n        Name of a :class:`numpy.ndarray` method or a vector -> scalar function.\n    errorbar : str, (str, float) tuple, or callable\n        Name of errorbar method (one of \"ci\", \"pi\", \"se\" or \"sd\"), or a tuple\n        with a method name and a level parameter, or a function that maps from a\n        vector to a (min, max) interval.\n    n_boot : int\n       Number of bootstrap samples to draw for \"ci\" errorbars.\n    seed : int\n        Seed for the PRNG used to draw bootstrap samples.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Est.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.15,
    "strip_verify_summary": "Most claims are verifiable from stripped code; only comment-based rationale explanations and a pandas issue reference need removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_15",
    "repo": "seaborn",
    "question": "Where does data flow through the `Hist` stat class, from when `__call__` is invoked to the final returned DataFrame?",
    "category": "where",
    "sub_type": "data_control_flow",
    "gold_answer": "The `Hist` class data flows through the following call chain:\n\n1. **Entry point**: `Hist.__call__` accepts `(data, groupby, orient, scales)` as its parameters, matching the `Stat` base class signature. It is called as part of the plotting pipeline.\n\n2. **Scale type determination**: In `__call__`, the scale type is derived via `scales[orient].__class__.__name__.lower()`. For example, a `Nominal` scale results in `'nominal'`, which triggers discrete binning in `_define_bin_params` (`discrete = self.discrete or scale_type == 'nominal'`).\n\n3. **Binning phase**: If `common_bins is True` or no grouping vars exist, bins are computed once across all data via `_define_bin_params`, then `groupby.apply(data, self._eval, orient, bin_kws)` is called. If `common_bins is False`, a new `GroupBy(grouping_vars)` is created and `_get_bins_and_eval` is called per group-level, computing separate bin parameters per subset.\n\n4. **GroupBy.apply**: `GroupBy.apply` (in `seaborn/_core/groupby.py`) splits data using `data.groupby(grouper, sort=False, observed=False)`, applies the function to each part, then reassembles results using `pd.concat` in the order defined by `groups` (a Cartesian product of ordered levels from `pd.MultiIndex.from_product`).\n\n5. **`_eval` method**: Calls `np.histogram(vals, **bin_kws, weights=weights, density=density)` where `density=True` only when `self.stat == 'density'`. It returns a DataFrame with columns `{orient: center, 'count': hist, 'space': width}`, where `center = edges[:-1] + width / 2`.\n\n6. **`_normalize` method**: Operates on the `'count'` column. For cumulative with `density` or `frequency` stats, it computes `(hist * data['space']).cumsum()` rather than a simple `hist.cumsum()`.\n\n7. **Final output**: The `__call__` method assigns the opposite axis variable: `data.assign(**{other: data[self.stat]})` where `other = {'x': 'y', 'y': 'x'}[orient]`.\n\n8. **`_define_bin_edges`**: Replaces both `-np.inf` and `np.inf` with `np.nan` and drops NaN before computing edges. For discrete mode, bins are `np.arange(start - .5, stop + 1.5)`. When `binwidth` is set, it converts to a bin count via `int(round((stop - start) / binwidth))`.\n\n9. **Inheritance**: `Hist` inherits from `Stat` (base class) with `group_by_orient = False` (the default from `Stat`). Since `group_by_orient` is `False`, the orient dimension would not be included in grouper logic that checks this attribute. In contrast, `Count` explicitly sets `group_by_orient = True`.",
    "rubric": [
      "Explains that `__call__` determines the scale type by getting `scales[orient].__class__.__name__.lower()` and that nominal triggers discrete binning",
      "Describes the branching: common_bins=True or no grouping vars computes bins once via `_define_bin_params` then uses `groupby.apply`; common_bins=False creates a new GroupBy and calls `_get_bins_and_eval` per group",
      "Explains that `GroupBy.apply` splits data via `data.groupby(grouper, sort=False, observed=False)`, applies the function per group, then reassembles with `pd.concat` in Cartesian-product order from `pd.MultiIndex.from_product`",
      "Describes `_eval`: calls `np.histogram` with density=True only when stat=='density', returns a DataFrame with orient center values (edges[:-1] + width/2), count, and space columns",
      "Describes `_normalize`: handles cumulative density/frequency specially via `(hist * data['space']).cumsum()` rather than simple `hist.cumsum()`",
      "Mentions the final output assigns the opposite axis via `data.assign(**{other: data[self.stat]})` where other is the cross-axis of orient",
      "Notes that `_define_bin_edges` replaces inf with nan and drops NaN; discrete mode uses `np.arange(start - .5, stop + 1.5)`; binwidth is converted to bin count via `int(round(...))`",
      "Mentions that Hist inherits from Stat with `group_by_orient = False` (the default), unlike Count which sets it to True"
    ],
    "key_files": [
      "seaborn/_stats/counting.py",
      "seaborn/_core/groupby.py",
      "seaborn/_stats/base.py",
      "seaborn/_core/plot.py"
    ],
    "source_doc": "[docstring: seaborn/_stats/counting.py] seaborn._stats.counting.Hist\nseaborn._stats.counting.Hist:\n  Bin observations, count them, and optionally normalize or cumulate.\n\n    Parameters\n    ----------\n    stat : str\n        Aggregate statistic to compute in each bin:\n\n        - `count`: the number of observations\n        - `density`: normalize so that the total area of the histogram equals 1\n        - `percent`: normalize so that bar heights sum to 100\n        - `probability` or `proportion`: normalize so that bar heights sum to 1\n        - `frequency`: divide the number of observations by the bin width\n\n    bins : str, int, or ArrayLike\n        Generic parameter that can be the name of a reference rule, the number\n        of bins, or the bin breaks. Passed to :func:`numpy.histogram_bin_edges`.\n    binwidth : float\n        Width of each bin; overrides `bins` but can be used with `binrange`.\n        Note that if `binwidth` does not evenly divide the bin range, the actual\n        bin width used will be only approximately equal to the parameter value.\n    binrange : (min, max)\n        Lowest and highest value for bin edges; can be used with either\n        `bins` (when a number) or `binwidth`. Defaults to data extremes.\n    common_norm : bool or list of variables\n        When not `False`, the normalization is applied across groups. Use\n        `True` to normalize across all groups, or pass variable name(s) that\n        define normalization groups.\n    common_bins : bool or list of variables\n        When not `False`, the same bins are used for all groups. Use `True` to\n        share bins across all groups, or pass variable name(s) to share within.\n    cumulative : bool\n        If True, cumulate the bin values.\n    discrete : bool\n        If True, set `binwidth` and `binrange` so that bins have unit width and\n        are centered on integer values\n\n    Notes\n    -----\n    The choice of bins for computing and plotting a histogram can exert\n    substantial influence on the insights that one is able to draw from the\n    visualization. If the bins are too large, they may erase important features.\n    On the other hand, bins that are too small may be dominated by random\n    variability, obscuring the shape of the true underlying distribution. The\n    default bin size is determined using a reference rule that depends on the\n    sample size and variance. This works well in many cases, (i.e., with\n    \"well-behaved\" data) but it fails in others. It is always a good to try\n    different bin sizes to be sure that you are not missing something important.\n    This function allows you to specify bins in several different ways, such as\n    by setting the total number of bins to use, the width of each bin, or the\n    specific locations where the bins should break.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Hist.rst\n\nseaborn._stats.counting.Hist._define_bin_edges:\n  Inner function that takes bin parameters as arguments.\n\nseaborn._stats.counting.Hist._define_bin_params:\n  Given data, return numpy.histogram parameters to define bins.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Most of the answer contains detailed code-level claims (line numbers, method internals, inheritance details, GroupBy behavior) that go far beyond what the provided documentation covers. These cannot be verified from the documentation alone but are plausible code-level extrapolations.",
      "The description of the `stat` parameter values and their behavior (density, frequency, etc.) is consistent with the documentation.",
      "The description of `discrete` mode setting bins to unit width centered on integer values is consistent with the documentation.",
      "The description of `common_bins` and `common_norm` behavior is consistent with the documentation's parameter descriptions.",
      "The claim about `binwidth` overriding `bins` is consistent with the documentation.",
      "Claims about specific line numbers, internal method implementations, and inheritance details cannot be verified from the provided documentation but are not contradicted by it."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; removed references to unseen files (distributions.py, objects.py) and unverifiable line numbers.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_16",
    "repo": "seaborn",
    "question": "How does the KDE stat class implement density estimation, handle edge cases, and normalize results within seaborn's stat pipeline?",
    "category": "how",
    "sub_type": "system_design",
    "gold_answer": "The `KDE` class in `seaborn/_stats/density.py` implements kernel density estimation as a stat class within seaborn's objects interface. Here's how it's designed and how it connects to the broader system:\n\n**Class hierarchy and initialization:**\n- `KDE` inherits from `Stat` (in `seaborn/_stats/base.py`), which is a `@dataclass`.\n- `Stat` defines `group_by_orient: ClassVar[bool] = False`, and `KDE` does not override this, meaning KDE does not add the orient dimension to grouping variables when called from the plotting pipeline.\n- `KDE` has a `__post_init__` method that raises a `RuntimeError` with message 'Cumulative KDE evaluation requires scipy' if `cumulative=True` and scipy is not available.\n\n**Bandwidth handling (two-step process in `_fit`):**\n1. First, `gaussian_kde` is instantiated with `bw_method` (defaults to 'scott').\n2. Then `kde.set_bandwidth(kde.factor * self.bw_adjust)` is called \u2014 this multiplies the auto-computed factor by `bw_adjust` and passes the scalar result back to `set_bandwidth`, which treats it as a constant factor.\n\n**Support grid computation (`_get_support`):**\n- If `gridsize is None`, returns the raw data points as the evaluation grid.\n- Otherwise, it fits a KDE, extracts `bw = np.sqrt(kde.covariance.squeeze())`, and computes `gridmin/gridmax` as `data.min() - bw * cut` / `data.max() + bw * cut`, creating a `np.linspace` with `gridsize` points (default 200).\n\n**Error handling:**\n- `_fit_and_evaluate` returns an empty DataFrame with columns `[orient, 'weight', 'density']` if the group has fewer than 2 data points or if a `np.linalg.LinAlgError` occurs.\n- `_transform` also catches `LinAlgError` from `_get_support` and returns empty.\n\n**Normalization design:**\n- After computing densities, the `__call__` method normalizes by computing `res['density'] *= res.eval('weight / group_weight')`. The `weight` column stores the sum of original weights for each group (from `_fit_and_evaluate`), and `group_weight` is the total weight across the normalization scope.\n- The final density is assigned to the cross-axis variable: `value = {'x': 'y', 'y': 'x'}[orient]`, and `res[value] = res['density']`.\n- Columns 'weight' and 'group_weight' are dropped before returning.\n\n**Integration into the plotting pipeline:**\n- The Stat's `__call__` is invoked with a DataFrame, a `GroupBy` object constructed from grouping variables, the orient string, and scales dict.\n- `GroupBy.apply` splits data by group keys, applies a function to each part, then reassembles results in the specified order using `pd.concat`.\n\n**Scipy fallback:**\n- If scipy is not installed, `seaborn/external/kde.py` provides a `gaussian_kde` implementation with some methods removed (notably `integrate_box_1d`, which is why cumulative mode requires scipy).\n- The fallback implementation uses numpy.linalg instead of scipy.linalg.",
    "rubric": [
      "Explains that KDE inherits from Stat (a dataclass) and that group_by_orient defaults to False and is not overridden",
      "Describes the __post_init__ check that raises RuntimeError if cumulative=True and scipy is unavailable",
      "Explains the two-step bandwidth process in _fit: gaussian_kde is instantiated with bw_method, then kde.set_bandwidth(kde.factor * bw_adjust) is called",
      "Describes _get_support: returns raw data points if gridsize is None, otherwise fits a KDE to compute bw from covariance, extends grid by bw*cut on each side, and creates a linspace of gridsize points",
      "Explains error handling: _fit_and_evaluate returns an empty DataFrame if fewer than 2 data points or LinAlgError occurs; _transform also catches LinAlgError from _get_support",
      "Describes the normalization logic in __call__: density is multiplied by weight/group_weight, where weight is the group's sum of weights and group_weight is the total across the normalization scope",
      "Explains that the final density is assigned to the cross-axis variable (x\u2192y or y\u2192x) and weight/group_weight columns are dropped",
      "Mentions the scipy fallback: seaborn/external/kde.py provides gaussian_kde without integrate_box_1d, which is why cumulative mode requires scipy"
    ],
    "key_files": [
      "seaborn/_stats/density.py",
      "seaborn/_stats/base.py",
      "seaborn/_core/groupby.py",
      "seaborn/external/kde.py",
      "seaborn/_core/plot.py"
    ],
    "source_doc": "[docstring: seaborn/_stats/density.py] seaborn._stats.density.KDE\nseaborn._stats.density.KDE:\n  Compute a univariate kernel density estimate.\n\n    Parameters\n    ----------\n    bw_adjust : float\n        Factor that multiplicatively scales the value chosen using\n        `bw_method`. Increasing will make the curve smoother. See Notes.\n    bw_method : string, scalar, or callable\n        Method for determining the smoothing bandwidth to use. Passed directly\n        to :class:`scipy.stats.gaussian_kde`; see there for options.\n    common_norm : bool or list of variables\n        If `True`, normalize so that the areas of all curves sums to 1.\n        If `False`, normalize each curve independently. If a list, defines\n        variable(s) to group by and normalize within.\n    common_grid : bool or list of variables\n        If `True`, all curves will share the same evaluation grid.\n        If `False`, each evaluation grid is independent. If a list, defines\n        variable(s) to group by and share a grid within.\n    gridsize : int or None\n        Number of points in the evaluation grid. If None, the density is\n        evaluated at the original datapoints.\n    cut : float\n        Factor, multiplied by the kernel bandwidth, that determines how far\n        the evaluation grid extends past the extreme datapoints. When set to 0,\n        the curve is truncated at the data limits.\n    cumulative : bool\n        If True, estimate a cumulative distribution function. Requires scipy.\n\n    Notes\n    -----\n    The *bandwidth*, or standard deviation of the smoothing kernel, is an\n    important parameter. Much like histogram bin width, using the wrong\n    bandwidth can produce a distorted representation. Over-smoothing can erase\n    true features, while under-smoothing can create false ones. The default\n    uses a rule-of-thumb that works best for distributions that are roughly\n    bell-shaped. It is a good idea to check the default by varying `bw_adjust`.\n\n    Because the smoothing is performed with a Gaussian kernel, the estimated\n    density curve can extend to values that may not make sense. For example, the\n    curve may be drawn over negative values when data that are naturally\n    positive. The `cut` parameter can be used to control the evaluation range,\n    but datasets that have many observations close to a natural boundary may be\n    better served by a different method.\n\n    Similar distortions may arise when a dataset is naturally discrete or \"spiky\"\n    (containing many repeated observations of the same value). KDEs will always\n    produce a smooth curve, which could be misleading.\n\n    The units on the density axis are a common source of confusion. While kernel\n    density estimation produces a probability distribution, the height of the curve\n    at each point gives a density, not a probability. A probability can be obtained\n    only by integrating the density across a range. The curve is normalized so\n    that the integral over all possible values is 1, meaning that the scale of\n    the density axis depends on the data values.\n\n    If scipy is installed, its cython-accelerated implementation will be used.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.KDE.rst\n\nseaborn._stats.density.KDE._check_var_list_or_boolean:\n  Do input checks on grouping parameters.\n\nseaborn._stats.density.KDE._fit:\n  Fit and return a KDE object.\n\nseaborn._stats.density.KDE._get_support:\n  Define the grid that the KDE will be evaluated on.\n\nseaborn._stats.density.KDE._fit_and_evaluate:\n  Transform single group by fitting a KDE and evaluating on a support grid.\n\nseaborn._stats.density.KDE._transform:\n  Transform multiple groups by fitting KDEs and evaluating.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The claim that 'KDE does not override group_by_orient' is not verifiable from the provided documentation \u2014 it is a plausible code-level detail but unsupported.",
      "The claim about __post_init__ raising RuntimeError for cumulative=True without scipy is plausible but not directly stated in the docs (the docs say 'Requires scipy' for cumulative).",
      "The detailed bandwidth handling mechanism (kde.set_bandwidth(kde.factor * self.bw_adjust)) is a plausible implementation detail but not directly stated in the docs \u2014 the docs say bw_adjust 'multiplicatively scales the value chosen using bw_method'.",
      "The claim that gridsize defaults to 200 is not stated in the documentation (docs say 'int or None' with no explicit default mentioned).",
      "The detailed normalization mechanism (res['density'] *= res.eval('weight / group_weight')) is an implementation detail not in the docs.",
      "The claim about seaborn/external/kde.py fallback implementation is plausible but not mentioned in the docs (the docs only say 'If scipy is installed, its cython-accelerated implementation will be used').",
      "The claim about violin plot usage in categorical.py is a plausible code-level detail but unsupported by the provided docs.",
      "The claim about error handling returning empty DataFrames for <2 data points or LinAlgError is plausible but not documented.",
      "The claim about GroupBy.apply behavior is a plausible implementation detail not in the provided docs."
    ],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed unverifiable categorical.py reference and minor documentation-only descriptions.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_17",
    "repo": "seaborn",
    "question": "What is the `Perc` stat class, and how does it compute and return percentile values from grouped data?",
    "category": "what",
    "sub_type": "concept_definition",
    "gold_answer": "The `Perc` class in `seaborn/_stats/order.py` is a statistical transformation that computes percentile values from observations. It is a dataclass that inherits from `Stat` (defined in `seaborn/_stats/base.py`). Key implementation details:\n\n1. **Default values**: `k` defaults to `5` (int) and `method` defaults to `\"linear\"`. When `k` is an integer, `np.linspace(0, 100, k)` is used to generate evenly-spaced percentile values.\n\n2. **Class variable `group_by_orient`**: Set to `True` (overriding the base `Stat` class default of `False`). This flag indicates that the orient dimension should be included in the grouping variables used to partition data before applying the stat.\n\n3. **Orientation handling**: The `__call__` method determines the variable to compute percentiles on using `var = {\"x\": \"y\", \"y\": \"x\"}[orient]` \u2014 i.e., it computes percentiles on the non-orient axis.\n\n4. **NumPy version compatibility**: The `_percentile` method checks if numpy version predates 1.22 using `_version_predates(np, \"1.22\")`. For older versions, it passes the method as `interpolation=method`; for newer versions it uses `method=method`.\n\n5. **NaN handling**: The `_percentile` method calls `data[var].dropna()` before computing percentiles, ensuring NaN values don't affect the computation.\n\n6. **Output format**: The `_percentile` method returns a DataFrame with two columns: the variable column containing the computed percentile values, and a `\"percentile\"` column containing the corresponding percentile levels (k values).\n\n7. **GroupBy.apply mechanism**: The `__call__` method delegates to `groupby.apply(data, self._percentile, var)`. The `GroupBy.apply` method (in `seaborn/_core/groupby.py`) splits data by group, applies the function to each part, then reassembles by assigning group identifiers back to each result part and concatenating with `pd.concat(stack, ignore_index=True)`.\n\n8. **Type casting**: The method parameter is cast to `_MethodKind` (a `Literal` type with 13 valid numpy percentile interpolation methods including 'inverted_cdf', 'averaged_inverted_cdf', 'closest_observation', 'interpolated_inverted_cdf', 'hazen', 'weibull', 'linear', 'median_unbiased', 'normal_unbiased', 'lower', 'higher', 'midpoint', and 'nearest').",
    "rubric": [
      "States that Perc is a dataclass inheriting from Stat and computes percentile values from observations",
      "Mentions that `k` defaults to 5 and `method` defaults to 'linear', and that an integer `k` produces evenly-spaced percentiles via np.linspace(0, 100, k)",
      "Notes that `group_by_orient` is set to True, overriding the base Stat default of False",
      "Explains orientation handling: percentiles are computed on the non-orient axis (e.g., orient='x' computes on 'y')",
      "Describes numpy version compatibility: checks _version_predates(np, '1.22') to decide between `interpolation` and `method` keyword for np.percentile",
      "Mentions NaN handling via dropna() before computing percentiles",
      "Explains that _percentile returns a DataFrame with the variable column (percentile values) and a 'percentile' column (the k levels)",
      "Describes how GroupBy.apply splits data by group, applies the function, assigns group identifiers back, and concatenates results with pd.concat"
    ],
    "key_files": [
      "seaborn/_stats/order.py",
      "seaborn/_stats/base.py",
      "seaborn/_core/groupby.py",
      "seaborn/_core/plot.py"
    ],
    "source_doc": "[docstring: seaborn/_stats/order.py] seaborn._stats.order.Perc\nseaborn._stats.order.Perc:\n  Replace observations with percentile values.\n\n    Parameters\n    ----------\n    k : list of numbers or int\n        If a list of numbers, this gives the percentiles (in [0, 100]) to compute.\n        If an integer, compute `k` evenly-spaced percentiles between 0 and 100.\n        For example, `k=5` computes the 0, 25, 50, 75, and 100th percentiles.\n    method : str\n        Method for interpolating percentiles between observed datapoints.\n        See :func:`numpy.percentile` for valid options and more information.\n\n    Examples\n    --------\n    .. include:: ../docstrings/objects.Perc.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.1,
    "strip_verify_summary": "Minor doc leakage in the description of Perc's purpose and an unverifiable claim about _version_predates internals; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_19",
    "repo": "seaborn",
    "question": "Where does legend management live in the grid class hierarchy, and how do the different grid classes interact with it?",
    "category": "where",
    "sub_type": "feature_location",
    "gold_answer": "The `Grid` class (in `seaborn/axisgrid.py`) is located in the class hierarchy as a subclass of `_BaseGrid` and serves as the parent for `FacetGrid`, `PairGrid`, and `ClusterGrid` (from `seaborn/matrix.py`). Notably, `JointGrid` does NOT inherit from `Grid` \u2014 it inherits directly from `_BaseGrid`, meaning it lacks legend management, `tight_layout`, `tick_params`, and `_get_palette` methods.\n\nKey implementation details:\n\n1. **`_update_legend_data`** is called inside `FacetGrid._facet_plot()` after every plot function call, and also within `PairGrid._plot_bivariate()` and `PairGrid._plot_bivariate_iter_hue()`. It extracts handles/labels from `ax.get_legend_handles_labels()` and also from `ax.legend_` (using `get_legend_handles` from `_compat.py` which handles the matplotlib 3.7 API change from `legendHandles` to `legend_handles`). After extraction, it clears the per-axis legend by setting `ax.legend_ = None`.\n\n2. **`add_legend`** uses a `mpl.patches.Patch(alpha=0, linewidth=0)` as a blank handle for labels not found in legend_data. When `_legend_out=True`, it draws a figure-level legend, calls `_draw_figure` twice (once to calculate legend width, once after resizing), increases the figure width by the legend width, and sets `subplots_adjust(right=...)`. The margin is `.04` if `_margin_titles` is True, `.01` otherwise.\n\n3. **`tick_params`** iterates over `self.figure.axes` (all axes on the figure), whereas the inherited `set()` method from `_BaseGrid` iterates over `self.axes.flat`.\n\n4. **`_extract_legend_handles`** is a flag initialized to `False` in `Grid.__init__()` and set to `True` in `PairGrid._map_bivariate()` specifically when the function is `histplot` or `kdeplot` from `seaborn.distributions`.\n\n5. **`PairGrid`** sets `_tight_layout_rect = [.01, .01, .99, .99]` and `_tight_layout_pad = layout_pad` (default 0.5), while `Grid.__init__` sets `_tight_layout_rect = [0, 0, 1, 1]` and `_tight_layout_pad = None`.\n\n6. **`ClusterGrid`** inherits from `Grid` but does NOT call `super().__init__()`, so it never initializes `_tight_layout_rect`, `_tight_layout_pad`, or `_extract_legend_handles`.\n\n7. **`_disable_autolayout`** (from `seaborn/utils.py`) is a context manager used when creating figures in `FacetGrid` (and `PairGrid`) via `with _disable_autolayout(): fig = plt.figure(...)`.",
    "rubric": [
      "Identifies that Grid inherits from _BaseGrid and is parent to FacetGrid, PairGrid, and ClusterGrid",
      "Notes that JointGrid inherits directly from _BaseGrid, not Grid, and therefore lacks legend management methods",
      "Explains that _update_legend_data is called in FacetGrid._facet_plot() and PairGrid._plot_bivariate()/_plot_bivariate_iter_hue(), extracting handles/labels and clearing per-axis legends (ax.legend_ = None)",
      "Describes add_legend's use of a blank Patch(alpha=0, linewidth=0) for missing handles and the figure resizing logic when _legend_out=True",
      "Mentions that _extract_legend_handles is a flag set to True in PairGrid._map_bivariate() specifically for histplot or kdeplot",
      "Notes that ClusterGrid inherits from Grid but does not call super().__init__(), skipping initialization of legend-related attributes"
    ],
    "key_files": [
      "seaborn/axisgrid.py",
      "seaborn/utils.py",
      "seaborn/_compat.py",
      "seaborn/matrix.py"
    ],
    "source_doc": "[docstring: seaborn/axisgrid.py] seaborn.axisgrid.Grid\nseaborn.axisgrid.Grid:\n  A grid that can have multiple subplots and an external legend.\n\nseaborn.axisgrid.Grid.tight_layout:\n  Call fig.tight_layout within rect that exclude the legend.\n\nseaborn.axisgrid.Grid.add_legend:\n  Draw a legend, maybe placing it outside axes and resizing the figure.\n\n        Parameters\n        ----------\n        legend_data : dict\n            Dictionary mapping label names (or two-element tuples where the\n            second element is a label name) to matplotlib artist handles. The\n            default reads from ``self._legend_data``.\n        title : string\n            Title for the legend. The default reads from ``self._hue_var``.\n        label_order : list of labels\n            The order that the legend entries should appear in. The default\n            reads from ``self.hue_names``.\n        adjust_subtitles : bool\n            If True, modify entries with invisible artists to left-align\n            the labels and set the font size to that of a title.\n        kwargs : key, value pairings\n            Other keyword arguments are passed to the underlying legend methods\n            on the Figure or Axes object.\n\n        Returns\n        -------\n        self : Grid instance\n            Returns self for easy chaining.\n\nseaborn.axisgrid.Grid._update_legend_data:\n  Extract the legend data from an axes object and save it.\n\nseaborn.axisgrid.Grid._get_palette:\n  Get a list of colors for the hue variable.\n\nseaborn.axisgrid.Grid.legend:\n  The :class:`matplotlib.legend.Legend` object, if present.\n\nseaborn.axisgrid.Grid.tick_params:\n  Modify the ticks, tick labels, and gridlines.\n\n        Parameters\n        ----------\n        axis : {'x', 'y', 'both'}\n            The axis on which to apply the formatting.\n        kwargs : keyword arguments\n            Additional keyword arguments to pass to\n            :meth:`matplotlib.axes.Axes.tick_params`.\n\n        Returns\n        -------\n        self : Grid instance\n            Returns self for easy chaining.",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer contains extensive implementation details (class hierarchy, line numbers, internal method behaviors, specific variable values like `.04` margins, `_tight_layout_rect` values, etc.) that are not present in the provided documentation. These cannot be verified from the doc chunk alone.",
      "The claim that JointGrid does NOT inherit from Grid is not verifiable from the provided documentation.",
      "The claim about `_BaseGrid` as a parent class is not mentioned in the documentation.",
      "Details about `_extract_legend_handles`, `_disable_autolayout`, `ClusterGrid` not calling `super().__init__()`, and `PairGrid` specific settings are all beyond the scope of the provided documentation.",
      "The claims that ARE verifiable against the documentation (e.g., `add_legend` parameters, `tick_params` parameters, `Grid` description, return types) appear consistent with the documentation."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc-leakage in explanatory rationales ('because these functions don't add proxy artists', 'to prevent auto-layout interference', 'includes auxiliary axes') and a specific line number reference; core structural claims are all verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_18",
    "repo": "seaborn",
    "question": "Why does seaborn split its grid hierarchy into `_BaseGrid` and `Grid`, and what's the purpose of the `None` check in `_BaseGrid.set()`?",
    "category": "why",
    "sub_type": "purpose_exploration",
    "gold_answer": "The `_BaseGrid` class in `seaborn/axisgrid.py` exists as a minimal base class that provides core functionality shared across all grid types. The key design decisions revealed by reading the code are:\n\n1. **The `None` check in `set()` exists specifically to handle `PairGrid` corner mode**: When `PairGrid` is created with `corner=True`, it removes upper-triangle axes by calling `axes[i, j].remove()` and then sets `axes[i, j] = None`. The `if ax is not None` guard in `_BaseGrid.set()` prevents AttributeError when iterating over the numpy array that still contains these None entries. This is confirmed by the test `test_corner_set` in `tests/test_axisgrid.py`.\n\n2. **Two-tier inheritance separating legend/layout capabilities**: `_BaseGrid` is the minimal interface (set, fig, figure, apply, pipe, savefig). `Grid(_BaseGrid)` adds legend management, `tight_layout`, `tick_params`, and palette handling. `JointGrid` inherits directly from `_BaseGrid` (not `Grid`). `FacetGrid` and `PairGrid` inherit from `Grid`, while `ClusterGrid` (in `matrix.py`) also inherits from `Grid`.\n\n3. **`savefig` defensively copies kwargs** (`kwargs = kwargs.copy()`) before calling `setdefault(\"bbox_inches\", \"tight\")` to avoid mutating the caller's dictionary - a pattern seen throughout the codebase.\n\n4. **`apply` and `pipe` behavior**: `apply` always returns `self` (enabling method chaining), while `pipe` returns the function's return value. Both pass the grid object as the first positional argument.\n\n5. **`Grid.tick_params` iterates over `self.figure.axes`** (all axes in the figure) while `_BaseGrid.set` iterates over `self.axes.flat` (only the tracked grid axes). This is a notable distinction in which axes are affected by each method.",
    "rubric": [
      "Explains that `_BaseGrid` provides minimal shared functionality (set, fig/figure, apply, pipe, savefig) while `Grid` adds legend management, tight_layout, tick_params, and palette handling",
      "Identifies that `JointGrid` inherits from `_BaseGrid` (not `Grid`), while `FacetGrid`, `PairGrid`, and `ClusterGrid` inherit from `Grid`",
      "Explains the `None` check in `set()` handles `PairGrid` corner mode where upper-triangle axes are removed and replaced with `None` in the axes array",
      "Mentions that `Grid.tick_params` iterates over `self.figure.axes` while `_BaseGrid.set` iterates over `self.axes.flat`, affecting different sets of axes",
      "Notes that `savefig` defensively copies kwargs before modifying them to avoid mutating the caller's dictionary"
    ],
    "key_files": [
      "seaborn/axisgrid.py",
      "seaborn/utils.py",
      "tests/test_axisgrid.py"
    ],
    "source_doc": "[docstring: seaborn/axisgrid.py] seaborn.axisgrid._BaseGrid\nseaborn.axisgrid._BaseGrid:\n  Base class for grids of subplots.\n\nseaborn.axisgrid._BaseGrid.set:\n  Set attributes on each subplot Axes.\n\nseaborn.axisgrid._BaseGrid.fig:\n  DEPRECATED: prefer the `figure` property.\n\nseaborn.axisgrid._BaseGrid.figure:\n  Access the :class:`matplotlib.figure.Figure` object underlying the grid.\n\nseaborn.axisgrid._BaseGrid.apply:\n  Pass the grid to a user-supplied function and return self.\n\n        The `func` must accept an object of this type for its first\n        positional argument. Additional arguments are passed through.\n        The return value of `func` is ignored; this method returns self.\n        See the `pipe` method if you want the return value.\n\n        Added in v0.12.0.\n\nseaborn.axisgrid._BaseGrid.pipe:\n  Pass the grid to a user-supplied function and return its value.\n\n        The `func` must accept an object of this type for its first\n        positional argument. Additional arguments are passed through.\n        The return value of `func` becomes the return value of this method.\n        See the `apply` method if you want to return self instead.\n\n        Added in v0.12.0.\n\nseaborn.axisgrid._BaseGrid.savefig:\n  Save an image of the plot.\n\n        This wraps :meth:`matplotlib.figure.Figure.savefig`, using bbox_inches=\"tight\"\n        by default. Parameters are passed through to the matplotlib function.",
    "verification_verdict": "warn",
    "verification_issues": [
      "Claim about specific line numbers (e.g., line 43, 98, 50-53, 1295-1296, 1453-1457) cannot be verified from the documentation provided - these are plausible code-level details but unverifiable.",
      "Claim about PairGrid corner mode and None checks in set() is not mentioned in documentation - plausible but unverifiable extrapolation.",
      "Claim about two-tier inheritance (Grid vs _BaseGrid, JointGrid inheriting from _BaseGrid, FacetGrid/PairGrid from Grid, ClusterGrid from Grid) is not documented here - plausible code-level detail but unverifiable.",
      "Claim about savefig defensively copying kwargs is not in documentation - the doc only mentions bbox_inches='tight' default and passing parameters through.",
      "Claim about fig deprecation using a 'soft strategy' with specific version timeline (v0.13 warning, eventual removal) is not supported by documentation - doc only says 'DEPRECATED: prefer the figure property'.",
      "Claim about tick_params iterating over figure.axes vs set iterating over self.axes.flat is unverifiable from this documentation.",
      "The description of apply and pipe behavior (return self vs return value, first positional argument) is well-supported by the documentation.",
      "The description of savefig using bbox_inches='tight' by default is supported by documentation."
    ],
    "strip_verify_leakage": 0.2,
    "strip_verify_summary": "Most claims are verifiable from stripped code; removed comment-based deprecation rationale and softened design intent claims to code-observable behavior.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_20",
    "repo": "seaborn",
    "question": "How does the seaborn heatmap function render a matrix, from data preparation through colormap selection, annotation, and tick label handling?",
    "category": "how",
    "sub_type": "algorithm_implementation",
    "gold_answer": "The `heatmap` function in `seaborn/matrix.py` is implemented as a thin wrapper around the `_HeatMapper` class. Here is how the algorithm works step by step:\n\n**1. Initialization (`_HeatMapper.__init__`):**\n- If `data` is a pandas DataFrame, it extracts `.values` as the plot data; otherwise it coerces to ndarray and wraps in a DataFrame for semantic labels.\n- The mask is processed by `_matrix_mask()`, which combines any user-provided mask with `pd.isnull(data)` to also mask NaN values. The mask is then applied via `np.ma.masked_where`.\n- Tick labels support 5 modes: `True` (use DataFrame index/column names), `False` (empty), `\"auto\"` (density-optimized), integer (every nth label), or a custom list. The `_skip_ticks` method generates ticks at positions `np.arange(n) + 0.5` (centering ticks in cells).\n- Axis labels are extracted from the DataFrame index/column `.name` attribute via `_index_to_label()`.\n\n**2. Colormap determination (`_determine_cmap_params`):**\n- Masked data is converted to float and filled with NaN for calculations.\n- When `robust=True`, vmin/vmax default to the 2nd and 98th percentiles (`np.nanpercentile(calc_data, 2)` and `np.nanpercentile(calc_data, 98)`).\n- Default colormap when no `center` is set: `cm.rocket`. When `center` is set: `cm.icefire`.\n- When a `cmap` is given as a list of colors, it's converted to `mpl.colors.ListedColormap(cmap)`.\n- **Divergent colormap recentering**: When `center` is not None, the algorithm computes `vrange = max(vmax - center, center - vmin)`, creates a symmetric normalization around center, then samples 256 colors from the original cmap using `np.linspace(cmin, cmax, 256)` to produce a new `ListedColormap`. It preserves the original cmap's bad/under/over color settings.\n\n**3. Drawing (`_HeatMapper.plot`):**\n- All four axes spines are removed by calling `despine(ax=ax, left=True, bottom=True)` (which combined with the default `top=True, right=True` removes all spines).\n- If a `norm` kwarg is present, vmin/vmax are NOT set on pcolormesh kwargs.\n- The actual rendering uses `ax.pcolormesh(self.plot_data, cmap=self.cmap, **kws)` \u2014 NOT imshow.\n- Y-axis is inverted with `ax.invert_yaxis()`.\n- Colorbar outline linewidth is set to 0. If `rasterized=True` was passed, the colorbar solids are also rasterized.\n- After setting tick labels, the figure is force-drawn via `_draw_figure()` (calls `fig.canvas.draw()` and falls back to `fig.draw(fig.canvas.get_renderer())` if the figure is stale). Then tick label overlap is checked, and overlapping x-labels are rotated to vertical while overlapping y-labels are rotated to horizontal.\n- Y-tick labels have `va=\"center\"` set explicitly.\n\n**4. Annotation (`_annotate_heatmap`):**\n- Text positions are at cell centers: `np.arange(width) + 0.5` for x, `np.arange(height) + 0.5` for y.\n- Masked cells are skipped (`if m is not np.ma.masked`).\n- Text color is determined by cell background luminance using `relative_luminance()`, which converts RGB to linear space with `((rgb + .055) / 1.055) ** 2.4` for values > 0.03928, then dots with `[.2126, .7152, .0722]`. The threshold is **0.408**: if luminance > 0.408, text color is `\".15\"` (dark gray), otherwise `\"w\"` (white).\n- The default annotation format string is `\".2g\"`.\n\n**5. The `heatmap()` function itself:**\n- All parameters after `data` are keyword-only (note the `*` in the signature).\n- `linewidths` and `linecolor` are mapped to the `linewidths` and `edgecolor` kwargs of `pcolormesh`.\n- Default `linewidths=0` and `linecolor=\"white\"`.\n- When `square=True`, `ax.set_aspect(\"equal\")` is called.\n- Default tick label mode is `\"auto\"` for both axes.\n- The `_auto_ticks` method calculates maximum displayable ticks as `int(size // (fontsize / 72))` where size is the axes dimension in inches.",
    "rubric": [
      "Explains that _HeatMapper wraps pandas DataFrame or coerces ndarray to DataFrame, extracting .values as plot_data",
      "Describes mask handling via _matrix_mask combining user mask with pd.isnull(data), applied with np.ma.masked_where",
      "Explains tick label modes: True (index/column names), False (empty), 'auto' (density-based), integer (every nth), or custom list, with ticks at np.arange(n) + 0.5",
      "Describes _determine_cmap_params: robust=True uses 2nd/98th nanpercentile for vmin/vmax; default cmaps are rocket (no center) and icefire (with center)",
      "Explains divergent colormap recentering: computes vrange = max(vmax-center, center-vmin), normalizes, samples 256 colors via np.linspace to create new ListedColormap",
      "Notes that rendering uses ax.pcolormesh (not imshow), with y-axis inverted via ax.invert_yaxis()",
      "Mentions spine removal via despine(ax=ax, left=True, bottom=True)",
      "Describes annotation: text at cell centers (np.arange + 0.5), skips masked cells, uses relative_luminance with 0.408 threshold to choose dark vs white text",
      "Notes that _draw_figure forces canvas draw, then overlapping tick labels are rotated (x to vertical, y to horizontal)",
      "Mentions linewidths/linecolor map to pcolormesh's linewidths/edgecolor kwargs, and square=True calls ax.set_aspect('equal')"
    ],
    "key_files": [
      "seaborn/matrix.py",
      "seaborn/utils.py",
      "seaborn/cm.py",
      "seaborn/_compat.py"
    ],
    "source_doc": "[docstring: seaborn/matrix.py] seaborn.matrix.heatmap\nPlot rectangular data as a color-encoded matrix.\n\n    This is an Axes-level function and will draw the heatmap into the\n    currently-active Axes if none is provided to the ``ax`` argument.  Part of\n    this Axes space will be taken and used to plot a colormap, unless ``cbar``\n    is False or a separate Axes is provided to ``cbar_ax``.\n\n    Parameters\n    ----------\n    data : rectangular dataset\n        2D dataset that can be coerced into an ndarray. If a Pandas DataFrame\n        is provided, the index/column information will be used to label the\n        columns and rows.\n    vmin, vmax : floats, optional\n        Values to anchor the colormap, otherwise they are inferred from the\n        data and other keyword arguments.\n    cmap : matplotlib colormap name or object, or list of colors, optional\n        The mapping from data values to color space. If not provided, the\n        default will depend on whether ``center`` is set.\n    center : float, optional\n        The value at which to center the colormap when plotting divergent data.\n        Using this parameter will change the default ``cmap`` if none is\n        specified.\n    robust : bool, optional\n        If True and ``vmin`` or ``vmax`` are absent, the colormap range is\n        computed with robust quantiles instead of the extreme values.\n    annot : bool or rectangular dataset, optional\n        If True, write the data value in each cell. If an array-like with the\n        same shape as ``data``, then use this to annotate the heatmap instead\n        of the data. Note that DataFrames will match on position, not index.\n    fmt : str, optional\n        String formatting code to use when adding annotations.\n    annot_kws : dict of key, value mappings, optional\n        Keyword arguments for :meth:`matplotlib.axes.Axes.text` when ``annot``\n        is True.\n    linewidths : float, optional\n        Width of the lines that will divide each cell.\n    linecolor : color, optional\n        Color of the lines that will divide each cell.\n    cbar : bool, optional\n        Whether to draw a colorbar.\n    cbar_kws : dict of key, value mappings, optional\n        Keyword arguments for :meth:`matplotlib.figure.Figure.colorbar`.\n    cbar_ax : matplotlib Axes, optional\n        Axes in which to draw the colorbar, otherwise take space from the\n        main Axes.\n    square : bool, optional\n        If True, set the Axes aspect to \"equal\" so each cell will be\n        square-shaped.\n    xticklabels, yticklabels : \"auto\", bool, list-like, or int, optional\n        If True, plot the column names of the dataframe. If False, don't plot\n        the column names. If list-like, plot these alternate labels as the\n        xticklabels. If an integer, use the column names but plot only every\n        n label. If \"auto\", try to densely plot non-overlapping labels.\n    mask : bool array or DataFrame, optional\n        If passed, data will not be shown in cells where ``mask`` is True.\n        Cells with missing values are automatically masked.\n    ax : matplotlib Axes, optional\n        Axes in which to draw the plot, otherwise use the currently-active\n        Axes.\n    kwargs : other keyword arguments\n        All other keyword arguments are passed to\n        :meth:`matplotlib.axes.Axes.pcolormesh`.\n\n    Returns\n    -------\n    ax : matplotlib Axes\n        Axes object with the heatmap.\n\n    See Also\n    --------\n    clustermap : Plot a matrix using hierarchical clustering to arrange the\n                 rows and columns.\n\n    Examples\n    --------\n\n    .. include:: ../docstrings/heatmap.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.08,
    "strip_verify_summary": "Most claims are directly verifiable from stripped code; only a few rationale/standards references (W3C, GitHub issue, deprecation warnings, PDF white lines) needed removal.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_21",
    "repo": "seaborn",
    "question": "What does the `_DendrogramPlotter` class do, and how does it handle linkage computation, rendering, and integration with `ClusterGrid`?",
    "category": "what",
    "sub_type": "dependency_tracing",
    "gold_answer": "The `_DendrogramPlotter` class in `seaborn/matrix.py` is a non-inheriting standalone class (it does not extend any base class) responsible for computing and drawing hierarchical clustering dendrograms. Here are the key implementation details discovered from the code:\n\n**Constructor (`__init__`)**: Takes 7 parameters: `data`, `linkage`, `metric`, `method`, `axis`, `label`, `rotate`. When `axis=1`, the data is transposed before processing. If the input is not a DataFrame, it's converted to a numpy array and then wrapped in a `pd.DataFrame`. The `self.axis` attribute is set twice in the constructor (lines 474 and 490).\n\n**Linkage calculation strategy**: The `calculated_linkage` property implements a fallback pattern \u2014 it first tries to use `fastcluster` (imported lazily inside `_calculate_linkage_fastcluster`), and if that's not installed, falls back to `scipy.cluster.hierarchy.linkage`. When falling back to scipy for large matrices (product of shape >= 10,000), it emits a warning suggesting `fastcluster` for better performance.\n\n**Fastcluster optimization**: `_calculate_linkage_fastcluster` uses `fastcluster.linkage_vector` only when `method == 'single'` OR when `metric == 'euclidean'` AND method is one of `('centroid', 'median', 'ward')`. For all other combinations, it uses the regular `fastcluster.linkage`.\n\n**Dendrogram calculation**: `calculate_dendrogram()` calls `scipy.cluster.hierarchy.dendrogram` with `no_plot=True` and `color_threshold=-np.inf`.\n\n**`reordered_ind` property**: Returns `self.dendrogram['leaves']`.\n\n**Tick positioning**: Ticks are placed at `10 * np.arange(n) + 5` (multiples of 10, offset by 5).\n\n**Plot method**: Uses `matplotlib.collections.LineCollection` for rendering the dendrogram tree. Default line width is 0.5 and default color is `(.2, .2, .2)` (dark gray). The `tree_kws` dict key `'color'` is popped and remapped to `'colors'`. Axis limits use a `1.05` multiplier on the max dependent coordinate. After plotting, `despine` is called with `bottom=True, left=True`. The method also calls `_draw_figure`, then checks for tick label overlaps using `axis_ticklabels_overlap` and rotates labels if needed.\n\n**Integration with ClusterGrid**: The module-level `dendrogram()` function instantiates `_DendrogramPlotter` and calls its `plot()` method. `ClusterGrid` (which inherits from `Grid` -> `_BaseGrid`) uses `dendrogram()` in its `plot_dendrograms` method, passing `label=False` for both row and column dendrograms in clustermap context. The `ClusterGrid.plot` method accesses `self.dendrogram_row.reordered_ind` and `self.dendrogram_col.reordered_ind` to reorder the heatmap data.\n\n**Module exports**: Only `heatmap` and `clustermap` are in `__all__`; `dendrogram` and `_DendrogramPlotter` are not publicly exported.\n\n**Scipy dependency**: The module uses a try/except import pattern setting `_no_scipy` flag; both `dendrogram()` and `ClusterGrid.__init__` raise `RuntimeError` if scipy is unavailable.",
    "rubric": [
      "Identifies _DendrogramPlotter as a standalone class (no base class) responsible for computing and drawing hierarchical clustering dendrograms",
      "Explains the constructor takes data, linkage, metric, method, axis, label, rotate; transposes data when axis=1; converts non-DataFrame input to numpy array then wraps in pd.DataFrame",
      "Describes the calculated_linkage property's fallback pattern: tries fastcluster first, falls back to scipy if not installed, warns for large matrices (shape product >= 10000)",
      "Explains _calculate_linkage_fastcluster uses fastcluster.linkage_vector only when method=='single' OR (metric=='euclidean' AND method in centroid/median/ward), otherwise uses regular fastcluster.linkage",
      "Notes calculate_dendrogram calls scipy.cluster.hierarchy.dendrogram with no_plot=True and color_threshold=-np.inf",
      "Describes the plot method uses LineCollection with default linewidth 0.5 and color (.2, .2, .2), pops 'color' key and remaps to 'colors', uses 1.05 multiplier for axis limits",
      "Mentions despine is called with bottom=True, left=True, and tick label overlap detection triggers rotation",
      "Explains ClusterGrid.plot_dendrograms passes label=False to dendrogram() for both row and column, and uses reordered_ind (which returns dendrogram['leaves']) to reorder heatmap data",
      "Notes that only heatmap and clustermap are in __all__; dendrogram and _DendrogramPlotter are not publicly exported",
      "Mentions the module uses try/except for scipy import with _no_scipy flag, raising RuntimeError if scipy is unavailable"
    ],
    "key_files": [
      "seaborn/matrix.py",
      "seaborn/utils.py",
      "seaborn/axisgrid.py",
      "tests/test_matrix.py"
    ],
    "source_doc": "[docstring: seaborn/matrix.py] seaborn.matrix._DendrogramPlotter\nseaborn.matrix._DendrogramPlotter:\n  Object for drawing tree of similarities between data rows/columns\n\nseaborn.matrix._DendrogramPlotter.__init__:\n  Plot a dendrogram of the relationships between the columns of data\n\n        Parameters\n        ----------\n        data : pandas.DataFrame\n            Rectangular data\n\nseaborn.matrix._DendrogramPlotter.calculate_dendrogram:\n  Calculates a dendrogram based on the linkage matrix\n\n        Made a separate function, not a property because don't want to\n        recalculate the dendrogram every time it is accessed.\n\n        Returns\n        -------\n        dendrogram : dict\n            Dendrogram dictionary as returned by scipy.cluster.hierarchy\n            .dendrogram. The important key-value pairing is\n            \"reordered_ind\" which indicates the re-ordering of the matrix\n\nseaborn.matrix._DendrogramPlotter.reordered_ind:\n  Indices of the matrix, reordered by the dendrogram\n\nseaborn.matrix._DendrogramPlotter.plot:\n  Plots a dendrogram of the similarities between data on the axes\n\n        Parameters\n        ----------\n        ax : matplotlib.axes.Axes\n            Axes object upon which the dendrogram is plotted",
    "verification_verdict": "warn",
    "verification_issues": [
      "The answer states that `reordered_ind` returns `self.dendrogram['leaves']`, not `self.dendrogram['reordered_ind']` as 'the docstring might suggest'. The docstring for `calculate_dendrogram` mentions that the important key-value pairing is 'reordered_ind', but the docstring for `reordered_ind` property only says 'Indices of the matrix, reordered by the dendrogram' without specifying the implementation. This is a plausible code-level detail but subtly implies the docstring is misleading.",
      "Most claims in the answer go far beyond what the documentation covers (constructor parameters, fastcluster fallback, tick positioning, LineCollection details, etc.). These are plausible code-level implementation details but cannot be verified from the provided documentation alone.",
      "The documentation for `calculate_dendrogram` says the important key-value pairing is 'reordered_ind' which indicates the re-ordering of the matrix. The answer claims the property actually returns `self.dendrogram['leaves']` \u2014 this doesn't directly contradict the doc since 'leaves' and 'reordered_ind' could be equivalent in scipy's dendrogram output, but the framing suggests a discrepancy."
    ],
    "strip_verify_leakage": 0.12,
    "strip_verify_summary": "Minor doc-leakage from comments and behavioral explanations; most claims are directly verifiable from stripped code.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_22",
    "repo": "seaborn",
    "question": "Why does seaborn's color_palette machinery include various short-circuits and clipping steps to handle numerical precision issues?",
    "category": "why",
    "sub_type": "performance",
    "gold_answer": "The `color_palette` function in seaborn contains several notable design decisions:\n\n1. **Short-circuit in `desaturate()` when `prop == 1`**: In `seaborn/utils.py`, when `prop == 1` (no desaturation), the function immediately returns the RGB value without performing the RGB\u2192HLS\u2192RGB round-trip conversion through `colorsys.rgb_to_hls` and `colorsys.hls_to_rgb`. This avoids both the computation cost and potential numerical imprecision of the round-trip.\n\n2. **`itertools.cycle` for palette extension**: When more colors are requested than exist in a palette, the function uses `itertools.cycle`: `pal_cycle = cycle(palette)` followed by `[next(pal_cycle) for _ in range(n_colors)]`. This lazily wraps the palette to produce the requested number of colors.\n\n3. **Exclusion of endpoints in continuous colormap sampling**: In `mpl_palette()`, for non-qualitative colormaps, bins are calculated as `np.linspace(0, 1, int(n_colors) + 2)[1:-1]` \u2014 excluding the 0 and 1 endpoints. This means the extreme min/max values of continuous colormaps are not sampled.\n\n4. **Fixed 256-color resolution for colormaps**: Both `hls_palette` and `husl_palette` override `n_colors` to 256 when `as_cmap=True`. Similarly, `cubehelix_palette` uses 256 samples (`np.linspace(light, dark, 256)`) for its ListedColormap.\n\n5. **`np.clip` for HUSL color space numerical safety**: In `_color_to_rgb()`, after converting from HUSL to RGB via `husl.husl_to_rgb()`, the result is clipped to [0, 1] with `tuple(np.clip(color, 0, 1))`. The HUSL\u2192RGB conversion goes through LCH\u2192LUV\u2192XYZ\u2192RGB (as seen in husl.py), and clipping ensures values remain in the valid range.\n\n6. **`QUAL_PALETTE_SIZES` dictionary lookup for default n_colors**: The code uses `QUAL_PALETTE_SIZES.get(palette, 6)` to determine the default number of colors. This dict is pre-computed at module load time by combining `MPL_QUAL_PALS` and lengths from `SEABORN_PALETTES` via `{k: len(v) for k, v in SEABORN_PALETTES.items()}`.",
    "rubric": [
      "Mentions that desaturate() short-circuits when prop == 1 to avoid the RGB\u2192HLS\u2192RGB round-trip conversion and its potential numerical imprecision",
      "Explains that itertools.cycle is used to lazily extend palettes when more colors are requested than exist in the palette",
      "Notes that continuous colormap sampling in mpl_palette excludes endpoints (0 and 1) via np.linspace(0, 1, n_colors + 2)[1:-1]",
      "Mentions the fixed 256-color resolution used when generating colormaps (hls_palette, husl_palette, or cubehelix_palette use 256 samples)",
      "Explains that np.clip is applied after HUSL-to-RGB conversion to ensure values stay in [0, 1] due to potential out-of-range results from the color space transform",
      "Describes how QUAL_PALETTE_SIZES dictionary provides default n_colors values via a pre-computed lookup combining MPL_QUAL_PALS and SEABORN_PALETTES lengths"
    ],
    "key_files": [
      "seaborn/palettes.py",
      "seaborn/utils.py",
      "seaborn/_compat.py",
      "seaborn/external/husl.py"
    ],
    "source_doc": "[docstring: seaborn/palettes.py] seaborn.palettes.color_palette\nReturn a list of colors or continuous colormap defining a palette.\n\n    Possible ``palette`` values include:\n        - Name of a seaborn palette (deep, muted, bright, pastel, dark, colorblind)\n        - Name of matplotlib colormap\n        - 'husl' or 'hls'\n        - 'ch:<cubehelix arguments>'\n        - 'light:<color>', 'dark:<color>', 'blend:<color>,<color>',\n        - A sequence of colors in any format matplotlib accepts\n\n    Calling this function with ``palette=None`` will return the current\n    matplotlib color cycle.\n\n    This function can also be used in a ``with`` statement to temporarily\n    set the color cycle for a plot or set of plots.\n\n    See the :ref:`tutorial <palette_tutorial>` for more information.\n\n    Parameters\n    ----------\n    palette : None, string, or sequence, optional\n        Name of palette or None to return current palette. If a sequence, input\n        colors are used but possibly cycled and desaturated.\n    n_colors : int, optional\n        Number of colors in the palette. If ``None``, the default will depend\n        on how ``palette`` is specified. Named palettes default to 6 colors,\n        but grabbing the current palette or passing in a list of colors will\n        not change the number of colors unless this is specified. Asking for\n        more colors than exist in the palette will cause it to cycle. Ignored\n        when ``as_cmap`` is True.\n    desat : float, optional\n        Proportion to desaturate each color by.\n    as_cmap : bool\n        If True, return a :class:`matplotlib.colors.ListedColormap`.\n\n    Returns\n    -------\n    list of RGB tuples or :class:`matplotlib.colors.ListedColormap`\n\n    See Also\n    --------\n    set_palette : Set the default color cycle for all plots.\n    set_color_codes : Reassign color codes like ``\"b\"``, ``\"g\"``, etc. to\n                      colors from one of the seaborn palettes.\n\n    Examples\n    --------\n\n    .. include:: ../docstrings/color_palette.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.25,
    "strip_verify_summary": "Most claims are verifiable from code structure, but design rationale phrases (e.g., 'avoid floating point issues', 'balance between fidelity and memory') come from comments/documentation.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_24",
    "repo": "seaborn",
    "question": "How does `set_theme` coordinate context, style, and palette configuration, and how do the rc override behaviors differ at each stage?",
    "category": "how",
    "sub_type": "api_framework_support",
    "gold_answer": "The `set_theme` function in `seaborn/rcmod.py` orchestrates the visual theming by delegating to three separate functions in a specific order:\n\n1. **`set_context(context, font_scale)`** \u2014 Called first, it invokes `plotting_context()` which returns a `_PlottingContext` object (a dict subclass), then applies it via `mpl.rcParams.update()`. The base context is \"notebook\" with specific scaling factors: paper=0.8, notebook=1, talk=1.5, poster=2. The notebook base font.size is 12, axes.linewidth is 1.25, lines.linewidth is 1.5, and lines.markersize is 6. Font scaling is applied independently on top of context scaling \u2014 font parameters are first scaled by the context multiplier, then multiplied again by `font_scale`.\n\n2. **`set_style(style, rc={\"font.family\": font})`** \u2014 Called second, passing the `font` parameter as an rc override specifically for \"font.family\". The `axes_style()` function builds the style dict from one of 5 presets: \"white\", \"dark\", \"whitegrid\", \"darkgrid\", \"ticks\". The default is \"darkgrid\", which sets axes.facecolor to \"#EAEAF2\" and axes.edgecolor to \"white\". The style dict is filtered to only include keys in the `_style_keys` list, and the rc override is also filtered the same way \u2014 meaning only style-related keys from the rc dict are applied. The result is wrapped in an `_AxesStyle` object before being applied.\n\n3. **`set_palette(palette, color_codes=color_codes)`** \u2014 Called third, it uses `palettes.color_palette()` to resolve the palette, wraps the colors in a `cycler('color', colors)` object, and sets `mpl.rcParams['axes.prop_cycle']`. When `color_codes=True` (the default in `set_theme`), it calls `palettes.set_color_codes(palette)`, which maps the shorthand codes \"b\", \"g\", \"r\", \"m\", \"y\", \"c\", \"k\" to the first 7 colors of the palette's \"6\" variant (6 colors + dark gray (.1,.1,.1)). This is wrapped in a try/except that silently catches ValueError and TypeError, so non-seaborn palettes just skip color code remapping.\n\n4. **Finally**, if the user passed an `rc` dict, it is applied last via `mpl.rcParams.update(rc)` with NO filtering \u2014 unlike the rc overrides in `set_style` and `set_context` which are filtered to their respective key lists. This means the final `rc` parameter can override any matplotlib rcParam.\n\nThe default image colormap set by style presets is \"rocket\". The default sans-serif font stack is [\"Arial\", \"DejaVu Sans\", \"Liberation Sans\", \"Bitstream Vera Sans\", \"sans-serif\"]. The palette \"jet\" raises `ValueError(\"No.\")` preventing its use.\n\nBoth `_AxesStyle` and `_PlottingContext` inherit from `_RCAesthetics(dict)`, which supports use as a context manager (`with` statement) and as a decorator (`__call__` wraps a function with `functools.wraps`). The SEABORN_PALETTES dictionary defines 12 named palettes (6 base palettes each with a 10-color and 6-color variant): deep, muted, pastel, bright, dark, and colorblind.\n\nThe `set` function is defined as a simple alias for `set_theme`.",
    "rubric": [
      "Explains that set_theme calls set_context, set_style, and set_palette in that specific order, then applies the final rc dict",
      "Describes that set_context delegates to plotting_context which scales a notebook base (font.size=12, axes.linewidth=1.25, etc.) by context multipliers (paper=0.8, notebook=1, talk=1.5, poster=2), with font_scale applied independently on top",
      "Describes that set_style delegates to axes_style which supports 5 presets (white, dark, whitegrid, darkgrid, ticks), and that the font parameter is passed as rc={'font.family': font}",
      "Notes that rc overrides in axes_style and plotting_context are filtered to only their respective _style_keys/_context_keys lists",
      "Notes that the final rc dict in set_theme is applied unfiltered via mpl.rcParams.update, meaning it can override any rcParam",
      "Explains that set_palette uses palettes.color_palette() and sets axes.prop_cycle via a cycler, and that color_codes=True triggers set_color_codes which remaps shorthand codes (b,g,r,m,y,c,k) to the palette's 6-color variant plus (.1,.1,.1), with errors silently caught",
      "Mentions that _AxesStyle and _PlottingContext inherit from _RCAesthetics(dict) enabling use as context managers and decorators",
      "Notes that 'set' is a simple alias for set_theme"
    ],
    "key_files": [
      "seaborn/rcmod.py",
      "seaborn/palettes.py",
      "seaborn/__init__.py"
    ],
    "source_doc": "[docstring: seaborn/rcmod.py] seaborn.rcmod.set_theme\nSet aspects of the visual theme for all matplotlib and seaborn plots.\n\n    This function changes the global defaults for all plots using the\n    matplotlib rcParams system. The themeing is decomposed into several distinct\n    sets of parameter values.\n\n    The options are illustrated in the :doc:`aesthetics <../tutorial/aesthetics>`\n    and :doc:`color palette <../tutorial/color_palettes>` tutorials.\n\n    Parameters\n    ----------\n    context : string or dict\n        Scaling parameters, see :func:`plotting_context`.\n    style : string or dict\n        Axes style parameters, see :func:`axes_style`.\n    palette : string or sequence\n        Color palette, see :func:`color_palette`.\n    font : string\n        Font family, see matplotlib font manager.\n    font_scale : float, optional\n        Separate scaling factor to independently scale the size of the\n        font elements.\n    color_codes : bool\n        If ``True`` and ``palette`` is a seaborn palette, remap the shorthand\n        color codes (e.g. \"b\", \"g\", \"r\", etc.) to the colors from this palette.\n    rc : dict or None\n        Dictionary of rc parameter mappings to override the above.\n\n    Examples\n    --------\n\n    .. include:: ../docstrings/set_theme.rst",
    "verification_verdict": "warn",
    "verification_issues": [
      "The documentation does not mention specific implementation details like `_PlottingContext`, `_AxesStyle`, `_RCAesthetics`, or their inheritance structure \u2014 these are plausible code-level details but are not verifiable from the provided documentation alone.",
      "The documentation does not mention specific default values like 'notebook' base font.size of 12, axes.linewidth of 1.25, context scaling factors (paper=0.8, notebook=1, talk=1.5, poster=2), or the default style being 'darkgrid' with axes.facecolor '#EAEAF2' \u2014 these are plausible but unverifiable from the given doc.",
      "The documentation does not mention the 'rocket' colormap default, the specific sans-serif font stack, or the 'jet' palette Easter egg \u2014 these are plausible code-level details but entirely unsupported by the provided documentation.",
      "The documentation does not mention the order of internal function calls (set_context, set_style, set_palette), filtering behavior of rc keys, or the unfiltered nature of the final rc parameter \u2014 these are plausible implementation details but not verifiable from the doc.",
      "The documentation does not describe color_codes defaulting to True in set_theme \u2014 the doc says 'If True and palette is a seaborn palette, remap...' but doesn't specify the default value. This could be a hallucination if the actual default differs.",
      "The claim about SEABORN_PALETTES containing 12 named palettes (6 base \u00d7 2 variants) is not mentioned in the documentation.",
      "The claim that `set` is an alias for `set_theme` that may be removed in the future is not mentioned in the provided documentation."
    ],
    "strip_verify_leakage": 0.06,
    "strip_verify_summary": "Nearly all claims are directly verifiable from stripped code; minor removals for external knowledge claims and documentation-only notes.",
    "generation_status": "completed",
    "generation_error": null
  },
  {
    "id": "seaborn_gen_23",
    "repo": "seaborn",
    "question": "Where is `hls_palette` defined and how does it get invoked within the palette system?",
    "category": "where",
    "sub_type": "identifier_location",
    "gold_answer": "The `hls_palette` function is defined in `seaborn/palettes.py` at line 258. It is called from within the `color_palette` function (line 195) when the palette argument is the string `\"hls\"`. The function uses Python's built-in `colorsys.hls_to_rgb` to convert HLS values to RGB tuples. Its default parameters are `n_colors=6, h=.01, l=.6, s=.65, as_cmap=False`. When `as_cmap=True`, it generates 256 colors and returns a `mpl.colors.ListedColormap` with the name `\"hls\"`. Otherwise, it returns a `_ColorPalette` instance (which subclasses `list`). The hues are computed using `np.linspace(0, 1, int(n_colors) + 1)[:-1]`, offset by `h`, and taken modulo 1. The `_ColorPalette` class (line 61) supports context manager usage (`__enter__`/`__exit__`) which temporarily sets the matplotlib color cycle via `seaborn.rcmod.set_palette`, and has an `as_hex()` method and `_repr_html_()` method that generates an SVG with colored rectangles for display. In contrast, `husl_palette` uses `seaborn.external.husl.husl_to_rgb` (via the `_color_to_rgb` helper) and scales its h values to 0-359 and s/l to 0-99 before conversion, while `hls_palette` directly passes 0-1 values to `colorsys.hls_to_rgb`.",
    "rubric": [
      "Identifies that hls_palette is defined in seaborn/palettes.py",
      "Explains that color_palette calls hls_palette when the palette argument is the string 'hls'",
      "Mentions that hls_palette uses colorsys.hls_to_rgb for color conversion",
      "Lists the default parameters (n_colors=6, h=.01, l=.6, s=.65, as_cmap=False)",
      "Explains that when as_cmap=True, it generates 256 colors and returns a ListedColormap",
      "Explains that otherwise it returns a _ColorPalette instance",
      "Describes how hues are computed using np.linspace(0, 1, n_colors + 1)[:-1] offset by h and taken modulo 1",
      "Mentions that _ColorPalette supports context manager usage for temporarily setting the color cycle",
      "Notes the difference from husl_palette which uses husl_to_rgb and scales values differently"
    ],
    "key_files": [
      "seaborn/palettes.py",
      "seaborn/external/husl.py",
      "seaborn/rcmod.py"
    ],
    "source_doc": "[docstring: seaborn/palettes.py] seaborn.palettes.hls_palette\nReturn hues with constant lightness and saturation in the HLS system.\n\n    The hues are evenly sampled along a circular path. The resulting palette will be\n    appropriate for categorical or cyclical data.\n\n    The `h`, `l`, and `s` values should be between 0 and 1.\n\n    .. note::\n        While the separation of the resulting colors will be mathematically\n        constant, the HLS system does not construct a perceptually-uniform space,\n        so their apparent intensity will vary.\n\n    Parameters\n    ----------\n    n_colors : int\n        Number of colors in the palette.\n    h : float\n        The value of the first hue.\n    l : float\n        The lightness value.\n    s : float\n        The saturation intensity.\n    as_cmap : bool\n        If True, return a matplotlib colormap object.\n\n    Returns\n    -------\n    palette\n        list of RGB tuples or :class:`matplotlib.colors.ListedColormap`\n\n    See Also\n    --------\n    husl_palette : Make a palette using evenly spaced hues in the HUSL system.\n\n    Examples\n    --------\n    .. include:: ../docstrings/hls_palette.rst",
    "verification_verdict": "pass",
    "verification_issues": [],
    "strip_verify_leakage": 0.03,
    "strip_verify_summary": "Nearly all claims are directly verifiable from the stripped code; only a minor characterization of _repr_html_ needed slight rewording.",
    "generation_status": "completed",
    "generation_error": null
  }
]