# Migrating from version 2 to version 3 Find the version 2 import, class, method, argument, or behavior in the table. Rows are sorted alphabetically. Names are relative to `pytorch_metric_learning` unless another package is named. A removed class's direct module import follows the same replacement as its package export. Version 3 has no compatibility aliases. The [release notes](release-notes/3.0.0.md) explain the changes and corrected numerical behavior; the examples below run in the test suite. | v2 symbol or call | v3 replacement | Note | | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A non-`DivisorReducer` on `ProxyAnchorLoss`, `MarginLoss`, or `SparseCentersRegularizer` | `DivisorReducer`, or `MultipleReducers` routing those terms to it | Terms with a `divisor` raise `ValueError` in other reducers. | | A subclass's automatically discovered `calculate_*` method | `AccuracyCalculator(custom_metrics={"name": RetrievalMetric(function, neighbors=...)})` | `function(inputs: RetrievalInputs)` returns one value per query (NaN where undefined); the calculator drops lone queries and reduces. See [custom accuracy metrics](accuracy_calculation.md#adding-custom-accuracy-metrics). | | `accuracy_calculator.get_label_match_counts` / `get_lone_query_labels` | `RetrievalInputs.relevant_counts` in a custom metric | Counts honor custom comparators; the calculator excludes lone queries before reducing. | | `accuracy_calculator.precision_at_k(knn_labels, gt_labels, k, avg_of_avgs, return_per_class, label_comparison_fn)` | `evaluation.precision_at_k(inputs, k)` | Returns per-query float64 values; reduction belongs to the calculator. | | `accuracy_calculator.r_precision` / `mean_average_precision` / `mean_reciprocal_rank(..., avg_of_avgs, return_per_class, ...)` | `evaluation.r_precision(inputs)`, `evaluation.mean_average_precision(inputs)`, `evaluation.mean_average_precision_at_r(inputs)`, `evaluation.mean_reciprocal_rank(inputs)` | Per-query float64 values. | | `AccuracyCalculator` fixed `k` larger than a gallery | `k` capped at the available neighbors of each call | The configured `k` is unchanged, so one calculator serves galleries of different sizes. | | `AccuracyCalculator()` with implicit Faiss | Default PyTorch squared-Euclidean search | Pass `knn_func=FaissKNN()` explicitly to retain Faiss; tie order may differ. | | `AccuracyCalculator(device=None)` selecting CUDA | The query tensor's device | Move query/reference tensors explicitly or pass `device`. | | `AccuracyCalculator(label_comparison_fn=..., include=("NMI",))` raising `NotImplementedError` | `ValueError` | Clustering metrics still require the default comparison. | | `AccuracyCalculator(return_per_class=True)` | `calculator.get_accuracy_per_class(...)` | Returns per-class values keyed by label, with NaN for classes without relevant references. | | `AccuracyCalculator.description()` | Application-owned metric/run labels | Read `avg_of_avgs` if the application needs that setting in its label. | | `AccuracyCalculator.get_accuracy` casting labels to float32 | Labels keep their dtype | Large integer labels no longer collide; embeddings keep their floating dtype. | | `AccuracyCalculator.get_accuracy` returning lists for per-class metrics | `get_accuracy` returning `dict[str, float]` | Per-class values come only from `get_accuracy_per_class`. | | `AccuracyCalculator.get_curr_metrics()` | `calculator.metrics` | A tuple of the metric names selected at construction. | | `AccuracyCalculator.requires_knn()` / `requires_clustering()` overrides | The registration type: `RetrievalMetric` or `ClusteringMetric(function)` | `ClusteringMetric` functions take `(query_labels, cluster_labels)` and return a float. | | `AngularLoss.set_stats` | Compute diagnostics in caller code | No recorded `avg_angle` state remains. | | `AngularMiner(angle=...)` | `AngularMiner(alpha=...)` | Degrees, like `AngularLoss(alpha=...)`; angles use normalized embeddings. | | `AngularMiner.set_stats` | Diagnostics computed from returned indices | No stored miner statistics remain. | | `ArcFaceLoss(margin=28.6)` and `SubCenterArcFaceLoss(margin=28.6)` in degrees | `margin=0.5` in radians | The paper's unit; convert with `math.radians`. | | `ArcFaceLoss.cast_types` | `.to(device, dtype)` | Placement belongs to the caller. | | Automatic parameter movement/casting during `forward` | `loss_func.to(device, dtype)` before use | Include loss parameters in the optimizer; see the example below. | | `BaseDistance(is_inverted=True)` | Set `is_inverted = True` on the custom similarity class | The constructor no longer accepts this flag; `p` still controls normalization. | | `BaseDistance(normalize_embeddings, p, power)` positionally | Keyword-only options | For example `LpDistance(normalize_embeddings=False, power=2)`. | | `BaseDistance.set_default_stats` | Application diagnostics | No recorded norm statistics remain. | | `BaseDistance.smallest_dist` | `torch.minimum`/`torch.maximum` chosen by `is_inverted` | `BaseDistance.pairwise(query, ref)` computes normalized, powered row-wise values. | | `BaseMetricLossFunction.all_regularization_loss_names()` | `regularization_terms(embeddings)` | Regularization is composed explicitly after objective reduction. | | `BaseMetricLossFunction.get_default_distance()` / `get_default_reducer()` | `default_distance()` / `default_reducer()` | The same applies to miners and regularizers. | | `BaseMetricLossFunction.sub_loss_names` / `zero_losses(embeddings)` | Return `{}` when there is nothing to compute | Reducers sum an empty dictionary to a differentiable zero. | | `BaseMiner.output_assertion` | Validation in `BaseMiner.forward` | Custom miners implement `mine(embeddings, labels, indices_tuple, ref_emb, ref_labels)`; see [custom miners](extend/miners.md). | | `BaseReducer.forward` requiring exactly one term | `forward` reduces every term and sums them | An empty term dictionary reduces to a differentiable zero; `MultipleReducers` chooses a reducer per term name. | | `BaseReducer.pos_pair_reduction`, `neg_pair_reduction`, `triplet_reduction` | `element_reduction(losses, anchors, embeddings, labels)` | Every term is reduced through its anchor indices. | | `BaseReducer.set_losses_size_stat` | Compute counts from the loss dictionary | No recorded reducer statistics remain. | | `BatchEasyHardMiner.get_func_for_stats` | Explicit application reductions | Choose `min`, `max`, or `mean` for the desired diagnostic. | | `BatchEasyHardMiner.HARD`, `SEMIHARD`, `EASY`, `ALL` | The strings `"hard"`, `"semihard"`, `"easy"`, `"all"` | Typed as `types.MiningStrategy`. | | `BatchEasyHardMiner.set_stats` | Diagnostics computed from returned indices | No stored miner statistics remain. | | `BatchedDistance(..., iter_fn=callback)` | Iterate over `distance(query, ref)` | Each item is `(mat, start, end)`; no callback is involved. | | Catching `AssertionError` for invalid arguments | Catch `TypeError` or `ValueError` | Validation remains active under `python -O`. | | `CenterInvariantRegularizer.compute_loss(weights=...)` | `compute_loss(x=...)` | Prefer calling the regularizer on the tensor directly. | | `CircleLoss(m=...)` | `CircleLoss(margin=...)` | Same value and default (0.4). | | `ClassWeightedReducer` moving weights to the losses' device | `reducer.to(device)` | A device mismatch raises `ValueError`. | | `ClassWeightedReducer.element_reduction_helper` | `element_reduction(losses, anchors, embeddings, labels)` | Every term is reduced through its anchor indices. | | `collect_stats=True` on losses, miners, distances, reducers, or regularizers | Compute diagnostics from returned tensors or use PyTorch forward hooks | Recorded statistic attributes were removed; see [statistics](release-notes/3.0.0rc1.md#removed). | | `CosFaceLoss.cast_types` | `.to(device, dtype)` | Placement belongs to the caller. | | `CosineSimilarity(normalize_embeddings=..., p=..., power=...)` | `CosineSimilarity()`, or `DotProductSimilarity(normalize_embeddings=True, p=..., power=...)` for other norms or powers | Losses and regularizers that require `CosineSimilarity` rely on signed unit-L2 cosines, so it takes no options. | | `CrossBatchMemory` checkpoint `_extra_state` | A v3 checkpoint with `queue_state` | Old queue metadata is not loaded through a compatibility path. | | `CrossBatchMemory` moving its queue during `forward` | Place the wrapper with `.to(device, dtype)` | `label_dtype` is keyword-only and defaults to `torch.long`. | | `CrossBatchMemory` storing labels after `torch.can_cast` | Exact label conversion to `label_dtype` | Labels that change value, such as `2**32` in `int32`, raise `ValueError`. | | `CrossBatchMemory.add_to_memory` / `curr_batch_idx` | `CrossBatchMemory.forward` | The queue is updated only after the miner and loss succeed. | | `CrossBatchMemory.check_loss_support` / `supported_losses()` | Any `BaseMetricLossFunction` that accepts `ref_emb` | Losses that reject references, such as `LiftedStructureLoss` and the proxy losses, raise `ValueError` on the first call. | | Custom `already_reduced` handling of regularization penalties | Customize the loss's `regularization_terms()` | A reducer never receives those penalties; `loss_terms()` returns them in the dictionary. | | `datasets.BaseDataset` | `torch.utils.data.Dataset` | Applications own downloads, transforms, labels, and split selection. | | `datasets.Cars196` | An application dataset for Cars196 | Preserve the [published split and citation](recipes/evaluation.md#published-benchmark-splits). | | `datasets.CUB` | An application dataset for CUB-200-2011 | Preserve the published split and citation. | | `datasets.INaturalist2018` | An application dataset for iNaturalist 2018 | Preserve the published split and citation. | | `datasets.StanfordOnlineProducts` | An application dataset for Stanford Online Products | Preserve the published split and citation. | | Default `NMI` and `AMI` metrics | Explicit `include` plus `kmeans_func` | Install `[clustering]` and choose a clustering backend. | | `DistributedLossWrapper` warning without a process group | Silent delegation to the wrapped loss | One-process and uninitialized runs return the wrapped loss unchanged. | | `DistributedLossWrapper(loss, efficient=False)` | `DistributedLossWrapper(loss, objective="global")` | The default full-batch objective. Version 2 multiplied the gathered loss by the world size; the global objective returns the unscaled loss, and DDP's gradient average recovers the single-process gradient. | | `DistributedLossWrapper(loss, efficient=True)` | `DistributedLossWrapper(loss, objective="local_anchor")` | A different objective and gradient; memory losses support only `"global"`. | | `DistributedMinerWrapper(miner, efficient=...)` | `DistributedMinerWrapper(miner, objective=...)` | Use the same objective as the loss wrapper. | | `DotProductSimilarity()` normalizing embeddings | `DotProductSimilarity(normalize_embeddings=True)` | The default is now a plain dot product. `NPairsLoss` and `NormalizedSoftmaxLoss` keep their normalized default. | | `DynamicSoftMarginLoss` moving its histogram during `forward` | Place the loss with `.to(device, dtype)` | Its checkpointed histogram keeps its configured precision. | | `DynamicSoftMarginLoss.hist_` | `DynamicSoftMarginLoss.hist` | A buffer saved in `state_dict`. | | Empty mined tuples treated as unweighted (`convert_to_weights` returning ones) | Pass `indices_tuple=None` for no restriction | A supplied tuple with no indices selects nothing: miner-weighted losses return a differentiable zero. | | Evaluation with string labels | Convert labels to integer tensors in caller code | Use one mapping shared by query and reference sets. | | Fixed `MarginLoss.beta` outside its state dictionary | A checkpointed buffer that follows `.to()` | Account for the added `beta` key when importing an old checkpoint. | | Float, bool, or uint8 index tensors in `indices_tuple` | `int64` or `int32` tensors | Other dtypes raise `TypeError`. | | Forwarded attributes such as `batched_distance.normalize_embeddings` | `batched_distance.distance.normalize_embeddings` | Only `is_inverted` is exposed directly for search ordering. | | `GenericPairLoss(mat_based_loss=...)` with `_compute_loss(...)` | `GenericPairLoss(**options)` with `pair_loss(mat, pairs)` | Use `pair_values` or `pair_masks`; return `{}` when nothing is valid; see [pair-based losses](extend/losses.md#pair-based-losses). | | `HDCMiner.filter_original_indices` | Use the tuple returned by the miner | Pass candidates as `indices_tuple`; filtering never mutates them. | | `HDCMiner.reset_idx()` | Omit `indices_tuple` | The miner keeps no candidate state between calls. | | `HDCMiner.set_idx_externally(indices_tuple, labels)` | `miner(embeddings, labels, indices_tuple=pairs)` | Every built-in miner accepts candidate tuples. | | `HDCMiner.should_select_largest` | The miner handles orientation from `distance.is_inverted` | Selection follows `distance.is_inverted`. | | `HierarchicalSampler.reshuffle()` | Begin a new iteration over the sampler | Batch generation occurs during iteration, not construction or `len()`. | | `HistogramLoss(n_bins=..., delta=...)` | `HistogramLoss(num_bins=...)` | `delta` is `2 / num_bins`; the default remains 100 bins. | | Implicit scikit-learn installation | `pytorch-metric-learning[clustering]` | Required for NMI and AMI, alongside an explicit clustering backend. | | `InferenceModel.train_indexer` | `evaluation.FaissKNN.train(embeddings)` | Extract the reference embeddings first; use `add`, `save`, and `load` on the same backend. | | Labels on another device moved automatically | Labels on the embeddings' device | A device mismatch raises `ValueError`; place labels with `.to(embeddings.device)`. | | `LargeMarginSoftmaxLoss(margin=4.0)` float margins | An `int` margin | Non-integer margins raise `ValueError` instead of being truncated. | | `LargeMarginSoftmaxLoss.cast_types` | `.to(device, dtype)` | This also applies to inherited classification losses. | | `LiftedStructureLoss` with `ref_emb`, `CrossBatchMemory`, or asymmetric `SelfSupervisedLoss` | A loss that supports reference embeddings | It compared query and reference indices incorrectly and now raises `ValueError`. | | `LiftedStructureLoss(neg_margin, pos_margin)` / `GeneralizedLiftedStructureLoss(neg_margin, pos_margin)` positionally | `(pos_margin, neg_margin)` | Same order as `ContrastiveLoss` and `PairMarginMiner`; keyword calls are unchanged. | | Loss terms with `reduction_type` | Terms with `losses`, `indices`, and optional `divisor` | Kinds follow from `indices`: `None`, a tensor, a pair tuple, or a triplet tuple. | | `loss.W`, `loss.proxies`, `loss.fc` parameters | `loss.weight` with shape `(num_classes, embedding_size)` | One row per class or center; see [learnable weights](losses.md#learnable-weight-arguments). `ManifoldLoss.proxies` keeps its name because its rows are unlabeled points, not classes. | | `losses.BaseLossWrapper` | `torch.nn.Module` | `CrossBatchMemory` and `SelfSupervisedLoss` inherit it directly. | | `losses.dynamic_soft_margin_loss.find_hard_negatives` | `DynamicSoftMarginLoss` for its built-in mining, or an explicit miner | The loss no longer exposes a separate mining helper. | | `losses.EmbeddingRegularizerMixin` | `embedding_regularizer` and `embedding_reg_weight` on the loss | The same removal applies to `losses.mixins.EmbeddingRegularizerMixin`. | | `losses.large_margin_softmax_loss.CosineMarginSoftmaxLoss` | `losses.BaseProxyLoss` for new proxy losses | The cosine-margin base is private. | | `losses.mixins.WeightMixin` | A loss parameter plus `regularization_terms()` when needed | Weight initialization accepts an in-place callable. | | `losses.proxy_losses.ProxyNCALoss` | `losses.ProxyNCALoss` | Import loss classes from `losses`; submodule paths are not public. | | `losses.smooth_ap`, `losses.tcm_loss`, `losses.signal_to_noise_ratio_losses` modules | `losses.SmoothAPLoss`, `losses.ThresholdConsistentMarginLoss`, `losses.SignalToNoiseRatioContrastiveLoss` | Import loss classes from `losses`; submodule paths are not public. | | `losses.WeightRegularizerMixin` | Keyword-only `weight_regularizer` and `weight_reg_weight` | The same removal applies to `losses.mixins.WeightRegularizerMixin`. | | `LpRegularizer(..., distance=...)` / `ZeroMeanRegularizer(distance=...)` | Omit it | These regularizers never used a distance; the argument now raises `TypeError`. | | `LpRegularizer.compute_loss(embeddings=...)` | `compute_loss(x=...)` | Positional calls are unchanged. | | `ManifoldLoss(...)(embeddings, indices_tuple=assignments)` | `loss(embeddings, assignments)` | Proxy assignments are passed as `labels`; `indices_tuple` raises `ValueError`. | | `ManifoldLoss(l=..., K=..., lambdaC=...)` | `ManifoldLoss(embedding_size=..., num_proxies=..., lambda_c=...)` | Parameter meanings are unchanged. | | `MarginLoss` term `"margin_loss"` | Term `"loss"` | Rename `MultipleReducers` keys; `beta_reg_loss` is unchanged. | | `MatchFinder.get_matching_pairs(..., return_tuples=True)` | `MatchFinder.get_match_indices(...)` | A `(num_matches, 2)` `torch.long` tensor; use `.tolist()` for Python pairs. | | `MatchFinder.get_matching_pairs(query_emb, ref_emb)` | `MatchFinder.get_match_matrix(query_emb, ref_emb)` | A boolean tensor on the input device instead of a NumPy array. | | `MatchFinder.is_match(...)` | A boolean tensor on the input device | Normalization and distance power now match matrix comparisons. | | Metric-method keyword arguments (`knn_labels`, `label_counts`, `not_lone_query_mask`, `cluster_labels`, ...) | `RetrievalInputs` fields `matches`, `relevant_counts`, `query_labels`, `knn_labels` | `matches` already applies the label comparison; neighbor distances are not passed. | | Mined tuples as plain tuples | `types.PairIndices` and `types.TripletIndices` named tuples | They unpack like tuples; plain tuples remain accepted as inputs. | | `miner(...)` or `DistributedMinerWrapper(...)(embeddings, labels, ref_emb, ref_labels)` positionally | `miner(embeddings, labels, ref_emb=..., ref_labels=...)` | The third positional argument is `indices_tuple`, as for losses. | | `miners.EmbeddingsAlreadyPackagedAsTriplets` | Construct the three index tensors directly | See the packed-triplet example below. | | `DataLoader(dataset, batch_size=32, sampler=MPerClassSampler(labels, m, 32))` | `DataLoader(dataset, batch_sampler=MPerClassSampler(labels, m, 32))` | The sampler yields whole batches, so `batch_size` is required and the `DataLoader` cannot break the per-class grouping. `len()` counts batches. | | `MPerClassSampler(length_before_new_iter=...)` | `MPerClassSampler(labels, m, batch_size, num_batches=...)` | Counts batches instead of indices. The default covers every example at least once. | | `MultipleLosses` containing `CrossBatchMemory` or another module | `CrossBatchMemory(MultipleLosses({...}), ...)` | `losses` values must be `BaseMetricLossFunction` or `MultipleLosses`; others raise `TypeError`. | | `MultipleLosses([loss_a, loss_b], weights=[1, 0.5])` | `MultipleLosses({"a": loss_a, "b": loss_b}, weights={"a": 1, "b": 0.5})` | Mappings only; see [MultipleLosses](losses/multiple_losses.md). | | `MultipleLosses.assertions_if_not_none` | Constructor validation | Invalid keys raise explicit exceptions. | | `MultipleLosses.forward(embeddings, labels, indices_tuple=None)` | `forward(embeddings, labels=None, indices_tuple=None, ref_emb=None, ref_labels=None)` | Reference arguments are passed to every loss and miner. | | `MultipleLosses.get_indices_tuple` | Pass a shared `indices_tuple` to `forward`, configure per-loss miners, or both | Per-loss miners select from a supplied `indices_tuple` as candidates; [MultipleLosses contract](losses/multiple_losses.md). | | `MultipleReducers` entries named `embedding_reg_loss` or `weight_reg_loss` | Configure regularizers on the loss | Penalties are added after objective reduction; these entries are no longer used. | | `NormalizedSoftmaxLoss.cast_types` | `.to(device, dtype)` | Placement belongs to the caller. | | NumPy-seeded `FixedSetOfTriplets` | Pass a keyword-only `torch.Generator` | Seeded sequences differ from version 2. | | NumPy-seeded `HierarchicalSampler` | Pass a keyword-only `torch.Generator` | Seeded sequences differ from version 2. | | NumPy-seeded `MPerClassSampler` | Pass a keyword-only `torch.Generator` | `torch.manual_seed` also controls its default generator. | | `P2SGradLoss(..., distance=...)` | Omit it | The distance was ignored and now raises `TypeError`. | | `P2SGradLoss(descriptors_dim, num_classes)` | `P2SGradLoss(num_classes, embedding_size)` | Argument order matches the other classification losses. | | `PairMarginMiner.set_stats` | Diagnostics computed from returned indices | No stored miner statistics remain. | | `PerAnchorReducer(aggregation_func=...)` | `PerAnchorReducer(aggregation="mean")` or `aggregation="sum"` | Losses are aggregated by anchor; the mean averages over anchors that have losses instead of over every row. | | `PNPLoss.VARIANTS` | `types.PNPVariant` | `typing.get_args(PNPVariant.__value__)` lists the variants. | | `ProxyAnchorLoss` miner weights adding `w - 1` to exponents | Weights multiply the exponential terms | Unmined samples (`w = 0`) are excluded; unweighted results are unchanged. | | `ProxyAnchorLoss.cast_types` | `.to(device, dtype)` | Placement belongs to the caller. | | `ProxyAnchorLoss.get_default_weight_init_func` | The `weight_init_func` constructor argument | Omit it to retain the loss's default initializer. | | `ProxyNCALoss` miner weights counting only anchors | Weights count every position in the mined tuples | This matches the other classification losses; a mined triplet also weights its positive and negative rows. | | Python 3.10 or 3.11 | Python 3.12 or newer | Minimum tested runtimes are PyTorch 2.14.0 and NumPy 2.5.3. | | `pytorch-metric-learning[dev]` | The frozen development environment | See [development setup](https://github.com/KevinMusgrave/pytorch-metric-learning/blob/master/DEVELOPMENT.md). | | `pytorch-metric-learning[docs]` | The frozen development environment | See [development setup](https://github.com/KevinMusgrave/pytorch-metric-learning/blob/master/DEVELOPMENT.md). | | `pytorch-metric-learning[with-hooks-cpu]` | Application-owned logging; install `[faiss]` separately when needed | record-keeper and TensorBoard integration were removed. | | `pytorch-metric-learning[with-hooks]` | Application-owned logging; install `[faiss]` separately when needed | record-keeper and TensorBoard integration were removed. | | `pytorch_metric_learning.__version__` | `importlib.metadata.version("pytorch-metric-learning")` | The installed distribution metadata is the single version source. | | `r_precision` or `mean_average_precision_at_r` with an integer `k` below the relevant count | `k="max_bin_count"` | The configuration logs a warning. R-precision now divides by the relevant count instead of by `k`. | | `RankedListLoss(margin, Tn, ..., Tp=...)` | `RankedListLoss(margin, neg_temperature, ..., pos_temperature=...)` | Weighted means are exact; the `1e-5` offset and the overflow warning are gone. | | `reducer=` deep-copied and wrapped in `MultipleReducers` for multi-term losses | The reducer object is used as given for every term | Pass a `MultipleReducers` to vary reducers per term. | | `reducer=` on `DynamicSoftMarginLoss`, `HistogramLoss`, `ManifoldLoss`, `P2SGradLoss`, `PNPLoss`, `ThresholdConsistentMarginLoss` | Omit it | These return one reduced value; the reducer was ignored and now raises `TypeError`. | | `reducers.DoNothingReducer` | `loss_func.loss_terms(...)` | Reducers always return a tensor; terms carry `losses`, `indices`, and an optional `divisor`. | | `reducers.DoNothingReducer()` to get the loss dictionary | `loss_func.loss_terms(embeddings, labels, ...)` | `forward` always returns a tensor; see [unreduced terms](losses.md#unreduced-loss-terms). | | `ref_emb=embeddings` (the same tensor) meaning self-comparison | Omit `ref_emb` | Passing `ref_emb` always describes a separate reference set, so the diagonal is kept. | | `RegularFaceRegularizer.compute_loss(weights=...)` | `compute_loss(x=...)` | Positional calls are unchanged. | | `samplers.TuplesToWeightsSampler` | `convert_to_weights` plus `torch.utils.data.WeightedRandomSampler` | Compute full-dataset weights in caller code; [weighted-sampling recipe](recipes/training.md#mining-weighted-sampling-contract). | | `SelfSupervisedLoss` cloning labels to force a separate set | `ref_emb` is a separate set by construction | No identity workaround is needed. | | `SelfSupervisedLoss.check_loss_support` / `supported_losses()` | Any `BaseMetricLossFunction`; asymmetric mode needs `ref_emb` support | Other modules raise `TypeError` at construction. | | `SmoothAPLoss` term `"ap_loss"` | Term `"loss"` | Matches every single-term loss for `MultipleReducers`. | | `SoftTripleLoss.cast_types` | `.to(device, dtype)` | Placement belongs to the caller. | | `SoftTripleLoss.gamma` storing `1 / gamma` | `SoftTripleLoss.gamma` storing `gamma` | The objective is unchanged. | | `SoftTripleLoss.get_default_weight_init_func` | The `weight_init_func` constructor argument | Omit it to retain the loss's default initializer. | | `SomeLoss(..., **kwargs)` forwarding `distance`, `reducer`, and regularizer arguments | `SomeLoss(..., **options)` typed by `types.LossOptions` | The same four keyword-only options; see [loss options](losses.md#loss-options). | | `SomeMiner(..., **kwargs)` forwarding `distance` | `SomeMiner(..., *, distance=None)` | `distance` is the only miner option. | | `SomeRegularizer(..., **kwargs)` forwarding `distance` and `reducer` | `SomeRegularizer(..., **options)` typed by `types.RegularizerOptions` | Both options are keyword-only. | | `SparseCentersRegularizer.compute_loss(weights=...)` | `compute_loss(x=...)` | Positional calls are unchanged. | | `SparseCentersRegularizer.set_stats` | Application diagnostics | Its same-class mask is now a non-persistent buffer that follows `.to()`. | | `SubCenterArcFaceLoss.get_outliers(embeddings, labels, threshold=75, return_dominant_centers=...)` | `get_outliers(embeddings, labels, threshold_radians=math.radians(75))` | Always returns `types.SubCenterOutliers(outliers, dominant_centers)`; the threshold is in radians like `margin`. | | `SupConLoss` returning zero for a batch without negatives | The paper's value | Each anchor's positives are still contrasted with every other sample. | | `testers.BaseTester` | Extract embeddings, then call `evaluation.AccuracyCalculator` | [Embedding extraction](recipes/evaluation.md#extract-embeddings). | | `testers.GlobalEmbeddingSpaceTester` | Explicit query/reference split composition | [Split composition](recipes/evaluation.md#compose-query-and-reference-splits). | | `testers.GlobalTwoStreamEmbeddingSpaceTester` | Evaluate one stream against the other | [Two-stream evaluation](recipes/evaluation.md#two-stream-evaluation). | | `testers.WithSameParentLabelTester` | Evaluate each parent-conditioned gallery explicitly | [Hierarchical retrieval](recipes/evaluation.md#parent-conditioned-hierarchical-retrieval). | | `ThresholdReducer` boolean indexing | Masked sum and count | Values are unchanged; the reduction no longer synchronizes with the host. | | `ThresholdReducer.element_reduction_helper` | `element_reduction(losses, anchors, embeddings, labels)` | Every term is reduced through its anchor indices. | | `ThresholdReducer.set_stats` | Compute diagnostics from caller-owned loss terms | No stored `num_past_filter` remains. | | `trainers.BaseTrainer` | An ordinary PyTorch training loop | Applications own data loading, optimizers, schedules, freezing, and checkpoints. | | `trainers.CascadedEmbeddings` | Pass mined pairs as `indices_tuple` to successive `HDCMiner` calls | [Cascaded mining recipe](recipes/training.md#cascaded-mining-contract). | | `trainers.deep_adversarial_metric_learning.NegativeLpDistance` | Form the adversarial distance objective in caller code | The adversarial recipe specifies signs and gradient ownership. | | `trainers.DeepAdversarialMetricLearning` | Explicit metric, classifier, and adversarial objectives | [Adversarial recipe](recipes/training.md#adversarial-objectives-and-gradient-ownership). | | `trainers.MetricLossOnly` | Compute embeddings, call the loss, then backward and step | [Metric training recipe](recipes/training.md#metric-classifier-proxy-and-two-stream-inputs). | | `trainers.TrainWithClassifier` | Add classifier and metric objectives in caller code | [Classifier recipe](recipes/training.md#metric-classifier-proxy-and-two-stream-inputs). | | `trainers.TwoStreamMetricLoss` | Pass explicit query and reference embeddings to the loss | [Two-stream recipe](recipes/training.md#metric-classifier-proxy-and-two-stream-inputs). | | `TripletMarginLoss()` with `margin=0.05` | `TripletMarginLoss(margin=0.05)` | The default is 0.2, matching `TripletMarginMiner`. | | `TripletMarginMiner.set_stats` | Diagnostics computed from returned indices | No stored miner statistics remain. | | `TupletMarginLoss.set_stats` | Compute diagnostics in caller code | The loss returns the objective without statistics side effects. | | `utils.accuracy_calculator` | `evaluation` | `from pytorch_metric_learning.evaluation import AccuracyCalculator`. | | `utils.common_functions` | Removed from the public API | Helpers used by the library are private; the rows below name replacements. | | `utils.common_functions.add_to_recordable_attributes` | Application diagnostics or forward hooks | The record-keeper attribute registry was removed. | | `utils.common_functions.append_map` | A Python comprehension or `map` | Apply the application's transformation directly. | | `utils.common_functions.assert_distance_type` | An `isinstance` check on `self.distance` in your module | The library's distance checks are private; see [custom losses](extend/losses.md#more-on-distances). | | `utils.common_functions.COLLECT_STATS` | Application diagnostics | No global statistics flag remains. | | `utils.common_functions.delete_dict_of_models` | `pathlib.Path.unlink` on chosen checkpoint files | The application owns retention rules. | | `utils.common_functions.EmbeddingDataset` | `torch.utils.data.TensorDataset(embeddings, labels)` | DataLoader returns the paired tensors. | | `utils.common_functions.exclude` | A Python comprehension | Make the filtering rule explicit. | | `utils.common_functions.get_eval_dataloader` | `torch.utils.data.DataLoader` | Choose evaluation batching and ordering explicitly. | | `utils.common_functions.get_hierarchy_label` | Select the desired label column explicitly | For a label matrix, use `labels[:, level]`. | | `utils.common_functions.get_label_map` | Construct one mapping per hierarchy level | The library no longer owns dataset metadata. | | `utils.common_functions.get_labels_to_indices` | Group indices with label comparisons | For label `c`, use `(labels == c).nonzero(as_tuple=True)[0]`. | | `utils.common_functions.get_train_dataloader` | `torch.utils.data.DataLoader` | Pass a sampler or batch sampler according to its PyTorch contract. | | `utils.common_functions.is_list_or_tuple` | `isinstance(value, (list, tuple))` | Use Python directly. | | `utils.common_functions.LabelMapper` | Application-owned label mapping | Do not build independent query/reference encodings. | | `utils.common_functions.latest_version` | Application checkpoint selection | Select naming, best-model, and resume rules explicitly. | | `utils.common_functions.list_of_recordable_attributes_list_names` | Application-owned metric names | The recordable-attribute lists were removed. | | `utils.common_functions.load_dict_of_models` | Load each state dictionary explicitly | The application owns missing-key and resume policy. | | `utils.common_functions.load_model` | `model.load_state_dict(torch.load(path, map_location=device, weights_only=True))` | Handle legacy key conversions explicitly outside the library. | | `utils.common_functions.LOGGER` | `logging.getLogger("pytorch_metric_learning")` | [Application logging setup](retrieval.md#logging). | | `utils.common_functions.LOGGER_NAME` | The standard `pytorch_metric_learning` logger namespace | Individual modules use their own module names. | | `utils.common_functions.make_label_to_rank_dict` | `{label: i for i, label in enumerate(sorted(unique_labels))}` | Persist the mapping if later data must use it. | | `utils.common_functions.map_labels` | Application-owned label mapping | Share one mapping across evaluated splits. | | `utils.common_functions.modelpath_creator` | `pathlib.Path` and application naming rules | The application owns checkpoint paths. | | `utils.common_functions.NUMPY_RANDOM` | A `torch.Generator` passed to a sampler | Seeded sequences differ from version 2. | | `utils.common_functions.numpy_to_torch` | `torch.as_tensor(array)` | Specify device and dtype when needed. | | `utils.common_functions.operate_on_dict_of_models` | Iterate over the application's model dictionary | No generic checkpoint-operation dispatcher remains. | | `utils.common_functions.process_label` | Select the label column and apply the application mapping | Hierarchy handling is explicit. | | `utils.common_functions.regex_replace` | `re.sub` | Apply replacement rules explicitly. | | `utils.common_functions.regex_wrapper` | Python's `re` module | No library wrapper remains. | | `utils.common_functions.reset_stats` | Reset application-owned accumulators | Modules no longer retain statistics. | | `utils.common_functions.return_input` | Return the value directly | Use `torch.nn.Identity` when a module is required. | | `utils.common_functions.safe_random_choice` | `torch.randperm` or `torch.randint` | Select without or with replacement explicitly. | | `utils.common_functions.save_dict_of_models` | Save a dictionary of model `state_dict()` values | Include optimizer and schedule state when resuming training. | | `utils.common_functions.save_model` | `torch.save(model.state_dict(), path)` | For a DDP-wrapped model, save the underlying `model.module` state. | | `utils.common_functions.set_layers_to_eval` | Call `.eval()` on explicitly selected modules | Select modules by reference or type, not class-name substrings. | | `utils.common_functions.set_logger_name` | Configure Python logging handlers and levels | No library logging configuration helper remains. | | `utils.common_functions.set_requires_grad` | `module.requires_grad_(enabled)` | Manage optimizer membership and model mode separately. | | `utils.common_functions.shift_indices_tuple` | Offset reference indices explicitly | For triplets shift positive/negative indices; for pairs shift their second/fourth tensors. | | `utils.common_functions.sqlite_obj_to_dict` | Application database handling | SQLite/record-keeper integration was removed. | | `utils.common_functions.to_device` | `tensor.to(device=device, dtype=dtype)` | Omit `dtype` to preserve it. | | `utils.common_functions.to_dtype` | `tensor.to(dtype=dtype)` | Explicit conversions also apply inside autocast. | | `utils.common_functions.to_numpy` | `tensor.detach().cpu().numpy()` | Omit `detach()` only for tensors without gradients. | | `utils.common_functions.torch_arange_from_size(x)` | `torch.arange(len(x), device=x.device)` | Write the index tensor directly. | | `utils.common_functions.torch_standard_scaler` | Explicit centering and scaling | Fit statistics on the reference/training data and reuse them for queries. | | `utils.common_functions.TorchInitWrapper` | An in-place `torch.nn.init` callable | Bind options with `functools.partial`; see [parameter placement](#loss-parameter-placement). | | `utils.common_functions.try_next_on_generator` | Own the iterator and handle `StopIteration` | Recreate the iterator when the application wants another pass. | | `utils.common_functions.unslice_by_n` | `torch.stack(parts, dim=1).flatten(0, 1)` | Interleave equal-shaped streams along the batch dimension. | | `utils.common_functions.use_cuda_if_available` | Select a `torch.device` in caller code | Library defaults no longer choose CUDA globally. | | `utils.distributed` | `distributed` | `from pytorch_metric_learning.distributed import DistributedLossWrapper`. | | `utils.distributed.all_gather` | PyTorch collectives for raw tensors; library distributed wrappers for losses/miners | [Distributed contracts](distributed.md) specify gradient ownership. | | `utils.distributed.all_gather_embeddings_and_labels` | `distributed.DistributedLossWrapper` or `distributed.DistributedMinerWrapper` | Use the wrappers to preserve their batch, label, and gradient contracts. | | `utils.inference.add_to_index_and_search` | `FaissKNN(...)` calls, or `FaissKNN.search(index, query, reference, k)` | The helper is private. | | `utils.inference.convert_to_cpu_index` | `FaissKNN.to_cpu_index(index)` | Use the backend instance that owns the index. | | `utils.inference.convert_to_gpu_index` | `FaissKNN.to_gpu_index(index)` | Configure the backend's `gpus` in its constructor. | | `utils.inference.CustomKNN` | `evaluation.CustomKNN` | Exact PyTorch search; no Faiss dependency. | | `utils.inference.FaissKMeans` | `evaluation.FaissKMeans` | Install `[faiss]`; pass this backend explicitly for clustering metrics. | | `utils.inference.FaissKNN` | `evaluation.FaissKNN` | Install `[faiss]` explicitly. | | `utils.inference.InferenceModel` | Extract embeddings in caller code, then use `evaluation` | [Evaluation recipe](recipes/evaluation.md); preprocessing and model mode belong to the application. | | `utils.inference.MatchFinder` | `evaluation.MatchFinder` | Supply a threshold in the constructor or call. | | `utils.inference.return_results` | The `FaissKNN` or `CustomKNN` result | Both return results with self-matches already removed when the query is in the reference set. | | `utils.inference.run_pca` | Application-owned PCA, such as `torch.pca_lowrank` | Center and project query/reference embeddings with the same fitted basis. | | `utils.inference.try_gpu` | `FaissKNN.search(index, query, reference, k)` | Backend selection belongs to the backend. | | `utils.key_checker.default_important` | Application validation rules | This trainer-schema callback was removed. | | `utils.key_checker.KeyChecker` | Explicit application argument validation | Trainer key schemas were removed. | | `utils.key_checker.KeyCheckerDict` | Ordinary dictionaries and explicit validation | No trainer schema container remains. | | `utils.logging_presets.EmptyContainer` | Omit callbacks that are not needed | No no-op hook container is required. | | `utils.logging_presets.get_hook_container` | Application callbacks | See the [training recipes](recipes/training.md). | | `utils.logging_presets.get_record_keeper` | Configure the application's chosen logger | record-keeper is no longer a dependency. | | `utils.logging_presets.HookContainer` | Application callbacks and logging | The library no longer owns training events. | | `utils.loss_and_miner_utils` | `utils.indexing` | Public pair, triplet, and weight utilities accept plain or named index tuples; internals are private. | | `utils.loss_and_miner_utils.convert_to_triplets` default `t_per_anchor=100` | `utils.indexing.convert_to_triplets(..., t_per_anchor=100)` | The default is now every triplet. | | `utils.loss_and_miner_utils.get_all_triplets_indices_loop_method` | `utils.indexing.get_all_triplets_indices(labels, ref_labels)` | The implementation bounds intermediate memory automatically. | | `utils.loss_and_miner_utils.get_all_triplets_indices_vectorized_method` | `utils.indexing.get_all_triplets_indices(labels, ref_labels)` | Call the label-based API instead of choosing an enumeration implementation. | | `utils.loss_and_miner_utils.get_random_triplet_indices(t_per_anchor=None)` | `utils.indexing.get_random_triplet_indices(t_per_anchor=None)` | Vectorized sampling; `None` draws one triplet per positive of each anchor. | | `utils.loss_and_miner_utils.matched_size_indices` | Choose positive/negative pairing in caller code | There is no automatic repetition helper. | | `utils.loss_and_miner_utils.remove_self_comparisons(indices, idx, ref_size)` | `CrossBatchMemory`, or filter with `positives != curr_batch_idx[anchors]` | `CrossBatchMemory` excludes each anchor's queued copy before mining. | | `utils.loss_and_miner_utils.repeat_to_match_size` | `tensor.repeat(repeats)[:size]` when repetition is intended | Handle empty inputs explicitly; repeated entries change weighting. | | `utils.loss_tracker.LossTracker` | Retain returned loss values in caller code | Detach values intended only for logging. | | `utils.module_with_records.ModuleWithRecords` | `torch.nn.Module` | Statistics registration was removed. | | `utils.module_with_records_and_reducer.ModuleWithRecordsAndDistance` | `torch.nn.Module` with a keyword-only `distance` option | `BaseMiner` declares `distance` directly. | | `utils.module_with_records_and_reducer.ModuleWithRecordsAndReducer` | `torch.nn.Module` with a keyword-only `reducer` option | `BaseRegularizer` declares `distance` and `reducer` directly. | | `utils.module_with_records_and_reducer.ModuleWithRecordsReducerAndDistance` | `BaseMetricLossFunction` keyword-only options | See [loss options](losses.md#loss-options); no composition mixins remain. | | `utils.retrieval` | `evaluation` | `CustomKNN`, `FaissKNN`, `FaissKMeans`, and `MatchFinder` are exported by `evaluation`. | | `VICRegLoss` with a one-row batch | A batch of at least 2 rows | Returned NaN; now raises `ValueError`. | | `VICRegLoss(...)(embeddings, ref_emb=ref_emb)` with `reducer` or `distance` | `VICRegLoss(...)(embeddings, ref_emb)` | A plain module without loss options; the value is unchanged. | | `ZeroMeanRegularizer.compute_loss(embeddings=...)` | `compute_loss(x=...)` | Positional calls are unchanged. | ## Examples ### Explicit query and reference evaluation ```python import torch from pytorch_metric_learning.evaluation import AccuracyCalculator reference = torch.tensor([[0.0, 0.0], [0.1, 0.0], [3.0, 0.0], [3.1, 0.0]]) reference_labels = torch.tensor([0, 0, 1, 1]) query = torch.tensor([[0.05, 0.0], [3.05, 0.0]]) query_labels = torch.tensor([0, 1]) calculator = AccuracyCalculator(include=("precision_at_1",), k=10) metrics = calculator.get_accuracy(query, query_labels, reference, reference_labels) assert metrics["precision_at_1"] == 1.0 ``` ### Loss parameter placement ```python from functools import partial from itertools import chain import torch from pytorch_metric_learning import losses, regularizers torch.manual_seed(7) model = torch.nn.Linear(4, 3).to(dtype=torch.float64) loss_func = losses.ProxyNCALoss( num_classes=2, embedding_size=3, weight_regularizer=regularizers.LpRegularizer(), weight_reg_weight=0.01, weight_init_func=partial(torch.nn.init.normal_, std=0.1), ).to(dtype=torch.float64) optimizer = torch.optim.SGD(chain(model.parameters(), loss_func.parameters()), lr=0.01) inputs = torch.randn(4, 4, dtype=torch.float64) labels = torch.tensor([0, 0, 1, 1]) optimizer.zero_grad(set_to_none=True) loss = loss_func(model(inputs), labels) loss.backward() assert torch.isfinite(loss) assert torch.isfinite(model.weight.grad).all() assert torch.isfinite(next(loss_func.parameters()).grad).all() optimizer.step() ``` ### Consecutively packed triplets ```python import torch from pytorch_metric_learning import losses embeddings = torch.tensor( [[1.0, 0.0], [0.9, 0.1], [0.0, 1.0], [0.0, 1.0], [0.1, 0.9], [1.0, 0.0]], requires_grad=True, ) labels = torch.tensor([0, 0, 1, 1, 1, 0]) indices_tuple = ( torch.arange(len(embeddings), device=embeddings.device).reshape(-1, 3).unbind(1) ) loss = losses.TripletMarginLoss(margin=1.4)(embeddings, labels, indices_tuple) loss.backward() assert loss > 0 assert torch.isfinite(embeddings.grad).all() ``` ### Per-call cascade candidates ```python import torch from pytorch_metric_learning import miners from pytorch_metric_learning.utils import indexing embeddings = torch.tensor([[1.0, 0.0], [0.8, 0.2], [0.0, 1.0], [0.2, 0.8]]) labels = torch.tensor([0, 0, 1, 1]) candidates = indexing.get_all_pairs_indices(labels) miner = miners.HDCMiner(filter_percentage=0.5) pairs = miner(embeddings, labels, indices_tuple=candidates) assert len(pairs[0]) == 2 assert len(pairs[2]) == 4 assert torch.equal(labels[pairs[0]], labels[pairs[1]]) assert torch.all(labels[pairs[2]] != labels[pairs[3]]) ``` ### Batched distance blocks ```python import torch from pytorch_metric_learning import distances query = torch.tensor([[0.0, 0.0], [1.0, 0.0], [0.0, 2.0]]) reference = torch.tensor([[0.0, 1.0], [2.0, 0.0]]) output = query.new_empty(len(query), len(reference)) distance = distances.BatchedDistance( distances.LpDistance(normalize_embeddings=False), batch_size=2 ) for matrix, start, end in distance(query, reference): output[start:end] = matrix expected = query.new_tensor([[1.0, 2.0], [2**0.5, 1.0], [1.0, 8**0.5]]) torch.testing.assert_close(output, expected) ```