# Apache Lucene Migration Guide ## Migration from Lucene 9.x to Lucene 10.0 ### OpenNLP dependency upgrade [Apache OpenNLP](https://opennlp.apache.org) 2.x opens the door to accessing various models via the ONNX runtime. To migrate you will need to update any deprecated OpenNLP methods that you may be using. ### Snowball dependency upgrade Snowball has folded the "German2" stemmer into their "German" stemmer, so there's no "German2" anymore. For Lucene APIs (TokenFilter, TokenFilterFactory) that accept String, "German2" will be mapped to "German" to avoid breaking users. If you were previously creating German2Stemmer instances, you'll need to change your code to create GermanStemmer instances instead. For more information see https://snowballstem.org/algorithms/german2/stemmer.html ### Romanian analysis RomanianAnalyzer now works with Romanian in its modern unicode form, and normalizes cedilla forms to forms with commas. Both forms are still in use in "the wild": you should reindex Romanian documents. ### IndexWriter requires a parent document field in order to use index sorting with document blocks (GITHUB#12829) For indices newly created as of 10.0.0 onwards, IndexWriter preserves document blocks indexed via IndexWriter#addDocuments or IndexWriter#updateDocuments when index sorting is configured. Document blocks are maintained alongside their parent documents during sort and merge. The internally used parent field must be configured in IndexWriterConfig only if index sorting is used together with documents blocks. See `IndexWriterConfig#setParendField` for reference. ### Minor API changes in MatchHighlighter and MatchRegionRetriever. (GITHUB#12881) The API of interfaces for accepting highlights has changed to allow performance improvements. Look at the issue and the PR diff to get a sense of what's changed (changes are minor). ### Removed deprecated IndexSearcher.doc, IndexReader.document, IndexReader.getTermVectors (GITHUB#11998) The deprecated Stored Fields and Term Vectors apis relied upon threadlocal storage and have been removed. Instead, call storedFields()/termVectors() to return an instance which can fetch data for multiple documents, and will be garbage-collected as usual. For example: ```java TopDocs hits = searcher.search(query, 10); StoredFields storedFields = reader.storedFields(); for (ScoreDoc hit : hits.scoreDocs) { Document doc = storedFields.document(hit.doc); } ``` Note that these StoredFields and TermVectors instances should only be consumed in the thread where they were acquired. For instance, it is illegal to share them across threads. ### Field can no longer configure a TokenStream independently from a value Lucene 9.x and earlier versions allowed to set a TokenStream on Field instances independently from a string, binary or numeric value. This is no longer allowed on the base Field class. If you need to replicate this behavior, you need to either provide two fields, one with a TokenStream and another one with a value, or create a sub-class of Field that overrides `TokenStream tokenStream(Analyzer, TokenStream)` to return a custom TokenStream. ### PersianStemFilter is added to PersianAnalyzer (LUCENE-10312) PersianAnalyzer now includes PersianStemFilter, that would change analysis results. If you need the exactly same analysis behaviour as 9.x, clone `PersianAnalyzer` in 9.x or create custom analyzer by using `CustomAnalyzer` on your own. ### AutomatonQuery/CompiledAutomaton/RunAutomaton/RegExp no longer determinize (LUCENE-10010) These classes no longer take a `determinizeWorkLimit` and no longer determinize behind the scenes. It is the responsibility of the caller to to call `Operations.determinize()` for DFA execution. ### DocValuesFieldExistsQuery, NormsFieldExistsQuery and KnnVectorFieldExistsQuery removed in favor of FieldExistsQuery (LUCENE-10436) These classes have been removed and consolidated into `FieldExistsQuery`. To migrate, caller simply replace those classes with the new one during object instantiation. ### Normalizer and stemmer classes are now package private (LUCENE-10561) Except for a few exceptions, almost all normalizer and stemmer classes are now package private. If your code depends on constants defined in them, copy the constant values and re-define them in your code. ### LongRangeFacetCounts / DoubleRangeFacetCounts #getTopChildren behavior change (LUCENE-10614) The behavior of `LongRangeFacetCounts`/`DoubleRangeFacetCounts` `#getTopChildren` actually returns the top-n ranges ordered by count from 10.0 onwards (as described in the `Facets` API) instead of returning all ranges ordered by constructor-specified range order. The pre-existing behavior in 9.x and earlier can be retained by migrating to the new `Facets#getAllChildren` API (LUCENE-10550). ### SortedSetDocValues#NO_MORE_ORDS removed (LUCENE-10603) `SortedSetDocValues#nextOrd()` no longer returns `NO_MORE_ORDS` when ordinals are exhausted for the currently-positioned document. Callers should instead use `SortedSetDocValues#docValueCount()` to determine the number of valid ordinals for the currently-positioned document up-front. It is now illegal to call `SortedSetDocValues#nextOrd()` more than `SortedSetDocValues#docValueCount()` times for the currently-positioned document (doing so will result in undefined behavior). ### IOContext removed from Directory#openChecksumInput (GITHUB#12027) `Directory#openChecksumInput` no longer takes in `IOContext` as a parameter, and will always use value `IOContext.READONCE` for opening internally, as that's the only valid usage pattern for checksum input. Callers should remove the parameter when calling this method. ### DaciukMihovAutomatonBuilder is renamed to StringsToAutomaton and made package-private The former `DaciukMihovAutomatonBuilder#build` functionality is exposed through `Automata#makeStringUnion`. Users should be able to directly migrate to the `Automata` static method as a 1:1 replacement. ### Remove deprecated IndexSearcher#getExecutor (GITHUB#12580) The deprecated getter for the `Executor` that was optionally provided to the `IndexSearcher` constructors has been removed. Users that want to execute concurrent tasks should rely instead on the `TaskExecutor` that the searcher holds, retrieved via `IndexSearcher#getTaskExecutor`. ### CheckIndex params -slow and -fast are deprecated, replaced by -level X (GITHUB#11023) The `CheckIndex` former `-fast` behaviour of performing checksum checks only, is now the default. Added a new parameter: `-level X`, to set the detail level of the index check. The higher the value, the more checks are performed. Sample `-level` usage: `1` (Default) - Checksum checks only, `2` - all level 1 checks as well as logical integrity checks, `3` - all level 2 checks as well as slow checks. ### Expressions module now uses `MethodHandle` and hidden classes (GITHUB#12873) Custom functions in the expressions module must now be passed in a `Map` using `MethodHandle` as values. To convert legacy code using maps of reflective `java.lang.reflect.Method`, use the converter method `JavascriptCompiler#convertLegacyFunctions`. This should make the mapping mostly compatible. The use of `MethodHandle` and [Dynamic Class-File Constants (JEP 309)](https://openjdk.org/jeps/309) now also allows to pass private methods or methods from different classloaders. It is also possible to adapt guards or filters using the `MethodHandles` class. The new implementation of the Javascript expressions compiler no longer supports use of custom `ClassLoader`, because it uses the new JDK 15 feature [hidden classes (JEP 371)](https://openjdk.org/jeps/371). Due to the use of `MethodHandle`, classloader isolation is no longer needed, because JS code can only call MHs that were resolved by the application before using the expressions module. ### `Expression#evaluate()` declares to throw IOException (GITHUB#12878) The expressions module has changed the `Expression#evaluate()` method signature: It now declares that it may throw `IOException`. This was an oversight because compiled expressions call `DoubleValues#doubleValue` behind the scenes, which may throw `IOException` on index problems, bubbling up unexpectedly to the caller. ### PathHierarchyTokenizer and ReversePathHierarchyTokenizer do not produce overlapping tokens `(Reverse)PathHierarchyTokenizer` now produces sequential (instead of overlapping) tokens with accurate offsets, making positional queries and highlighters possible for fields tokenized with this tokenizer. ### Removed Scorable#docID() (GITHUB#12407) This method has been removed in order to enable more search-time optimizations. Use the doc ID passed to `LeafCollector#collect` to know which doc ID is being collected. ### ScoreCachingWrappingScorer now wraps a LeafCollector instead of a Scorable (GITHUB#12407) In order to adapt to the removal of `Scorable#docID()`, `ScoreCachingWrappingScorer` now wraps a `LeafCollector` rather than a `Scorable`. ### Some classes converted to records classes (GITHUB#13207) Some classes with only final fields and no programming logic were converted to `record` classes. Those changes are mostly compatible with Lucene 9.x code (constructors, accessor methods), but record's fields are only available with accessor methods. Some code may need to be refactored to access the members using method calls instead of field accesses. Affected classes: - `IOContext`, `MergeInfo`, and `FlushInfo` (GITHUB#13205) - `BooleanClause` (GITHUB#13261) ### Boolean flags on IOContext replaced with a new ReadAdvice enum. The `readOnce`, `load` and `random` flags on `IOContext` have been replaced with a new `ReadAdvice` enum. ### IOContext.LOAD and IOContext.READ removed `IOContext#LOAD` has been removed, it should be replaced with `ioContext.withReadAdvice(ReadAdvice.RANDOM_PRELOAD)`. `IOContext.READ` has been removed, it should be replaced with `IOContext.DEFAULT`. ### TimeLimitingCollector removed (GITHUB#13243) `TimeLimitingCollector` has been removed, use `IndexSearcher#setTimeout(QueryTimeout)` to time out queries instead. ### IndexSearch#search(Query, Collector) being deprecated in favor of IndexSearcher#search(Query, CollectorManager) (LUCENE-10002) `IndexSearch#search(Query, Collector)` is now being deprecated in favor of `IndexSearcher#search(Query, CollectorManager)`, as `CollectorManager` implementation would allow taking advantage of intra-query concurrency via its map-reduce API design. To migrate, use a provided `CollectorManager` implementation that suits your use cases, or change your `Collector` implementation to follow the new API pattern. The straight forward approach would be to instantiate the single-threaded `Collector` in a wrapper `CollectorManager`. For example ```java public class CustomCollectorManager implements CollectorManager> { @Override public CustomCollector newCollector() throws IOException { return new CustomCollector(); } @Override public List