============================= Release Notes for MongoDB 4.2 ============================= --------------------------------- MongoDB 4.2 Released Aug 13, 2019 --------------------------------- .. default-domain:: mongodb .. role:: red(strong) :class: text-danger .. contents:: On this page :local: :backlinks: none :depth: 1 :class: twocols Patch Releases -------------- .. warning:: Past Release Limitations .. include:: /includes/release-notes/critical-warning-text.rst .. list-table:: :header-rows: 1 * - Issue - Affected Versions * - :issue:`WT-10461` - 4.2.0 - 4.2.23 (ARM64 or POWER system architectures) .. _4.2.24-release-notes: 4.2.24 - Feb 27, 2023 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.24: - :issue:`SERVER-71759` dataSize command doesn't yield - :issue:`SERVER-68115` Bug fix for "elemMatchRootLength > 0" invariant trigger - :issue:`SERVER-72535` Sharded clusters allow creating the 'admin', 'local', and 'config' databases with alternative casings - :issue:`SERVER-62738` Give mongos the ability to passthrough to a specific shard - :issue:`SERVER-68361` LogTransactionOperationsForShardingHandler::commit misses transferring documents from prepared and non-prepared transactions changing a document's shard key value - `All JIRA issues closed in 4.2.24 `__ - :ref:`4.2.24-changelog` .. _4.2.23-release-notes: 4.2.23 - Sep 29, 2022 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.23: - :issue:`SERVER-68925` Reintroduce check table logging settings at startup (revert :issue:`SERVER-43664`) - :issue:`SERVER-56127` Retryable update may execute more than once if chunk is migrated and shard key pattern uses nested fields - :issue:`SERVER-68359` [4.4] Prevent TTLMonitor from processing index if expireAfterSeconds is NaN - :issue:`SERVER-68714` NaN issues in secure random number generator in mongo shell - :issue:`SERVER-66310` Make ExpressionSetUnion::isCommutative() collation aware - `All JIRA issues closed in 4.2.23 `__ - :ref:`4.2.23-changelog` .. _4.2.22-release-notes: 4.2.22 - Aug 19, 2022 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.22: - :issue:`SERVER-64403` Find queries with SORT_MERGE collation-encode the missing sort attribute - :issue:`SERVER-68359` Prevent TTLMonitor from processing index if expireAfterSeconds is NaN - :issue:`SERVER-62656` Decouple mongos-to-config-server connection pool sizing from mongos-to-shard-server connection pool sizing - :issue:`SERVER-62272` Adding schema validation to a collection can prevent chunk migrations of failing documents - :issue:`SERVER-66418` Bad projection created during dependency analysis due to string order assumption - `All JIRA issues closed in 4.2.22 `__ - :ref:`4.2.22-changelog` .. _4.2.21-release-notes: 4.2.21 - Jun 21, 2022 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.21: - :issue:`SERVER-62272` Adding schema validation to a collection can prevent chunk migrations of failing documents - :issue:`SERVER-66176` [v4.4/v4.2] Minimize work under CollectionCatalog mutex - :issue:`SERVER-61097` SizeStorer can cause deadlocks with cache eviction - :issue:`SERVER-58506` Command to expose settability for Server Parameters - `All JIRA issues closed in 4.2.21 `__ - :ref:`4.2.21-changelog` .. _4.2.20-release-notes: 4.2.20 - May 9, 2022 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.20: - :issue:`SERVER-64983` Release Client lock before rolling back WT transaction in TransactionParticipant::_resetTransactionState - :issue:`SERVER-60412` Host memory limit check does not honor cgroups v2 - :issue:`SERVER-64642` Fix error where mongos returns CallbackCanceled and not InterruptedAtShutdown on shutdown - :issue:`SERVER-59435` Fix a double free inside DocumentSource::optimizeAt() - :issue:`SERVER-41714` $facet operator duplicates documents in the pipeline when preceded by $addFields and $match operators (in this exact order) - `All JIRA issues closed in 4.2.20 `__ - :ref:`4.2.20-changelog` .. _4.2.19-release-notes: 4.2.19 - Mar 7, 2022 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.19: - :issue:`SERVER-62065` Upgrade path from 3.6 to 4.0 can leave chunk entries without history on the shards - :issue:`SERVER-60685` TransactionCoordinator may interrupt locally executing update with non-Interruption error category, leading to server crash - :issue:`SERVER-60682` TransactionCoordinator may block acquiring WiredTiger write ticket to persist its decision, prolonging transactions being in the prepared state - :issue:`SERVER-53335` Queries, updates, and deletes with non-"simple" collations may miss documents when using hashed sharding - :issue:`SERVER-40691` $nin:[...] queries are not indexed - `All JIRA issues closed in 4.2.19 `__ - :ref:`4.2.19-changelog` .. _4.2.18-release-notes: 4.2.18 - Jan 6, 2022 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.18: - :issue:`SERVER-61427` Unique index builds can cause a loss of availability during commit due to checking many false duplicates - :issue:`SERVER-56227` Add user-facing command to set allowMigrations to false for a sharded collection - :issue:`SERVER-59226` Deadlock when stepping down with a profile session marked as uninterruptible - :issue:`SERVER-56226` [v4.4] Introduce 'permitMigrations' field on config.collections entry to prevent chunk migrations from committing - :issue:`SERVER-54064` Sessions on arbiters accumulate and cannot be cleared out - `All JIRA issues closed in 4.2.18 `__ - :ref:`4.2.18-changelog` .. _4.2.17-release-notes: 4.2.17 - Sep 28, 2021 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.17: - :issue:`SERVER-59876`: Large delays in returning from libcrypto.so while establishing egress connections - :issue:`SERVER-59456`: Start the LDAPReaper threadpool - :issue:`SERVER-50549`: Transform connection-related error codes in proxied commands - :issue:`SERVER-49521`: fix tests in core/txn to use write concern "majority" for createIndexes commands run before starting transactions - :issue:`SERVER-44152`: Pre-warm connection pools in mongos - `All JIRA issues closed in 4.2.17 `__ - :ref:`4.2.17-changelog` .. _4.2.16-release-notes: 4.2.16 - Sep 13, 2021 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.16: - :issue:`SERVER-59074`: Do not acquire storage tickets just to set/wait on oplog visibility - :issue:`SERVER-54729`: MongoDB Enterprise Debian/Ubuntu packages should depend on libsasl2-modules and libsasl2-modules-gssapi-mit - :issue:`SERVER-39621`: Disabled chaining should enforce sync source change when the primary steps down even if the oplog fetcher isn't killed on sync source - :issue:`SERVER-34938`: Secondary slowdown or hang due to content pinned in cache by single oplog batch - :issue:`WT-7776`: Add a hard limit on the number of modify updates before we instantiate a complete update - `All JIRA issues closed in 4.2.16 `__ - :ref:`4.2.16-changelog` .. _4.2.15-release-notes: 4.2.15 - Jul 13, 2021 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.15: - :issue:`SERVER-57476`: Operation may block on prepare conflict while holding oplog slot, stalling replication indefinitely - :issue:`SERVER-56779`: Do not use the collection distributed lock for chunk merges - :issue:`SERVER-56240`: Turn on checkpointing for the keystore Data Store - :issue:`SERVER-56054`: Change minThreads value for replication writer thread pool to 0 - :issue:`SERVER-47699`: Change yield type used by range deleter from YIELD_MANUAL to YIELD_AUTO - `All JIRA issues closed in 4.2.15 `__ - :ref:`4.2.15-changelog` .. _4.2.14-release-notes: 4.2.14 - May 6, 2021 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.14: - :issue:`SERVER-54710`: Large number of $or clauses can create profiling entry exceeding max BSON size, causing the query to fail when it should not - :issue:`SERVER-54136`: Make the authenticate command respect enforceUserClusterSeparation - :issue:`SERVER-53566`: Investigate and reproduce "opCtx != nullptr && _opCtx == nullptr" invariant - :issue:`SERVER-52564`: Deadlock between step down and MongoDOperationContextSession - :issue:`WT-7373`: Improve slow random cursor operations on oplog - `All JIRA issues closed in 4.2.14 `__ - :ref:`4.2.14-changelog` .. _4.2.13-release-notes: 4.2.13 - Mar 19, 2021 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.13: - :issue:`SERVER-46686`: Explain does not respect maxTimeMS - :issue:`SERVER-46740`: establishCursors() must always drain the AsyncRequestsSender::_baton - :issue:`SERVER-46876`: During the eviction pressure, we should quit the compact operation instead of crashing the process - :issue:`SERVER-53394`: Make ShardingTaskExecutorPoolReplicaSetMatching default to disabled for MongoD - :issue:`WT-7028`: Sweep thread shouldn't lock during checkpoint gathering handles - `All JIRA issues closed in 4.2.13 `__ - :ref:`4.2.13-changelog` .. _4.2.12-release-notes: 4.2.12 - Jan 22, 2021 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.12: - :issue:`SERVER-40361`: Reduce memory footprint of plan cache entries - :issue:`SERVER-47863`: Initial Sync Progress Metrics - :issue:`SERVER-48471`: Hashed indexes may be incorrectly marked multikey and be ineligible as a shard key - :issue:`SERVER-50769`: server restarted after expr{"expr":"_currentApplyOps.getArrayLength() > 0","file":"src/mongo/db/pipeline/document_source_change_stream_transform.cpp","line":535}} - :issue:`SERVER-52654`: new signing keys not generated by the monitoring-keys-for-HMAC thread - :issue:`SERVER-52879`: Periodic operation latency spikes every 5 minutes due to closing idle cached WT sessions - `All JIRA issues closed in 4.2.12 `__ - :ref:`4.2.12-changelog` .. _4.2.11-release-notes: 4.2.11 - Nov 18, 2020 ~~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.11: - :issue:`SERVER-43664`: Speedup WiredTiger storage engine startup for many tables by optimizing WiredTigerUtil::setTableLogging() - :issue:`SERVER-45938`: Allow matching O/OU/DC in client x509 cert if clusterMode:keyFile - :issue:`SERVER-48523`: Unconditionally check the first entry in the oplog when attempting to resume a change stream - :issue:`SERVER-51120`: Find queries with SORT_MERGE incorrectly sort the results when the collation is specified - :issue:`WT-6507`: Exit cache eviction worker after our operation has timed out - `All JIRA issues closed in 4.2.11 `__ - :ref:`4.2.11-changelog` .. _4.2.10-release-notes: 4.2.10 - Oct 2, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.10: - :issue:`SERVER-26726`: Check number of arguments for createIndex() and throw error if more than two arguments - :issue:`SERVER-31368`: Log time spent waiting for other shards in merge cursors aggregation stage - :issue:`SERVER-37422`: Log balancer start and stop events in the actionlog - :issue:`SERVER-40317`: $facet execution has no limit on how much memory it can consume - :issue:`SERVER-43233`: Add ability to request only specific attribute(s) for the LDAP groups - :issue:`SERVER-47469`: applyOps does not take exclusive lock for views operation - :issue:`SERVER-50463`: Make PooledLDAPConnection::refresh take self-ownership - :issue:`SERVER-51041`: Throttle starting transactions for secondary reads - `All JIRA issues closed in 4.2.10 `__ - :ref:`4.2.10-changelog` .. _4.2.9-release-notes: 4.2.9 - Aug 21, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.9: - :issue:`SERVER-44051`: getShardDistribution() does not report "Collection XYZ is not sharded" on dropped but previously sharded collections - :issue:`SERVER-45610`: Some reads work while system is RECOVERING - :issue:`SERVER-47714`: Secondary asserts on system.profile collection with WiredTigerRecordStore::insertRecord 95: Operation not supported - :issue:`SERVER-48067`: Reduce memory consumption for unique index builds with large numbers of non-unique keys - :issue:`SERVER-49233`: Introduce a flag to toggle the logic for bumping collection's major version during split - :issue:`WT-6480`: Fix a bug where files without block modification information were repeatedly copied at each incremental backup - `All JIRA issues closed in 4.2.9 `__ - :ref:`4.2.9-changelog` .. _4.2.8-release-notes: 4.2.8 - Jun 15, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.8: - :issue:`SERVER-46897`: REMOVED node may never send heartbeat to fetch newest config - :issue:`SERVER-47799`: AsyncRequestsSender should update replica set monitor in between retries for InterruptedAtShutdown - :issue:`SERVER-47994`: Fix for numerical overflow in GeoHash - :issue:`SERVER-48307`: 3 Transactions that write to exactly one shard and read from one or more other shards may incorrectly indicate failure on retry after successful commit - :issue:`WT-6366`: Off-by-one overflow in block-modification bitmaps for incremental backup - `All JIRA issues closed in 4.2.8 `__ - :ref:`4.2.8-changelog` .. _4.2.7-release-notes: 4.2.7 - May 26, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.7: - :issue:`SERVER-47553`: mongos crashes due to client disconnecting when signing keys being refreshed - :issue:`SERVER-46487`: The mongos routing for scatter/gather ops can have unbounded latency - :issue:`SERVER-47190`: Shutdown command with force:true should ignore all stepdown errors - :issue:`SERVER-38731`: Ability to specify sync source read preference in initial sync - `All JIRA issues closed in 4.2.7 `_ - :ref:`4.2.7-changelog` .. _4.2.6-release-notes: 4.2.6 - Apr 21, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed in 4.2.6: - :issue:`SERVER-45119`: CollectionShardingState::getCurrentShardVersionIfKnown returns collection version instead of shard version - :issue:`SERVER-44892`: getShardDistribution should use $collStats agg stage instead of collStats command - :issue:`SERVER-43848`: find/update/delete w/o shard key predicate under txn with snapshot read can miss documents - :issue:`SERVER-42827`: Allow sessions collection to return OK for creating indexes if at least one shard returns OK and others return CannotImplicitlyCreateCollection - :issue:`SERVER-40805`: Indicate the reason for replanning in the log file - :issue:`SERVER-45389`: Add metrics tracking how often shards have inconsistent indexes - :issue:`SERVER-44689`: Add serverStatus counter for each use of an aggregation stage in a user's request - `All JIRA issues closed in 4.2.6 `_ - :ref:`4.2.6-changelog` .. _4.2.5-release-notes: 4.2.5 - Mar 26, 2020 ~~~~~~~~~~~~~~~~~~~~ .. note:: The release of version 4.2.4 was skipped due to an issue encountered during the release. However, the 4.2.5 release includes the fixes made in 4.2.4. Issues fixed in 4.2.5: - :issue:`SERVER-45770`: Add to information contained in logfile about "moveChunk.to" - `All JIRA issues closed in 4.2.5 `_ - :ref:`4.2.5-changelog` Issues fixed in 4.2.4: - :issue:`SERVER-44915`: Extend $indexStats output to include full index options and shard name - :issue:`SERVER-46121`: mongos crashes with invariant error after changing taskExecutorPoolSize - :issue:`SERVER-45137`: Increasing memory allocation in Top::record with high rate of collection creates and drops - :issue:`SERVER-44904`: Startup recovery should not delete corrupt documents while rebuilding unfinished indexes - :issue:`SERVER-44260`: Transaction can conflict with previous transaction on the session if the all committed point is held back - :issue:`SERVER-35050`: Don't abort collection clone due to negative document count - :issue:`SERVER-39112`: Primary drain mode can be unnecessarily slow - `All JIRA issues closed in 4.2.4 `_ - :ref:`4.2.4-changelog` .. _4.2.3-release-notes: 4.2.3 - Jan 27, 2020 ~~~~~~~~~~~~~~~~~~~~ Issues fixed: - :issue:`SERVER-42565`: Aggregations and find commands sort missing fields differently' - :issue:`SERVER-44174`: $push and $addToSet should restrict memory usage - :issue:`SERVER-40435`: A clearJumboFlag command to clear the jumbo flag - :issue:`SERVER-45270`: Increased vulnerability to slow DNS - :issue:`TOOLS-1952`: Use --forceTableScan by default when running against WiredTiger nodes - :issue:`TOOLS-2453`: Index keys not escaped correctly - :issue:`SERVER-45396`: fix the "me" field in isMaster responses when using splithorizon - :issue:`SERVER-45309`: Ensure bind credentials live longer than LDAP operations - :issue:`WT-5120`: Checkpoint hangs when reconciliation doesn't release the eviction generation - `All JIRA issues closed in 4.2.3 `_ - :ref:`4.2.3-changelog` .. note:: Fixed issues include those that resolve the following Common Vulnerabilities and Exposure (CVE): - CVE-2020-7921 (See :issue:`SERVER-45472`) .. _4.2.2-release-notes: 4.2.2 - Dec 9, 2019 ~~~~~~~~~~~~~~~~~~~~ Issues fixed: - :issue:`SERVER-31083`: Allow passing primary shard to "enableSharding" command for a new database - :issue:`SERVER-33272`: The DatabaseHolder::close() function no longer requires a global write lock and neither does the dropDatabase command - :issue:`SERVER-44050`: Arrays along 'hashed' index key path are not correctly rejected - :issue:`SERVER-43882`: Building indexes for startup recovery uses unowned RecordData after yielding its cursor - :issue:`SERVER-44617`: $regexFind crash when one of the capture group doesn't match the input but pattern matches - :issue:`SERVER-44721`: Shell KMS AWS support cannot decrypt responses - :issue:`SERVER-43860`: Pipeline style update in $merge can produce unexpected result - :issue:`WT-4961`: Checkpoints with cache overflow must keep history for reads - `All JIRA issues closed in 4.2.2 `_ - :ref:`4.2.2-changelog` .. _4.2.1-release-notes: 4.2.1 - Oct 18, 2019 ~~~~~~~~~~~~~~~~~~~~ Issues fixed: - :issue:`SERVER-37768`: Platform Support: Add Community & Enterprise Debian 10 x64 - :issue:`SERVER-37772`: Platform Support: Add Community & Enterprise RHEL 8 x64 - :issue:`SERVER-41506`: Track metrics associated with a node calling an election - :issue:`SERVER-41499`: Track number of elections called for each reason in serverStatus - :issue:`SERVER-42518`: Wildcard index plans miss results when the query path has multiple subsequent array indexes - :issue:`SERVER-42856`: Transactions with write can be sent to the wrong shard - `All JIRA issues closed in 4.2.1 `_ - :ref:`4.2.1-changelog` .. _4.2-distributed-txns: Distributed Transactions ------------------------ .. topic:: Distributed Transactions and Multi-Document Transactions Starting in MongoDB 4.2, the two terms are synonymous. Distributed transactions refer to multi-document transactions on sharded clusters and replica sets. Multi-document transactions (whether on sharded clusters or replica sets) are also known as distributed transactions starting in MongoDB 4.2. In version 4.2, MongoDB introduces distributed transactions. Distributed transactions: - Adds support for multi-document transactions on sharded clusters. - All members of the 4.2 sharded clusters must have :ref:`featureCompatibilityVersion ` of ``4.2``. - Clients :red:`must` use MongoDB drivers updated for MongoDB 4.2 - Incorporates the existing support for transactions on replica sets. - All members of the 4.2 replica set must have :ref:`featureCompatibilityVersion ` of ``4.2``. - Clients :red:`must` use MongoDB drivers updated for MongoDB 4.2 - Removes the 16MB total size limit for a transaction. In version 4.2, MongoDB creates as many oplog entries (maximum size ``16MB`` each) as necessary to encapsulate all write operations in a transaction. In MongoDB 4.0, MongoDB creates a single entry for all write operations in a transaction, thereby imposing a 16MB total size limit for a transaction. - Extends transaction support to deployments whose secondary members use the :doc:`in-memory storage engine `. That is, transactions are available for deployments that use the WiredTiger storage engine for the primary and either the WiredTiger or the :doc:`in-memory ` storage engine for the secondary members. In MongoDB 4.0, transactions are available for deployments that use the WiredTiger storage engine only. For more information, see :doc:`/core/transactions`. .. seealso:: :ref:`4.2 Transaction Compatibility Changes <4.2-compatibility-txn>` Removed MMAPv1 Storage Engine ----------------------------- MongoDB 4.2 removes the deprecated MMAPv1 storage engine. If your 4.0 deployment uses MMAPv1, you must change the deployment to :doc:`/core/wiredtiger` before upgrading to MongoDB 4.2. For details, see: - :doc:`/tutorial/change-standalone-wiredtiger` - :doc:`/tutorial/change-replica-set-wiredtiger` - :doc:`/tutorial/change-sharded-cluster-wiredtiger` .. _4.2-mmapv1-conf-options: MMAPv1 Specific Configuration Options ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB removes the following MMAPv1 specific configuration options: .. include:: /includes/removed-mmapv1-options.rst .. note:: Starting in version 4.2, MongoDB processes will not start with these options. Remove any MMAPv1 specific configuration options if using a WiredTiger deployment. MMAPv1 Specific Parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB removes the following MMAPv1 parameters: - ``newCollectionsUsePowerOf2Sizes`` - ``replIndexPrefetch`` MMAPv1 Specific Command ~~~~~~~~~~~~~~~~~~~~~~~ MongoDB removes the MMAPv1 specific ``touch`` command. MMAPv1 Specific Options for Binaries, Commands and Methods ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB removes the MMAPv1 specific options: - ``noPadding`` and ``usePowerOf2Sizes`` for :dbcommand:`collMod` - ``verbose`` for :dbcommand:`collStats` - ``flags`` for :dbcommand:`create` - ``paddingFactor``, ``paddingBytes``, ``preservePadding`` for the :method:`db.createCollection()` method and the :dbcommand:`compact` command. - ``repair`` for :binary:`~bin.mongodump` Removed Commands and Methods ---------------------------- .. list-table:: :header-rows: 1 * - Removed Command - Removed Method - Notes * - ``group`` - ``db.collection.group()`` - Use :method:`db.collection.aggregate()` with the :pipeline:`$group` stage instead. * - ``eval`` - - The MongoDB 4.2 :binary:`~bin.mongo` shell methods :method:`db.eval()` and :method:`db.collection.copyTo()` can only be run when connected to MongoDB 4.0 or earlier. * - ``copydb`` - - The corresponding :binary:`~bin.mongo` shell helpers ``db.copyDatabase()`` can only be run when connected to MongoDB 4.0 or earlier. As an alternative, users can use :binary:`~bin.mongodump` and :binary:`~bin.mongorestore` (see :ref:`mongodump-example-copy-clone-database`) or write a script using the drivers. * - ``clone`` - - The corresponding :binary:`~bin.mongo` shell helpers ``db.cloneDatabase()`` can only be run when connected to MongoDB 4.0 or earlier. As an alternative, users can use :binary:`~bin.mongodump` and :binary:`~bin.mongorestore` (see :ref:`mongodump-example-copy-clone-database`) or write a script using the drivers. * - ``geoNear`` - - Use :method:`db.collection.aggregate()` with the :pipeline:`$geoNear` stage instead. For more information, see :ref:`4.2-compat-remove-geoNear`. * - ``parallelCollectionScan`` - - * - ``repairDatabase`` - ``db.repairDatabase()`` - For more information, see :ref:`4.2-compat-remove-repairDatabase`. * - ``getPrevError`` - ``db.getPrevError()`` - Remove ``maxScan`` Option ~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB removes the deprecated option ``maxScan`` for the :dbcommand:`find` command and the :binary:`~bin.mongo` shell helper ``cursor.maxScan()``. Use either the ``maxTimeMS`` option for the :dbcommand:`find` command or the helper :method:`cursor.maxTimeMS()` instead. .. _4.2-drivers: MongoDB Drivers --------------- The following drivers are feature compatible [#fle]_ with MongoDB 4.2: .. include:: /includes/list-4.2-drivers.rst .. [#fle] For a complete list of official 4.2+ compatible drivers with support for Client-Side Field Level Encryption, see :ref:`field-level-encryption-drivers`. Retryable Reads ~~~~~~~~~~~~~~~ Retryable reads allow MongoDB 4.2+ compatible drivers to automatically retry certain read operations a single time if they encounter certain network or server errors. See :ref:`retryable-reads` for more information. Sharded Clusters ---------------- Mutable Shard Key Values ~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/limits-sharding-shardkey-document-immutable.rst Backups ~~~~~~~ .. include:: /includes/extracts/sharded-clusters-backup-restore-mongodump-mongorestore-restriction.rst Balancer State and Autosplit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-balancer-autosplit.rst ``mongos`` / ``mongod`` Connection Pool ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-mongos-repl-connection-pool.rst Sharded Collections and Replace Documents ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-sharded-collection-replacement.rst .. _4.2-security: Security Improvements --------------------- Resolved Common Vulnerabilities and Exposures ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 includes fixes that resolve the following Common Vulnerabilities and Exposures (CVEs): - CVE-2019-2389 (See :issue:`SERVER-40563`) - CVE-2019-2386 (See :issue:`SERVER-38984`) .. _4.2-tls: New ``TLS`` Options ~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds ``TLS`` options for the :ref:`mongod `, the :ref:`mongos `, and the :ref:`mongo shell` to replace the corresponding ``SSL`` options (deprecated in 4.2). The new TLS options provide **identical** functionality as the deprecated ``SSL`` options as MongoDB has always supported TLS 1.0 and later. - For the command-line TLS options, refer to the :ref:`mongod `, :ref:`mongos `, and :ref:`mongo shell ` pages. - For the corresponding ``mongod`` and ``mongos`` configuration file options, refer to the :ref:`configuration file ` page. - For the connection string ``tls`` options, refer to the :ref:`connection string ` page. .. tip:: Most of the new ``TLS`` option names are similar to the ``SSL`` option name; e.g. :option:`--tlsMode ` instead of :option:`--sslMode `. The exceptions are: - :setting:`net.tls.certificateKeyFile` vs. :setting:`net.ssl.PEMKeyFile` - :setting:`net.tls.certificateKeyFilePassword` vs. :setting:`net.ssl.PEMKeyPassword` - :option:`--tlsCertificateKeyFile ` vs. :option:`--sslPEMKeyFile ` - :option:`--tlsCertificateKeyFilePassword ` vs. :option:`--sslPEMKeyPassword ` .. seealso:: :ref:`4.2-tlsclustercafile` Deprecated ``SSL`` Options ~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 deprecates the ``SSL`` options for the :ref:`mongod `, the :ref:`mongos `, and the :ref:`mongo shell` as well as the corresponding :ref:`net-ssl-conf-options` configuration file options. Use the new :ref:`TLS <4.2-tls>` options instead. New ``tls`` Parameters ~~~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 :widths: 40 60 * - New Parameter - Description * - :parameter:`tlsWithholdClientCertificate` - Available for :binary:`~bin.mongod` and :binary:`~bin.mongos`, the parameter can be set to ``true`` to stop the instance from sending its ``TLS`` certificate when initiating intra-cluster communications with other :binary:`mongod ` or :binary:`mongos ` instances. For details, see :parameter:`tlsWithholdClientCertificate`. * - :parameter:`tlsX509ClusterAuthDNOverride` - Available for :binary:`~bin.mongod` and :binary:`~bin.mongos`, the parameter can be set to an alternative certificate ``DN`` to use for :doc:`x.509 membership authentication `. For details, see :parameter:`tlsX509ClusterAuthDNOverride`. You can use this parameter for a rolling update of certificates to new certificates that contain a new ``DN`` value. See :doc:`/tutorial/rotate-x509-membership-certificates`. .. _4.2-tlsclustercafile: New ``tlsClusterCAFile`` Option ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds the :option:`--tlsClusterCAFile ` option/:setting:`net.tls.clusterCAFile` for :binary:`~bin.mongod` and :binary:`~bin.mongos`, which specifies a :file:`.pem` file for validating the TLS certificate from a client establishing a connection. This lets you use separate Certificate Authorities to verify the client to server and server to client portions of the TLS handshake. .. seealso:: :ref:`4.2-tls` Forward Secrecy ~~~~~~~~~~~~~~~ Starting in version 4.2 on Linux: - If the platform's OpenSSL supports automatic curve selection for Elliptic Curve Diffie-Hellman, MongoDB enables support for :ref:`ecdhe`. - If the platform's OpenSSL does not support automatic curve selection for Elliptic Curve Diffie-Hellman, MongoDB attempts to enable ECDHE support using ``prime256v1`` as the named curve. - If support for ECDHE is enabled, MongoDB attempts to enable support for :ref:`dhe` if :ref:`dhe` is not explicitly enabled. In earlier versions of MongoDB (3.6.14+ and 4.0.3+), MongoDB enables support for Ephemeral Elliptic Curve Diffie-Hellman (ECDHE) if, during compile time, the Linux platform's OpenSSL supports automatic curve selection of ECDH parameters. On Windows and macOS, MongoDB's support for ECDHE and DH remain unchanged from earlier versions; that is, support is implicit through the use of the platform's respective native TLS/SSL OS libraries. For more information, see :ref:`tls-forward-secrecy`. ``passwordPrompt()`` ~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-passwordPrompt.rst For example: .. code-block:: javascript db.createUser( { user:"user123", pwd: passwordPrompt(), // Instead of specifying the password in cleartext roles:[ "readWrite" ] } ) Keyfile Format Change to YAML ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-keyfile-fmt.rst The new format allows for rolling upgrade of the keys without downtime. See :doc:`/tutorial/rotate-key-replica-set` and :doc:`/tutorial/rotate-key-sharded-cluster`. ``libldap`` and ``libldap_r`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-libldap.rst Encrypted Storage Engine ~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-ese-key-rollover.rst For more information, see :ref:`encrypted storage engine ` and :option:`--eseDatabaseKeyRollover `. Client-Side Field Level Encryption ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The official :ref:`MongoDB 4.2+ compatible drivers ` provide a client-side field level encryption framework. Applications can encrypt fields in documents *prior* to transmitting data over the wire to the server. Only applications with access to the correct encryption keys can decrypt and read the protected data. Deleting an encryption key renders all data encrypted with that key as permanently unreadable. For a complete list of official 4.2+ compatible drivers with support for client-side field level encryption, see :ref:`field-level-encryption-drivers`. For an end-to-end procedure for configuring field level encryption using select MongoDB 4.2+ compatible drivers, see the :ecosystem:`Client Side Field Level Encryption Guide `. Explicit (manual) encryption of fields Official MongoDB 4.2+ compatible drivers and the MongoDB 4.2 or later :binary:`~bin.mongo` shell support explicitly encrypting or decrypting fields with a specific data encryption key and encryption algorithm. Applications must modify any code associated with constructing read and write operations to include encryption/decryption logic via the driver encryption library. Applications are responsible for selecting the appropriate data encryption key for encryption/decryption on a per-operation basis. For more information, see :doc:`/core/security-explicit-client-side-encryption`. Automatic encryption of fields .. include:: /includes/extracts/csfle-enterprise-atlas-only.rst Official MongoDB 4.2+ compatible drivers and the MongoDB 4.2 or later :binary:`~bin.mongo` shell support automatically encrypting fields in read and write operations. Applications must create a database connection object (e.g. ``MongoClient``) with the automatic encryption configuration settings. The configuration settings must include automatic encryption encryption rules using a strict subset of the `JSON Schema Draft 4 standard syntax `_ and encryption-specific schema keywords. Applications do not have to modify code associated with constructing the read/write operation. See :ref:`field-level-encryption-json-schema` for complete documentation on automatic encryption rules. For more information, see :doc:`/core/security-automatic-client-side-encryption`. For complete documentation on client-side field level encryption, see :doc:`/core/security-client-side-encryption`. General Security Enhancements ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Add :authaction:`serverStatus` to the :authrole:`backup` built-in role. - To connect a client over TLS/SSL connection, MongoDB 4.2 supports matching by IP addresses as well as DNS for Subject Alternative Name (SAN) matching. For example, a :binary:`~bin.mongod` instance's x.509 certificate has the following SAN: .. code-block:: none :copyable: false X509v3 Subject Alternative Name: DNS:hostname.example.com, DNS:localhost, IP Address:127.0.0.1 Then, to connect a :binary:`~bin.mongo` shell to the instance, you can specify the host of ``127.0.0.1`` or the DNS names: - .. code-block:: none :copyable: false mongo "mongodb:\\127.0.0.1:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ... - .. code-block:: none :copyable: false mongo "mongodb:\\hostname.example.com:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ... - .. code-block:: none :copyable: false mongo "mongodb:\\localhost:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ... In previous versions, MongoDB only supported DNS entries for SAN matching. - .. code-block:: none :copyable: false mongo "mongodb:\\hostname.example.com:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ... - .. code-block:: none :copyable: false mongo "mongodb:\\localhost:27017\test" --tls --tlsCAFile /etc/ssl/ca.pem ... LDAP Query Template ``{PROVIDED_USER}`` Token ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in version 4.2, MongoDB Enterprise adds a new token ``{PROVIDED_USER}`` that can be used in :setting:`security.ldap.authz.queryTemplate`. When used in the template, MongoDB substitutes the supplied username, i.e. before either authentication or :setting:`LDAP transformation `. .. _4.2-agg: Aggregation Improvements ------------------------ .. _4.2-agg-merge: On-Demand Materialized View (``$merge`` Stage) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds the :pipeline:`$merge` aggregation stage. With the new stage you can: - Can output to a collection in the same or different database. - Can incorporate results (merge documents, replace documents, keep existing documents, fail the operation, process documents with an custom update pipeline) into an existing collection. - Can output to an existing sharded collection. The new stage allows users to create :doc:`on-demand materialized views `, where the content of the output collection can be incrementally updated each time the pipeline is run. .. _4.2-trigonometry-expressions: Aggregation Trigonometry Expressions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds new trigonometry expressions for use in aggregation pipelines. .. include:: /includes/extracts/agg-operators-trigonometry.rst .. _4.2-round-trig-expressions: Aggregation Arithmetic Expressions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds the :expression:`$round` aggregation expression. Use :expression:`$round` to round numerical values to a specific digit or decimal place. MongoDB 4.2 adds expanded functionality and new syntax to :expression:`$trunc`. Use :expression:`$trunc` with the new syntax to truncate numerical values to a specific digit or decimal place. .. _4.2-agg-regex-operators: Aggregation Regular Expressions (regex) Operators ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds the following regular expression (regex) pattern matching operators for use in the aggregation pipeline: .. list-table:: :header-rows: 1 :widths: 20 80 * - Operator - Description * - :expression:`$regexFind` - Applies a regular expression (regex) to a string and returns information on the *first* matched substring. * - :expression:`$regexFindAll` - Applies a regular expression (regex) to a string and returns information on all matched substrings. * - :expression:`$regexMatch` - Applies a regular expression (regex) to a string and returns ``true`` if a match is found and ``false`` if a match is not found. Prior to MongoDB 4.2, aggregation pipeline can only use the query operator :query:`$regex` in the :pipeline:`$match` stage. New Stages ~~~~~~~~~~ MongoDB 4.2 adds the following new aggregation pipeline stages: .. list-table:: :header-rows: 1 :widths: 15 85 * - New Stage - Description * - :pipeline:`$merge` - Writes the aggregation results to a collection. The :pipeline:`$merge` stage can incorporate results (merge documents, replace documents, keep existing documents, fail the operation, process documents with an custom update pipeline) into an existing collection. * - :pipeline:`$planCacheStats` - Provides :doc:`plan cache ` information for a collection. .. include:: /includes/extracts/4.2-changes-planCacheStats-pref.rst .. seealso:: :ref:`4.2-deprecated-plan-cache` * - :pipeline:`$replaceWith` - Replaces the input document with the specified document. The operation replaces all existing fields in the input document, including the ``_id`` field. The new :pipeline:`$replaceWith` stage is an alias to the :pipeline:`$replaceRoot` stage. * - :pipeline:`$set` - Adds new fields to documents. The stage outputs documents that contains all existing fields from the input documents as well as the newly added fields. The new :pipeline:`$set` stage is an alias to the :pipeline:`$addFields` stage. * - :pipeline:`$unset` - Excludes fields from documents. The new :pipeline:`$unset` stage is an alias to the :pipeline:`$project` stage that excludes fields. New Variables ~~~~~~~~~~~~~ MongoDB 4.2 adds the following new aggregation pipeline variables: .. list-table:: :header-rows: 1 :widths: 30 70 * - Variable - Description * - :variable:`NOW` - Returns the current datetime value. * - :variable:`CLUSTER_TIME` - Returns the current timestamp value. *Only available on replica sets and sharded clusters.* Availability ~~~~~~~~~~~~ Starting in MongoDB 4.2, you can use the aggregation pipeline for updates in: .. include:: /includes/table-update-with-aggregation-availability.rst For the updates, the pipeline can consist of the following stages: .. include:: /includes/list-update-agg-stages.rst Using the aggregation pipeline allows for a more expressive update statement, such as expressing conditional updates based on current field values or updating one field using the value of another field(s). See the individual reference pages for details and examples. .. seealso:: - :doc:`/tutorial/update-documents-with-aggregation-pipeline` - :ref:`4.2-compat-agg` .. _4.2-change-stream: Change Stream ------------- ``startAfter`` Option for Change Streams ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds ``startAfter`` as an option for :ref:`changeStreams`, which starts a new change stream after the event indicated by a resume token. With this option, you can start a change stream from an :ref:`invalidate event `, thereby guaranteeing no missed notifications after the previous stream was invalidated. Change Streams Resume Tokens ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 uses the version 1 (i.e. ``v1``) change streams :ref:`resume tokens `, introduced in version 4.0.7. .. include:: /includes/extracts/4.2-changes-change-stream-modification-error.rst Availability ~~~~~~~~~~~~ .. include:: /includes/extracts/changestream-rc-majority-4.2.rst Change Stream Pipeline ~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, you can use additional stages in the :ref:`change stream aggregation pipeline ` to modify the change stream output (i.e. the event documents): - :pipeline:`$replaceWith` - :pipeline:`$set` - :pipeline:`$unset` .. include:: /includes/extracts/4.2-changes-change-stream-modification-error.rst .. seealso:: :ref:`Change Streams Compatibility Changes <4.2-compatibility-change-streams>` Update Enhancements ------------------- .. _4.2-update-using-aggregation: Update and Aggregation ~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, you can use the aggregation pipeline for updates in: .. include:: /includes/table-update-with-aggregation-availability.rst For the updates, the pipeline can consist of the following stages: .. include:: /includes/list-update-agg-stages.rst Using the aggregation pipeline allows for a more expressive update statement, such as expressing conditional updates based on current field values or updating one field using the value of another field(s). See the individual reference pages for details and examples. Update and Hint ~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the :dbcommand:`update` command and the associated :binary:`~bin.mongo` shell method :method:`db.collection.update()` can accept a hint argument to specify the index to use. See: - :dbcommand:`update` - :method:`db.collection.update()` Sharded Collections and Replace Documents ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-sharded-collection-replacement.rst .. _wildcard-releasenotes: Wildcard Indexes ---------------- MongoDB 4.2 introduces :ref:`wildcard indexes ` for supporting queries against fields whose names are unknown or arbitrary. .. include:: /includes/extracts/wildcard-index-summary.rst .. include:: /includes/extracts/wildcard-index-planning.rst .. include:: /includes/extracts/wildcard-index-fcv.rst You can create a wildcard index using the :dbcommand:`createIndexes` database command or its shell helpers :method:`db.collection.createIndex()` and :method:`db.collection.createIndexes()`. For examples of creating a wildcard index, see :ref:`createIndex-method-wildcard-examples`. See :ref:`wildcard-index-core` for complete documentation. Platform Support ---------------- - MongoDB 4.2 adds support for: - Ubuntu 18.04 on ARM64 - MongoDB 4.2 removes support for: - Debian 8 - Ubuntu 14.04 - Ubuntu 16.04 ARM64 for MongoDB Community Edition - Ubuntu 16.04 POWER/PPC64LE (Also removed in version 3.6.13 and 3.4.21) - macOS 10.11 See :ref:`mongodb-supported-platforms`. MongoDB Tools ------------- FIPS Mode ~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-fips.rst ``--uri`` Option ~~~~~~~~~~~~~~~~ Starting in version 4.2, - For the following Database Tools, if the write concern is specified in both the ``--uri`` connection string and the ``--writeConcern`` option, the ``--writeConcern`` option overrides the one in the connection string: - :binary:`~bin.mongofiles` - :binary:`~bin.mongoimport` - :binary:`~bin.mongorestore` - For the following Database Tools, if the read preference is specified in both the ``--uri`` connection string and the ``--readPreference`` option, the ``--readPreference`` option overrides the one in the connection string: - :binary:`~bin.mongodump` - :binary:`~bin.mongoexport` - :binary:`~bin.mongofiles` Extended JSON v2 ~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-extended-json-v2.rst For details on MongoDB extended JSON v2, see :doc:`/reference/mongodb-extended-json`. .. seealso:: :ref:`4.2-compatibility-v2-query` ``mongofiles`` ~~~~~~~~~~~~~~ The :binary:`~bin.mongofiles` command ``get_id`` and ``delete_id`` can accept both ObjectId or non-ObjectId values for the ``_id``. Bulk Operations for ``mongoimport`` and ``mongorestore`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``mongoimport`` ``````````````` Starting in version 4.2: - :binary:`~bin.mongoimport` uses maximum batch size of 100,000 to perform bulk insert/upsert operations. - :binary:`~bin.mongoimport` by default, continues when it encounters duplicate key and document validation errors. To ensure that the program stops on these errors, specify :option:`--stopOnError `. - Specifying :option:`--maintainInsertionOrder ` for :binary:`~bin.mongoimport`: - Maintains document insertion order using ordered bulk write operations; i.e. both the batch order and document order within the batches are maintained. In earlier versions, only the batch order is maintained; document order within batches are not maintained. - Enables :option:`--stopOnError ` and sets :option:`numInsertionWorkers ` to 1. ``mongorestore`` ```````````````` Starting in version 4.2: - :binary:`~bin.mongorestore` by default, continues when it encounters duplicate key and document validation errors. To ensure that the program stops on these errors, specify :option:`--stopOnError `. - Specifying :option:`--maintainInsertionOrder ` for :binary:`~bin.mongorestore`: - Maintains document insertion order using ordered bulk write operations; i.e. both the batch order and document order within the batches are maintained. In earlier versions, only the batch order is maintained; document order within batches are not maintained. - Enables :option:`--stopOnError ` and sets :option:`--numInsertionWorkersPerCollection ` to 1. .. _4.2-finer-grained-locks-for-DDL: Lock Optimization for Specific DDL Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting with MongoDB 4.2, the following operations take an exclusive collection lock instead of an exclusive database lock: .. list-table:: * - Commands - Methods * - :dbcommand:`create` - :method:`db.createCollection()` :method:`db.createView()` * - :dbcommand:`createIndexes` - :method:`db.collection.createIndex()` :method:`db.collection.createIndexes()` * - :dbcommand:`drop` - :method:`db.collection.drop()` * - :dbcommand:`dropIndexes` - :method:`db.collection.dropIndex()` :method:`db.collection.dropIndexes()` * - :dbcommand:`renameCollection` - :method:`db.collection.renameCollection()` Prior to MongoDB 4.2, these operations took an exclusive lock on the database, blocking all operations on the database *and* its collections until the operation completed. .. _4.2-traffic-recorder: In earlier versions, ``get_id`` and ``delete_id`` can only accept ObjectId values for the ``_id``. Monitoring ---------- Starting in version 4.2, the :ref:`Storage Node Watchdog ` is available in both the MongoDB Community edition and the MongoDB Enterprise edition. In earlier versions, the feature is available in the MongoDB Enterprise edition only. Flow Control ------------ MongoDB 4.2 introduces a flow control mechanism to control the rate at which the primary applies its writes in order to keep the :data:`majority committed ` lag under a specified maximum value. Flow control is :parameter:`enabled ` by default. .. note:: .. include:: /includes/extracts/4.2-changes-flow-control-requirements.rst For more information, see :ref:`replication-flow-control`. Logging and Diagnostics ----------------------- Logging ~~~~~~~ - Added :data:`INITSYNC` component to log messages. - Added :data:`ELECTION` component to log messages. - For debug messages, include the verbosity level (i.e. ``D``\ [1-5]). For example, if verbosity level is 2, MongoDB logs ``D2``. In previous versions, MongoDB log messages only specified ``D`` for Debug level. - When logging to :option:`syslog `, the format of the message text includes the :ref:`component `. For example: .. code-block:: none ... ACCESS [repl writer worker 5] Unsupported modification to roles collection ... Previously, the :option:`syslog ` message text did not include the component. For example: .. code-block:: none ... [repl writer worker 1] Unsupported modification to roles collection ... - MongoDB 4.2 adds a ``usedDisk`` indicator to the :doc:`profiler log messages ` and :doc:`diagnostic log messages ` for the :dbcommand:`aggregate` operation. The ``usedDisk`` indicates whether any stages of an :dbcommand:`aggregate` operation wrote data to temporary files due to memory restrictions. For more information on aggregation memory restrictions, see :ref:`agg-memory-restrictions`. - .. _slow-oplog: Starting in version 4.2 (also available starting in 4.0.6), secondary members of a replica set now log oplog entries that take longer than the slow operation threshold to apply. These messages are :option:`logged ` for the secondaries under the :data:`REPL` component with the text ``applied op: took ms``. .. code-block:: none 2018-11-16T12:31:35.886-0500 I REPL [repl writer worker 13] applied op: command { ... }, took 112ms The slow oplog application logging on secondaries are: - Not affected by the :setting:`~operationProfiling.slowOpSampleRate`; i.e. all slow oplog entries are logged by the secondary. - Not affected by the :parameter:`logLevel`/:setting:`systemLog.verbosity` level (or the :setting:`systemLog.component.replication.verbosity` level); i.e. for oplog entries, the secondary logs only the slow oplog entries. Increasing the verbosity level does not log all oplog entries. - Not captured by the :doc:`profiler ` and not affected by the profiling level. For more information on setting the slow operation threshold, see - :option:`mongod --slowms` - :setting:`~operationProfiling.slowOpThresholdMs` - The :dbcommand:`profile` command or :method:`db.setProfilingLevel()` shell helper method. - Starting in MongoDB 4.2, the :dbcommand:`getLog` command truncates any event that contains more than 1024 characters. In earlier versions, :dbcommand:`getLog` truncates after 512 characters. - .. include:: /includes/extracts/4.2-changes-log-storage.rst - .. include:: /includes/extracts/4.2-changes-log-query-shapes-plan-cache-key.rst See :ref:`4.2-query-plans`. .. _4.2-currentOp: ``currentOp`` ~~~~~~~~~~~~~ MongoDB 4.2 adds a new option ``idleCursors`` to the :pipeline:`$currentOp` aggregation stage in order to return information on idle cursors. In addition, MongoDB 4.2 adds the following new fields to the documents returned from the :pipeline:`$currentOp` aggregation stage, :dbcommand:`currentOp` command, and :method:`db.currentOp()` helper: .. list-table:: :header-rows: 1 * - ``$currentOp`` - ``currentOp``/``db.currentOp()`` - Description * - :data:`$currentOp.type` - :data:`currentOp.type` - Specifies whether the reported operation is an ``op``, ``idleSession``, or ``idleCursor``. * - :data:`$currentOp.cursor` - :data:`currentOp.cursor` - Specifies cursor details. Available when returning ``getmore`` operations or ``idleCursor`` information. * - :data:`$currentOp.effectiveUsers` - :data:`currentOp.effectiveUsers` - Specifies users associated with the operation. * - :data:`$currentOp.prepareReadConflicts` - :data:`currentOp.prepareReadConflicts` - Specifies the number of times the current operation had to wait for a prepared transaction with a write to commit or abort. * - :data:`$currentOp.runBy` - :data:`currentOp.runBy` - Specifies users that are impersonating the effective users for the operation. * - :data:`$currentOp.writeConflicts` - :data:`currentOp.writeConflicts` - Specifies the number of times the current operation conflicted with another write operation. See also :ref:`4.2 current op compatibility changes <4.2-current-op-compat>` ``serverStatus`` Metrics ~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the :dbcommand:`serverStatus` command and the :binary:`~bin.mongo` shell method :method:`db.serverStatus()` include the following output changes: .. list-table:: * - Updates to - Changes * - :serverstatus:`shardingStatistics` - Added new fields: - :serverstatus:`shardingStatistics.countDocsClonedOnRecipient` - :serverstatus:`shardingStatistics.countDocsClonedOnDonor` - :serverstatus:`shardingStatistics.countDocsDeletedOnDonor` - :serverstatus:`shardingStatistics.countRecipientMoveChunkStarted` - :serverstatus:`shardingStatistics.countDonorMoveChunkLockTimeout` * - :serverstatus:`metrics` - Added new fields: - :serverstatus:`metrics.aggStageCounters` (Starting in 4.2.6 and also available in 4.0.19+) * - :serverstatus:`metrics.repl.network`: - Added new fields: - :serverstatus:`metrics.repl.network.notMasterLegacyUnacknowledgedWrites` - :serverstatus:`metrics.repl.network.notMasterUnacknowledgedWrites` * - :serverstatus:`metrics.repl` - Added new :serverstatus:`metrics.repl.stepDown` metrics: - :serverstatus:`metrics.repl.stepDown.userOperationsKilled` - :serverstatus:`metrics.repl.stepDown.userOperationsRunning` * - :serverstatus:`transactions` - - Now available for :binary:`~bin.mongos` (previously only for :binary:`~bin.mongod`) - Added new fields for :binary:`~bin.mongod` instances: - :serverstatus:`transactions.totalPrepared` - :serverstatus:`transactions.totalPreparedThenCommitted` - :serverstatus:`transactions.totalPreparedThenAborted` - :serverstatus:`transactions.currentPrepared` * - :serverstatus:`logicalSessionRecordCache` - Added new field :serverstatus:`logicalSessionRecordCache.sessionCatalogSize` * - :serverstatus:`locks` - Separate ``ParallelBatchWriterMode`` from ``Global`` lock information. Add ``ReplicationStateTransition`` lock information. Replica Set Status Metrics ~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in version MongoDB 4.2, :dbcommand:`replSetGetStatus` and its :binary:`~bin.mongo` shell helper :method:`rs.status()` return: - The IP address, :data:`replSetGetStatus.members[n].ip` for the replica set members. - ISODate-formatted date string fields that correspond to the various :data:`replSetGetStatus.optimes`. .. list-table:: :header-rows: 1 * - New ISODate-Formatted Date String Field - Corresponding Optime Field * - :data:`~replSetGetStatus.optimes.lastCommittedWallTime` - :data:`~replSetGetStatus.optimes.lastCommittedOpTime` * - :data:`~replSetGetStatus.optimes.readConcernMajorityWallTime` - :data:`~replSetGetStatus.optimes.readConcernMajorityOpTime` * - :data:`~replSetGetStatus.optimes.lastAppliedWallTime` - :data:`~replSetGetStatus.optimes.appliedOpTime` * - :data:`~replSetGetStatus.optimes.lastDurableWallTime` - :data:`~replSetGetStatus.optimes.durableOpTime` MongoDB 4.2 deprecates the field :data:`~replSetGetStatus.lastStableCheckpointTimestamp` returned from :dbcommand:`replSetGetStatus`. Lock Diagnostics Reporting ~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-global-lock-reporting.rst ``collStats`` Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the :pipeline:`$collStats` aggregation, the :dbcommand:`collStats` command, and the :binary:`~bin.mongo` shell helper :method:`db.collection.stats` return information on indexes that are currently being built. For details, see: - :data:`collStats.nindexes` - :data:`collStats.indexDetails` - :data:`collStats.indexBuilds` - :data:`collStats.totalIndexSize` - :data:`collStats.indexSizes` Starting in MongoDB 4.2, the :pipeline:`$collStats` aggregation, the :dbcommand:`collStats` command, and the :binary:`~bin.mongo` shell helper :method:`db.collection.stats` return the ``scaleFactor`` used to scale the various size data. ``dbStats`` Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the :dbcommand:`dbStats` command, and the :binary:`~bin.mongo` shell helper :method:`db.stats()` return the ``scaleFactor`` used to scale the various size data. General Improvements -------------------- Externally Sourced Values for Configuration Files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/fact-expansion-directive-intro.rst The following expansion directives are available: .. list-table:: :header-rows: 1 :widths: 30 70 * - Expansion Directive - Description * - :configexpansion:`__rest` - Allows users to specify a ``REST`` endpoint as the external source for configuration file options *or* the full configuration file. * - :configexpansion:`__exec` - Allows users to specify a shell or terminal command as the external source for configuration file options *or* the full configuration file. For complete documentation, see :ref:`externally-sourced-values`. ``outputConfig`` Option ~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds the :option:`--outputConfig ` option for :binary:`~bin.mongod` and :binary:`~bin.mongos`. The option outputs to ``stdout`` the :binary:`~bin.mongod`/:binary:`~bin.mongos` instance's configuration, in YAML format. If the configuration uses any :ref:`externally-sourced-values`, the option returns the resolved value for those options. .. warning:: This may include any configured passwords or secrets previously obfuscated through the external source. For usage examples, see: - :ref:`expansion-directive-output` - :doc:`/tutorial/convert-command-line-options-to-yaml` Remove Index Key Size Limit ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, for :ref:`featureCompatibilityVersion ` set to ``"4.2"`` or greater, MongoDB removes the :limit:`Index Key Limit`. For fCV set to ``"4.0"``, the limit still applies. .. seealso:: :ref:`4.2 Indexes Compatibility Changes <4.2-index-compat-changes>` Remove Index Name Length Limit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-index-names.rst .. seealso:: :ref:`4.2 Indexes Compatibility Changes <4.2-index-compat-changes>`, :ref:`4.2-compatibility-enabled` Improvements to ``dropIndexes`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Drop multiple indexes ````````````````````` Starting in MongoDB 4.2, you can specify multiple indexes to the :dbcommand:`dropIndexes` command and its :binary:`~bin.mongo` shell helper :method:`db.collection.dropIndexes()`. To specify multiple indexes to drop, pass an array of index names to :dbcommand:`dropIndexes`/:method:`db.collection.dropIndexes()`. Kill related queries only ````````````````````````` .. include:: /includes/extracts/fact-releaseNotes-dropIndexes-query-behavior.rst ``zstd`` Availability ~~~~~~~~~~~~~~~~~~~~~ Starting in version 4.2, MongoDB supports :term:`zstd` for: - block compression. See :setting:`storage.wiredTiger.collectionConfig.blockCompressor`. - journal compression. See :setting:`storage.wiredTiger.engineConfig.journalCompressor`. - network compression. See :setting:`net.compression.compressors`. - For clients using the MongoDB drivers, they must use drivers updated for MongoDB 4.2. - The network compressors for :binary:`~bin.mongod` and :binary:`~bin.mongos` default to both ``snappy,zstd,zlib`` compressors, in that order. In version 4.0, :binary:`~bin.mongod` and :binary:`~bin.mongos` enable network compression by default with ``snappy`` as the compressor. ``bulkWrite()`` Error Handling inside Transactions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-bulkWrite-txn-error-handling.rst .. _4.2-query-plans: Query Plan Improvements ----------------------- Plan Cache States ~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the cache entry is associated with a state: - :ref:`Missing ` - :ref:`Inactive ` - :ref:`Active ` Associating states with entries helps reduce the likelihood that sub-optimal cache entries remain in the cache. For more information, see :doc:`/core/query-plans`. ``queryHash`` and ``planCacheKey`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - .. _4.2-query-hash: ``queryHash`` .. include:: /includes/extracts/4.2-changes-query-shapes.rst - .. _4.2-plan-cache-key: ``planCacheKey`` .. include:: /includes/extracts/4.2-changes-plan-cache-key.rst .. seealso:: :ref:`plan-cache-key` - The ``queryHash`` and ``planCacheKey`` are available in: - :doc:`profiler entry ` fields :data:`~system.profile.queryHash` and :data:`~system.profile.planCacheKey` the logged query operations. - :ref:`diagnostic log messages (i.e. mongod/mongos log messages)` for the logged query operations. - :doc:`explain() output ` fields: :data:`~explain.queryPlanner.queryHash` and :data:`~explain.queryPlanner.planCacheKey` The fields are also available in operations that return information about the query plan cache: - :pipeline:`$planCacheStats` aggregation stage (*New in MongoDB 4.2*) - :method:`PlanCache.listQueryShapes()` method/:dbcommand:`planCacheListQueryShapes` command (*Deprecated in MongoDB 4.2*) - :method:`PlanCache.getPlansByQuery()` method/:dbcommand:`planCacheListPlans` command (*Deprecated in MongoDB 4.2*) .. seealso:: :ref:`4.2-deprecated-plan-cache` ``$regex`` and ``$not`` ~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2 (and 4.0.7), :query:`$not` operator can perform logical ``NOT`` operation on :query:`$regex` operator expressions as well as on regular expression objects (i.e. ``/pattern/``). In 4.0 and earlier versions, you could use :query:`$not` operator with regular expression objects (i.e. ``/pattern/``) but not with :query:`$regex` operator expressions. Kill Own Cursors ~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, users can always kill their own cursors, regardless of whether the users have the privilege to :authaction:`killCursors`. As such, the :authaction:`killCursors` privilege has no effect starting in MongoDB 4.2. In MongoDB 4.0, users required the :authaction:`killCursors` privilege in order to kill their own cursors. New Parameters ~~~~~~~~~~~~~~ MongoDB 4.2 adds the parameter :parameter:`replBatchLimitBytes` to configure the maximum oplog application batch size. The parameter is also available starting in MongoDB 4.0.10. Retryable Writes on Certain Single-Document Upserts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 will retry certain single-document upserts (update with ``upsert: true`` and ``multi: false``) that encounter a duplicate key exception. See :ref:`retryable-update-upsert` for conditions. Prior to MongoDB 4.2, MongoDB would not retry upsert operations that encountered a duplicate key error. ``db.dropDatabase()`` and Write Concern ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongODB 4.2, the :binary:`~bin.mongo` shell method :method:`db.dropDatabase()` can take an optional write concern document. ``dropConnections`` ~~~~~~~~~~~~~~~~~~~ The :dbcommand:`dropConnections` command drops the :binary:`~bin.mongod`/:binary:`~bin.mongos` instance's outgoing connections to the specified hosts. The :dbcommand:`dropConnections` must be run against the ``admin`` database. Client Disconnection ~~~~~~~~~~~~~~~~~~~~~ .. |operation| replace:: following operations For the following operations, if the issuing client disconnects before the operation completes, MongoDB marks the |operation| for termination (e.g. :dbcommand:`killOp` on the operation): .. list-table:: :header-rows: 1 * - Command - ``mongo`` Shell Method - Notes * - :dbcommand:`aggregate` - :method:`db.collection.aggregate()` - Behavior only applies if the pipeline does not include :pipeline:`$out` and :pipeline:`$merge` * - :dbcommand:`authenticate` - :method:`db.auth` - * - :dbcommand:`count` - | :method:`db.collection.count` | :method:`db.collection.countDocuments` | :method:`db.collection.estimatedDocumentCount` - * - :dbcommand:`distinct` - :method:`db.collection.distinct` - * - :dbcommand:`find` - | :method:`db.collection.find` | :method:`db.collection.findOne` - * - :dbcommand:`getnonce` - - * - :dbcommand:`isMaster` - - * - :dbcommand:`listCollections` - | :method:`db.getCollectionInfos` | :method:`db.getCollectionNames` - * - :dbcommand:`listDatabases` - - * - :dbcommand:`listIndexes` - :method:`db.collection.getIndexes` - Startup Warnings ~~~~~~~~~~~~~~~~ In-Memory Storage Engines ````````````````````````` .. include:: /includes/extracts/4.2-changes-inmem-startup-warning.rst ``mongo`` Shell ``````````````` .. include:: /includes/extracts/4.2-changes-mongoshell-emulation.rst Map-Reduce ~~~~~~~~~~ .. include:: /includes/extracts/4.2-changes-map-reduce-deprecation.rst Rollback Time Limit ~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the rollback time limit is calculated between the first operation after the common point and the last point in the oplog for the member to roll back. In MongoDB 4.0, the rollback time limit is calculated between the common point and the last point in the oplog for the member to roll back. For more information, see :ref:`rollback-time-limit`. ``isInteractive()`` ~~~~~~~~~~~~~~~~~~~~ MongoDB 4.2 adds a new :binary:`~bin.mongo` shell method :method:`isInteractive()` that returns a boolean indicating whether the :binary:`~bin.mongo` shell is running in interactive or script mode. Change to ``explain`` Output ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the ``explain`` output can include a new :data:`~explain.queryPlanner.optimizedPipeline` field. For details, refer to :data:`~explain.queryPlanner.optimizedPipeline`. Starting in version 4.2 (and 4.0.14, 3.6.16): - :doc:`Explain results ` for commands run on sharded clusters include a top-level :ref:`serverInfo ` object for the :binary:`~bin.mongos` in addition to the ``serverInfo`` objects returned for each shard. - :doc:`Explain results ` include the :ref:`serverInfo ` object when :data:`~explain.queryPlanner.optimizedPipeline` is ``true``. In previous versions of MongoDB, ``explain`` results would occasionally not include the ``serverInfo`` object when :data:`~explain.queryPlanner.optimizedPipeline` was ``true``. .. seealso:: :doc:`Explain results `. Change to ``isMaster`` Output ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Starting in MongoDB 4.2, the output for :dbcommand:`isMaster`, and the :method:`db.isMaster()` helper method, returns the :data:`~isMaster.connectionId` for the :binary:`~bin.mongod`/:binary:`~bin.mongos` instance's connection to the client. .. _4.2-index-builds: Optimized Index Builds ---------------------- MongoDB index builds against a populated collection require an exclusive read-write lock against the collection. Operations that require a read or write lock on the collection must wait until the :binary:`~bin.mongod` releases the lock. MongoDB uses an optimized build process that only holds the exclusive lock at the beginning and end of the index build. The rest of the build process yields to interleaving read and write operations. For :ref:`feature compatibility version (fcv) ` ``4.2``, MongoDB 4.2 index builds fully replace the index build processes supported in previous MongoDB versions. MongoDB ignores the ``background`` index build option if specified to :dbcommand:`createIndexes` or its shell helpers :method:`~db.collection.createIndex()` and :method:`~db.collection.createIndexes()`. .. admonition:: Requires featureCompatibilityVersion 4.2 :class: note For MongoDB clusters upgraded from 4.0 to 4.2, you must set the :ref:`feature compatibility version (fcv) ` to ``4.2`` to enable the optimized build process. For more information on setting the fCV, see :dbcommand:`setFeatureCompatibilityVersion`. MongoDB 4.2 clusters running with fCV ``4.0`` only support 4.0 index builds. For complete documentation on the index build process, see :ref:`index-operations`. Changes Affecting Compatibility ------------------------------- Some changes can affect compatibility and may require user actions. For a detailed list of compatibility changes, see :doc:`/release-notes/4.2-compatibility`. .. _4.2-upgrade: Upgrade Procedures ------------------ .. admonition:: Feature Compatibility Version To upgrade, the 4.0 instances must have ``featureCompatibilityVersion`` set to ``4.0``. To check the version: .. code-block:: javascript db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ) For specific details on verifying and setting the ``featureCompatibilityVersion`` as well as information on other prerequisites/considerations for upgrades, refer to the individual upgrade instructions: - :doc:`/release-notes/4.2-upgrade-standalone` - :doc:`/release-notes/4.2-upgrade-replica-set` - :doc:`/release-notes/4.2-upgrade-sharded-cluster` If you need guidance on upgrading to 4.2, `MongoDB professional services `_ offer major version upgrade support to help ensure a smooth transition without interruption to your MongoDB application. Download -------- To download MongoDB 4.2, go to the `MongoDB Download Center `_ .. seealso:: - `All Third Party License Notices `_. .. _4.2-known-issues: Known Issues ------------ .. list-table:: :header-rows: 1 :widths: 15 70 20 * - In Version - Issues - Status * - 4.2.0 - :issue:`SERVER-43075`: Missing :setting:`storage.journal.commitIntervalMs` - Fixed in 4.2.1 Report an Issue --------------- To report an issue, see https://github.com/mongodb/mongo/wiki/Submit-Bug-Reports for instructions on how to file a JIRA ticket for the MongoDB server or one of the related projects. .. toctree:: :hidden: /release-notes/4.2-compatibility /release-notes/4.2-upgrade-standalone /release-notes/4.2-upgrade-replica-set /release-notes/4.2-upgrade-sharded-cluster /release-notes/4.2-downgrade /release-notes/4.2-changelog