We looked at the very basics of starting and stopping MongoDB when we performed our Windows installation, but there are more details to be aware of when running MongoDB for real.
By default, MongoDB listens for client connections on port 27017, and stores data in the /data/db directory on Unix and on Windows \data\db on the drive from which you start MongoDB. To start MongoDB using all the defaults we simply run mongod from the command line.
To list the various options for use with the mongod process, we run
mongod --help
or
mongod -h
We can use the options listed to over-ride the defaults. Some common ones to use are:
--dbpath arg
Specifies an alternate location for the data directory
--bind_ip args
Specifies a comma-separated list of IP-addresses to listen for client connections
--port arg
Specifies an alternative port to listen for client connections
--logpath arg
Write all output from mongod to a file instead of the terminal. The directory path must exist but the file will be created by mongod.
--logappend
Keep previous log output instead of over-writing
--logRotate rename|reopen
Specifies the behavior to use when the logRotate command is issued. The default is rename, which archives the old log by appending a timestamp to it. If you want to take care of the archival yourself, for instance with the Linux/Unix logrotate utility, then use reopen and --logappend.
--syslog
Used on Unix systems to write the mongod output to the operating system log
--fork
Used on Unix systems to fork the mongod process and run it as a deamon. Must be run with a logpath
--journal
This will create a transaction log in a journal subdirectory in the data directory. All changes to MongoDB data will be recorded in the journal files here. Journaling is enabled by default on 64-bit installations. Journaling guarantees data integrity after a crash by replaying the writes from the journal files. The journal files are removed after a clean shutdown as they are no longer needed. Journaling is necessary for data persistency as changes are written in memory immediately but not to the datafiles on disk until the memory gets flushed.
--nojournal
Disables journaling on 64-bit systems
--journalCommitInterval arg
MongoDB commits changes in batches, writing to the journal every 100ms by default. This means you could lose 100ms of data in a crash. We can override this duration globally using journalCommitInterval as a startup option or a runtime configuration parameter. We can also use db.runCommand({"getLastError" : 1, "j" : true}) to wait 30ms for the previous write to be journaled.
--keyFile arg
Specifies a private key file for use in cluster authentication
--auth
Enforces secure logins
--noscripting
Disables the scripting engine on the server for improved security. Note that this can also break some shell helpers
--config arg
Specifes a configuration file containing additional startup options that were not specified on the command line. Format is described here
So we could start mongod to run as follows:
mongod --bind_ip 192.101.193.143 --port 27000 --dbpath "C:\mongodata\testdb" --logpath mongolog --journal
Or we could create config file (eg mongod.conf)containing
net:
bindIp: 192.101.193.143
port: 27000
storage:
dbPath: C:\mongodata\testdb
systemLog:
destination: file
path: C:\data\db\mongolog\mongod.log
logAppend: true
storage: journal:
enabled: true
and then run
mongod --config mongod.conf
to achieve the same thing
Thursday, 29 December 2016
Wednesday, 28 December 2016
MongoDB Basics V - Database Commands
There are many tasks we need to do with MongoDB that are not covered in the CRUD operations, and these are all performed by issuing Commands.
The available Commands can be shown by running db.listCommands() in the mongo shell.
Commands are run by using db.runCommand(). Some are "admin" commands that need to be run against the admin Database. There is a shell helper that we can use for this: db.adminCommand().
> db.adminCommand
function (obj, extra) {
if (this._name == "admin")
return this.runCommand(obj, extra);
return this.getSiblingDB("admin").runCommand(obj, extra);
}
As you can see, it simply runs the command against the admin Database unless it is already connected to admin.
Many of the other shell helpers work as functions that call db.runCommand(). For instance, to drop a Collection the helper runs the drop command. We can compare the behavior of the two methods on a successful and unsuccessful drop:
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.dropMe.drop()
true
> db.dropMe.drop()
false
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.runCommand({drop: "dropMe"})
{ "ns" : "next.dropMe", "nIndexesWas" : 1, "ok" : 1 }
> db.runCommand({drop: "dropMe"})
{
"ok" : 0,
"errmsg" : "ns not found",
"code" : 26,
"codeName" : "NamespaceNotFound"
}
The helper returns true or false whereas db.runCommand() returns a Document containing the key "ok". If "ok" is 0 then there will be an additional field "errmsg" that contains the error message returned by MongoDB. We can look at the code for the db.collection.drop helper itself to see an example of how this can be handled in an application:
> db.collection.drop
function () {
if (arguments.length > 0)
throw Error("drop takes no argument");
var ret = this._db.runCommand({drop: this.getName()});
if (!ret.ok) {
if (ret.errmsg == "ns not found")
return false;
throw _getErrorWithCode(ret, "drop failed: " + tojson(ret));
}
return true;
}
Internally, db.runCommand runs a query against a virtual collection called $cmd. We can do this ourselves, but it is recommended not to
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.$cmd.findOne({"drop" : "dropMe"});
{ "ns" : "next.dropMe", "nIndexesWas" : 1, "ok" : 1 }
An important command is getLastError as it used to get feedback on the previous statement that was run by the current session. The help for it under db.listCommands() shows:
getLastError: slaveOk
return error status of the last operation on this connection
options:
{ fsync:true } - fsync before returning, or wait for journal commit if running with --journal
{ j:true } - wait for journal commit if running with --journal
{ w:n } - await replication to n servers (including self) before returning
{ w:'majority' } - await replication to majority of set
{ wtimeout:m} - timeout for w in m milliseconds
Note that db.runCommand is order sensitive, we need to provide the command as the first field if there are additional options used, as we can do with getLastError. We will look into some command options later.
Here we will run a small update against multiple documents and use getLastError to return the status:
> db.testColl.drop()
true
> // first insert 10 documents
> for (var i = 0; i < 10; i++) { db.testColl.insert({field1 : "a", field2: i}) }
WriteResult({ "nInserted" : 1 })
> // the WriteResult shows just the last insert
> // we can prove it by running a count against the Collection
> db.runCommand({count:"testColl"})
{ "n" : 10, "ok" : 1 }
> // "n" : 10 shows 10 documents counted
> // now we update them all
> db.testColl.update({field1:"a"},{$set:{field2: 1000}},{multi:true})
WriteResult({ "nMatched" : 10, "nUpserted" : 0, "nModified" : 10 })
> db.runCommand({"getLastError" : 1})
{
"connectionId" : 7,
"updatedExisting" : true,
"n" : 10,
"syncMillis" : 0,
"writtenTo" : null,
"err" : null,
"ok" : 1
}
Our Documents are inserted by multiple insert statements in a loop, and the first writeResult shows only the result of the last statement from the loop as we did not add code to handle it better. But with runCommand we can count the Documents in the Collection and prove that there are 10 as expected.
Note that the "ok" : 1 in the Document returned with getLastError is for the db.runCommand() execution itself and does not mean that the previous statement completed ok. We can show that running an invalid statement:
> db.testColl.update({field1:"a"},{field2: 1},{multi:true})
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 9,
"errmsg" : "multi update only works with $ operators"
}
})
> db.runCommand({"getLastError" : 1})
{
"connectionId" : 7,
"err" : "multi update only works with $ operators",
"code" : 9,
"codeName" : "FailedToParse",
"n" : 0,
"ok" : 1
We will use db.runCommand a lot as we administer MongoDB and investigate the individual commands as we go.
The available Commands can be shown by running db.listCommands() in the mongo shell.
Commands are run by using db.runCommand(). Some are "admin" commands that need to be run against the admin Database. There is a shell helper that we can use for this: db.adminCommand().
> db.adminCommand
function (obj, extra) {
if (this._name == "admin")
return this.runCommand(obj, extra);
return this.getSiblingDB("admin").runCommand(obj, extra);
}
As you can see, it simply runs the command against the admin Database unless it is already connected to admin.
Many of the other shell helpers work as functions that call db.runCommand(). For instance, to drop a Collection the helper runs the drop command. We can compare the behavior of the two methods on a successful and unsuccessful drop:
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.dropMe.drop()
true
> db.dropMe.drop()
false
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.runCommand({drop: "dropMe"})
{ "ns" : "next.dropMe", "nIndexesWas" : 1, "ok" : 1 }
> db.runCommand({drop: "dropMe"})
{
"ok" : 0,
"errmsg" : "ns not found",
"code" : 26,
"codeName" : "NamespaceNotFound"
}
The helper returns true or false whereas db.runCommand() returns a Document containing the key "ok". If "ok" is 0 then there will be an additional field "errmsg" that contains the error message returned by MongoDB. We can look at the code for the db.collection.drop helper itself to see an example of how this can be handled in an application:
> db.collection.drop
function () {
if (arguments.length > 0)
throw Error("drop takes no argument");
var ret = this._db.runCommand({drop: this.getName()});
if (!ret.ok) {
if (ret.errmsg == "ns not found")
return false;
throw _getErrorWithCode(ret, "drop failed: " + tojson(ret));
}
return true;
}
Internally, db.runCommand runs a query against a virtual collection called $cmd. We can do this ourselves, but it is recommended not to
> db.dropMe.insert({a:1})
WriteResult({ "nInserted" : 1 })
> db.$cmd.findOne({"drop" : "dropMe"});
{ "ns" : "next.dropMe", "nIndexesWas" : 1, "ok" : 1 }
An important command is getLastError as it used to get feedback on the previous statement that was run by the current session. The help for it under db.listCommands() shows:
getLastError: slaveOk
return error status of the last operation on this connection
options:
{ fsync:true } - fsync before returning, or wait for journal commit if running with --journal
{ j:true } - wait for journal commit if running with --journal
{ w:n } - await replication to n servers (including self) before returning
{ w:'majority' } - await replication to majority of set
{ wtimeout:m} - timeout for w in m milliseconds
Note that db.runCommand is order sensitive, we need to provide the command as the first field if there are additional options used, as we can do with getLastError. We will look into some command options later.
Here we will run a small update against multiple documents and use getLastError to return the status:
> db.testColl.drop()
true
> // first insert 10 documents
> for (var i = 0; i < 10; i++) { db.testColl.insert({field1 : "a", field2: i}) }
WriteResult({ "nInserted" : 1 })
> // the WriteResult shows just the last insert
> // we can prove it by running a count against the Collection
> db.runCommand({count:"testColl"})
{ "n" : 10, "ok" : 1 }
> // "n" : 10 shows 10 documents counted
> // now we update them all
> db.testColl.update({field1:"a"},{$set:{field2: 1000}},{multi:true})
WriteResult({ "nMatched" : 10, "nUpserted" : 0, "nModified" : 10 })
> db.runCommand({"getLastError" : 1})
{
"connectionId" : 7,
"updatedExisting" : true,
"n" : 10,
"syncMillis" : 0,
"writtenTo" : null,
"err" : null,
"ok" : 1
}
Our Documents are inserted by multiple insert statements in a loop, and the first writeResult shows only the result of the last statement from the loop as we did not add code to handle it better. But with runCommand we can count the Documents in the Collection and prove that there are 10 as expected.
Note that the "ok" : 1 in the Document returned with getLastError is for the db.runCommand() execution itself and does not mean that the previous statement completed ok. We can show that running an invalid statement:
> db.testColl.update({field1:"a"},{field2: 1},{multi:true})
WriteResult({
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 9,
"errmsg" : "multi update only works with $ operators"
}
})
> db.runCommand({"getLastError" : 1})
{
"connectionId" : 7,
"err" : "multi update only works with $ operators",
"code" : 9,
"codeName" : "FailedToParse",
"n" : 0,
"ok" : 1
We will use db.runCommand a lot as we administer MongoDB and investigate the individual commands as we go.
Friday, 23 December 2016
MongoDB Basics IV - CRUD Operations
The so-called CRUD operations are what we use to Create, Read, Update, and Delete documents.
In MongoDB, write operations (Create, Update, Delete) target a single Collection.
All write operations in MongoDB are atomic on the level of a single Document, regardless of how many Documents they change. This means if the operation fails at some point, the writes it has already perfomed persist in the database.
Write operations generally accept a writeConcern option that controls the level of acknowledgement required from MongoDB before the operation is considered to be complete. We will look at writeConcern here.
In MongoDB, write operations (Create, Update, Delete) target a single Collection.
All write operations in MongoDB are atomic on the level of a single Document, regardless of how many Documents they change. This means if the operation fails at some point, the writes it has already perfomed persist in the database.
Write operations generally accept a writeConcern option that controls the level of acknowledgement required from MongoDB before the operation is considered to be complete. We will look at writeConcern here.
Create Operations
Create operations insert new Documents to a Collection.
If the Collection does not currently exist, inserting a Document will implicitly create the Collection.
MongoDB 3.4 provides the following methods to insert Documents into a Collection:
- db.collection.insert(document or array of documents,writeConcern, ordered)
- db.collection.insertOne(document, writeConcern)
- db.collection.insertMany(array of documents,writeConcern, ordered)
- some types of updates, which we will cover under Update Operations
If we use {ordered: false} then Documents are inserted in an unordered format and may be reordered by MongoDB for performance benefits. MongoDB will continue to insert remaining Documents from the array even if an error is encountered. The default, {ordered: true} will insert the Documents in the order given and if an error is encountered it will return an error and cease to insert the remaining Documents.
To see these in action, with a mongod process running, run the mongo executable to start the mongo shell. We can then insert Documents into MongoDB
Here we first see how to display the Databases and Collections in MongoDB (the commands: show dbs, db and show collections). Note that by default the MongoDB shell connects to the test Database, which currently does not exist and has no Collections.
After we insert our first Document:
db.play.insert({field1 : 1})
the test Database and the play Collection are created. Note also the _id field is added by MongoDB.
db.collection.insert() returns a WriteResult object:
WriteResult({ "nInserted" : 1 })
telling us how many documents were inserted. It will only ever be one for an insert.
We can use db.collection.insert() to insert multiple Documents, by encasing them in square brackets to form an Array. (See this post for discussion of data types)
Just for test purposes, I first dropped the test Database, using the db.dropDatabase() command.
Two documents are inserted with:
db.play.insert([{field1 : 2},{field1 : 3, field2 : 1}])
MongoDB responds with a BulkWriteResult object, which has many more fields in addition to "nInserted". They relate to other types of write operation.
The other two insert methods db.collection.insertOne and db.collection.insertMany behave similarly to the two ways of invoking db.collection.insert(), but they return a Document providing an acknowledged field and the ObjectId of the inserted Documents. The db.collection.insertOne() method will also return an error if you pass it an array of Documents.
The db.collection.save() function will perform an insert if the Document is supplied with no _id field, otherwise it will call db.collection.update() (see below).
Read Operations¶
We covered read operations in this post
Update Operations¶
Update operations modify existing Documents in a Collection. MongoDB provides the following methods to do this:
- db.collection.update(query document,update document,options)
- db.collection.updateOne(query document,update document,options)
- db.collection.updateMany(query document,update document,options)
- db.collection.replaceOne(query document,update document,options)
- db.collection.findAndModify(query document,update document,sort,options)
- db.collection.findOneAndReplace(query document,update document,options)
- db.collection.findOneAndUpdate(query document,update document,options)
- db.collection.save(document,writeConcern)
You can specify query criteria, or filters, that identify the documents to update. These filters use the same syntax as read operations.
There are also various options that may be applied:
Option | Type | Description |
---|---|---|
upsert | boolean | true: will create a new Document if no existing Documents match the filter false: will only update existing Documents [default] |
multi | boolean | true: will update any Document that matches the filter false: will update only one Document [default] |
writeConcern | document | control the level of acknowledgement of the write |
collation | document | language specific rules for the operation |
And various operators can be applied in the update document:
Name | Description |
---|---|
$inc | increments the value of the field by the given value |
$mul | multiplies the value of the field by the given value |
$rename | renames the field |
$setOnInsert | sets the field to the given value only if a new Document is inserted |
$set | sets the field to the given value |
$unset | removes the specified field |
$min | updates the field only if the given value is less than the current field value |
$max | updates the field only if the given value is greater than the current field value |
$currentDate | sets the field's value to current Date or Timestamp |
$ | placeholder to refer to the first element in an array that matches the query |
$addToSet | adds elements to an array if they do not exist |
$pop | removes the first or last element in an array |
$pullAll | removes all matching values from an array |
$pull | removes all elements from an array that match the query |
$pushAll | adds multiple elements to an array. Deprecated |
$push | adds an element to an array |
$each | modifies $push and $addToSet to append multiple elements to an array |
$slice | modifies $push to limit the size of the updated array |
$sort | modifies $push to reorder documents stored in an array |
$position | modifies $push to specify the position in the array to add elements |
$bit | performs bitwise AND, OR and XOR integer updates |
$isolated | When True prevents other clients from reading or writing changed Documents before the update operation is fully completed. Default False allows concurrent access to changed Documents |
Let's run the following to experiment with some basic features of update()
// examples for db.collection.update(query document,update document,options)
db.test.drop()
// 1. no documents match the query, so no update
db.test.update({field1: 'a'},{field1:'a', field2: 1})
db.test.find()
// 2. no documents match the query, so new document is inserted with upsert
db.test.update({field1: 'a'},{field1:'a', field2: 2},{upsert:1})
db.test.find()
// 3. one document matches the query, so it is updated with upsert
db.test.update({field1: 'a'},{field1:'a', field2: 3},{upsert:1})
db.test.find()
// 4. insert a duplicate document
db.test.insert({field1:'a', field2: 4})
db.test.find()
// 5. two documents match the query, but only first is updated
db.test.update({field1: 'a'},{field1:'a', field2: 5})
// 6. try to use multi, but it makes the simple update document format invalid
db.test.update({field1: 'a'},{field1:'a', field2: 6},{multi:true})
// 7. any matching documents are updated with multi if we use update operators
db.test.update({field1: 'a'},{$set:{field1:'a', field2: 7}},{multi:true})
db.test.find()
This gives the following results when run through the shell:
The db.collection.updateOne() function is similar to db.collection.update(,,{multi:true}) while db.collection.updateMany() functions similarly to db.collection.update(,,{multi:false})The db.collection.replaceOne() functions similarly to insert() and will effectively delete the first matching Document and replace it with the update Document. In other words, fields not specified in the update document will be removed. The insert() function also does this unless it is called using only $ update operators. Let's check that with the following test script:
// examples for db.collection.replaceOne(query document,update document,options)
db.test.drop()
// 1. no documents match the query, so new document is inserted with upsert
db.test.replaceOne({field1: 'a'},{field1:'a', field2: 1, field3:1},{upsert:1})
db.test.find()
// 2. one document matches the query, so the document is replaced
db.test.replaceOne({field1: 'a'},{field1:'a', field3: 2})
db.test.find()
// 3. normal update works the same way and replaces the matching document
db.test.update({field1: 'a'},{field1:'a', field4: 3, field5: 3})
db.test.find()
// 4. update with update operators updates only specified fields
db.test.update({field1: 'a'},{$set:{field1:'4'}, $inc:{field4:1 }})
db.test.find()
The db.collection.save() function will call db.collection.update(,,{upsert:true}) if the Document is supplied with an _id, otherwise it will call db.collection.insert().
> db.collection.save
function (obj, opts) {
if (obj == null)
throw Error("can't save a null");
if (typeof(obj) == "number" || typeof(obj) == "string")
throw Error("can't save a number or string");
if (typeof(obj._id) == "undefined") {
obj._id = new ObjectId();
return this.insert(obj, opts);
} else {
return this.update({_id: obj._id}, obj, Object.merge({upsert: true}, opts));
}
The db.collection.find...() functions return the before-version of the modified Document(s) in addition to performing an update
Delete Operations¶
Delete operations remove documents from a collection. MongoDB provides the following methods to do this
- db.collection.remove(query, justOne, writeConcern, collation)
- db.collection.deleteOne(query, writeConcern, collation)
- db.collection.deleteMany(query, writeConcern, collation)
- db.collection.findOneAndDelete(query document,options)
You can specify query criteria, or filters, that identify the documents to remove. These filters use the same syntax as read operations.
Bulk Write¶
MongoDB provides the ability to perform multiple write operations in bulk.
db.collection.bulkWrite(
[ <operation 1>, <operation 2>, ... ],
{
writeConcern : <document>,
ordered : <boolean>
}
)
The valid operations can be:
db.collection.bulkWrite(
[ <operation 1>, <operation 2>, ... ],
{
writeConcern : <document>,
ordered : <boolean>
}
)
The valid operations can be:
- db.collection.insertOne()
- db.collection.updateOne()
- db.collection.updateMany()
- db.collection.deleteOne()
- db.collection.replaceOne()
The ordered parameter controls whether the operations are executed in the order written or not. When false the results are unpredictable as the order of execution can change.
Thursday, 15 December 2016
MongoDB Shell
The MongoDB shell (mongo) is a fully-functioning JavaScript interpreter that is included with all MongoDB distributions.
Running the shell
To start the MongoDB shell we run the mongo executable from the command line:
mongo
We can run mongo with various options. To see what these are, first run it with the help option:
C:\Users>mongo --help
MongoDB shell version v3.4.0
usage: mongo [options] [db address] [file names (ending in .js)]
db address can be:
foo foo database on local machine
192.168.0.5/foo foo database on 192.168.0.5 machine
192.168.0.5:9999/foo foo database on 192.168.0.5 machine on port 9999
Options:
--shell run the shell after executing files
--nodb don't connect to mongod on startup - no
'db address' arg expected
--norc will not run the ".mongorc.js" file on
start up
--quiet be less chatty
--port arg port to connect to
--host arg server to connect to
--eval arg evaluate javascript
-h [ --help ] show this usage information
--version show version information
--verbose increase verbosity
--ipv6 enable IPv6 support (disabled by default)
--disableJavaScriptJIT disable the Javascript Just In Time
compiler
--disableJavaScriptProtection allow automatic JavaScript function
marshalling
--ssl use SSL for all connections
--sslCAFile arg Certificate Authority file for SSL
--sslPEMKeyFile arg PEM certificate/key file for SSL
--sslPEMKeyPassword arg password for key in PEM file for SSL
--sslCRLFile arg Certificate Revocation List file for SSL
--sslAllowInvalidHostnames allow connections to servers with
non-matching hostnames
--sslAllowInvalidCertificates allow connections to servers with invalid
certificates
--sslFIPSMode activate FIPS 140-2 mode at startup
--networkMessageCompressors arg Comma-separated list of compressors to
use for network messages
--jsHeapLimitMB arg set the js scope's heap size limit
Authentication Options:
-u [ --username ] arg username for authentication
-p [ --password ] arg password for authentication
--authenticationDatabase arg user source (defaults to dbname)
--authenticationMechanism arg authentication mechanism
--gssapiServiceName arg (=mongodb) Service name to use when authenticating
using GSSAPI/Kerberos
--gssapiHostName arg Remote host name to use for purpose of
GSSAPI/Kerberos authentication
file names: a list of files to run. files have to end in .js and will exit after unless --shell is specified
By default, mongo will try to connect to the test Database on localhost port 27107 or 127.0.0.1:27017/test
We can override these individually with the --host and --port options shown above, but these cannot be applied together and there is no Database option. Alternatively we can supply a [db address] string:
mongo 192.169.0.5:27108/admin
We can also start the shell without any Database connection using the --nodb option:
mongo --nodb
We can't do much with MongoDB in this mode, but we could run JavaScripts, eg:
C:\Users>mongo --nodb --quiet
> var result = "";
> var i = 0;
> do {i += 1;result += i + " ";} while (i < 5)
1 2 3 4 5
> print ("result: " + result)
result: 1 2 3 4 5
Here we start it with the defaults, as our mongod process is using them also:
As well as the --help option, there is an internal help command as shown above, that gives some info about specific MongoDB functions or helpers. This shows further help helpers, for instance Database-level help is provided by db.help() and Collection-level help by db.collection.help(). There are also some show helpers to help us navigate through our MongoDB.
Here we check the Databases in our MongoDB (show dbs), switch from test Database to admin Database (use admin), show the available Collections in the admin Database (show collections)and query the system.version Collection to show us our current version of MongoDB (db.system.find()).
Note that to run a function, such as db.collection.find , we need to supply parentheses to it:
db.system.version.find()
If we don't supply the parentheses the shell returns the implementation of the function itself:
This gives us some nice information about the inputs and outputs of the function.
Some functions will also have there own help helper, for instance
db.collection.find().help()
You will come to use a lot of these variations of find as you spend more time with the MongoDB shell.
Configuring the shell
We can format the shell prompt from the default ">" to something more informative, for instance to show the current Database and time:
prompt = function() { return db+"@" + new Date().toLocaleTimeString()+"> " ; };
We could run this directly in the shell to configure that particular session. However, there is a file called .mongorc.js in the users home directory (Unix: ~, Windows: %HOMEPATH%) that is sourced when mongo is started. This file is the place to store such customizations in order to make them permanent.
Here is an example that sets the default editor, the prompt and disables some functions that could damage our Database:
C:\Users>cat %HOMEPATH%\.mongorc.js
EDITOR = "notepad.exe";
prompt = function() { return db+"@" + new Date().toLocaleTimeString()+"> "; };
// Disable dangerous actions (not recommended)
var disabled = function() {
print("That command is disabled");
};
db.dropDatabase = DB.prototype.dropDatabase = disabled;
DBCollection.prototype.drop = disabled;
DBCollection.prototype.dropIndex = disabled;
Now if we start our shell and attempt to drop the admin Database, we should get this:
C:\Users>mongo --quiet
test@18:55:15> show dbs
admin 0.000GB
local 0.000GB
test 0.000GB
test@18:55:19> use admin
switched to db admin
admin@18:55:27> db.dropDatabase()
That command is disabled
admin@18:55:30> show dbs
admin 0.000GB
local 0.000GB
test 0.000GB
I don't recommend messing with functions in this way, and the behavior of this script seems to change between different versions and systems. For instance, on some versions the shell would not start --nodb as the db in db.dropDatabase is not defined. On others it was ok.
Starting mongo with the --norc option will disable mongorc.js for that session.
The official MongoDB shell documentation can be found here
Running the shell
To start the MongoDB shell we run the mongo executable from the command line:
mongo
We can run mongo with various options. To see what these are, first run it with the help option:
C:\Users>mongo --help
MongoDB shell version v3.4.0
usage: mongo [options] [db address] [file names (ending in .js)]
db address can be:
foo foo database on local machine
192.168.0.5/foo foo database on 192.168.0.5 machine
192.168.0.5:9999/foo foo database on 192.168.0.5 machine on port 9999
Options:
--shell run the shell after executing files
--nodb don't connect to mongod on startup - no
'db address' arg expected
--norc will not run the ".mongorc.js" file on
start up
--quiet be less chatty
--port arg port to connect to
--host arg server to connect to
--eval arg evaluate javascript
-h [ --help ] show this usage information
--version show version information
--verbose increase verbosity
--ipv6 enable IPv6 support (disabled by default)
--disableJavaScriptJIT disable the Javascript Just In Time
compiler
--disableJavaScriptProtection allow automatic JavaScript function
marshalling
--ssl use SSL for all connections
--sslCAFile arg Certificate Authority file for SSL
--sslPEMKeyFile arg PEM certificate/key file for SSL
--sslPEMKeyPassword arg password for key in PEM file for SSL
--sslCRLFile arg Certificate Revocation List file for SSL
--sslAllowInvalidHostnames allow connections to servers with
non-matching hostnames
--sslAllowInvalidCertificates allow connections to servers with invalid
certificates
--sslFIPSMode activate FIPS 140-2 mode at startup
--networkMessageCompressors arg Comma-separated list of compressors to
use for network messages
--jsHeapLimitMB arg set the js scope's heap size limit
Authentication Options:
-u [ --username ] arg username for authentication
-p [ --password ] arg password for authentication
--authenticationDatabase arg user source (defaults to dbname)
--authenticationMechanism arg authentication mechanism
--gssapiServiceName arg (=mongodb) Service name to use when authenticating
using GSSAPI/Kerberos
--gssapiHostName arg Remote host name to use for purpose of
GSSAPI/Kerberos authentication
file names: a list of files to run. files have to end in .js and will exit after unless --shell is specified
By default, mongo will try to connect to the test Database on localhost port 27107 or 127.0.0.1:27017/test
We can override these individually with the --host and --port options shown above, but these cannot be applied together and there is no Database option. Alternatively we can supply a [db address] string:
mongo 192.169.0.5:27108/admin
When we start the shell, it will give us some version and connection info and maybe some warnings. We can disable this with the --quiet option
We can also start the shell without any Database connection using the --nodb option:
mongo --nodb
We can't do much with MongoDB in this mode, but we could run JavaScripts, eg:
C:\Users>mongo --nodb --quiet
> var result = "";
> var i = 0;
> do {i += 1;result += i + " ";} while (i < 5)
1 2 3 4 5
> print ("result: " + result)
result: 1 2 3 4 5
Here we start it with the defaults, as our mongod process is using them also:
As well as the --help option, there is an internal help command as shown above, that gives some info about specific MongoDB functions or helpers. This shows further help helpers, for instance Database-level help is provided by db.help() and Collection-level help by db.collection.help(). There are also some show helpers to help us navigate through our MongoDB.
Here we check the Databases in our MongoDB (show dbs), switch from test Database to admin Database (use admin), show the available Collections in the admin Database (show collections)and query the system.version Collection to show us our current version of MongoDB (db.system.find()).
Note that to run a function, such as db.collection.find , we need to supply parentheses to it:
db.system.version.find()
If we don't supply the parentheses the shell returns the implementation of the function itself:
This gives us some nice information about the inputs and outputs of the function.
Some functions will also have there own help helper, for instance
db.collection.find().help()
You will come to use a lot of these variations of find as you spend more time with the MongoDB shell.
Configuring the shell
We can format the shell prompt from the default ">" to something more informative, for instance to show the current Database and time:
prompt = function() { return db+"@" + new Date().toLocaleTimeString()+"> " ; };
We could run this directly in the shell to configure that particular session. However, there is a file called .mongorc.js in the users home directory (Unix: ~, Windows: %HOMEPATH%) that is sourced when mongo is started. This file is the place to store such customizations in order to make them permanent.
Here is an example that sets the default editor, the prompt and disables some functions that could damage our Database:
C:\Users>cat %HOMEPATH%\.mongorc.js
EDITOR = "notepad.exe";
prompt = function() { return db+"@" + new Date().toLocaleTimeString()+"> "; };
// Disable dangerous actions (not recommended)
var disabled = function() {
print("That command is disabled");
};
db.dropDatabase = DB.prototype.dropDatabase = disabled;
DBCollection.prototype.drop = disabled;
DBCollection.prototype.dropIndex = disabled;
Now if we start our shell and attempt to drop the admin Database, we should get this:
C:\Users>mongo --quiet
test@18:55:15> show dbs
admin 0.000GB
local 0.000GB
test 0.000GB
test@18:55:19> use admin
switched to db admin
admin@18:55:27> db.dropDatabase()
That command is disabled
admin@18:55:30> show dbs
admin 0.000GB
local 0.000GB
test 0.000GB
I don't recommend messing with functions in this way, and the behavior of this script seems to change between different versions and systems. For instance, on some versions the shell would not start --nodb as the db in db.dropDatabase is not defined. On others it was ok.
Starting mongo with the --norc option will disable mongorc.js for that session.
The official MongoDB shell documentation can be found here
Sunday, 4 December 2016
Installing MongoDB on Windows 10
Here we will run through an installation of MongoDB 3.4.0 on Windows 10
Step 1: Download
Click the "Download" button on the top-right of the banner on any mongodb.com page to go to their download centre and download your preferred flavour of MongoDB
Click the Download button and save the installer file. Run it when the download completes.
Step 2: Install
Run the installer and click "Next" to confirm you want to go through the Setup:
Accept the License Agreement and click "Next":
You can choose Complete or Custom Setup:
Choose Complete Installation unless you really need to change something.
Custom Setup gives you a bit more control of the components to install:
It's not really worth bothering with and you can always come back if you find need to change some component.
The executable installed with each component are:
Component | Executable |
Server | mongod.exe |
Router | mongos.exe |
Client | mongo.exe |
MonitoringTools | mongostat.exe, mongotop.exe |
ImportExportTools | mongodump.exe, mongorestore.exe, mongoexport.exe, mongoimport.exe |
MiscellaneousTools | bsondump.exe, mongofiles.exe, mongooplog.exe, mongoperf.exe |
Click "Next" on your choice of Complete or Custom Installation and the installation is ready to begin:
Click "Install". You may get a Windows security pop-up asking if you want to allow the installer to make these changes, if so, simply click "Yes".
The Installation will begin in a few seconds:
When done, you should see the successful Completion message:
And that's all there is to it.
The files should be installed to C:\Program Files\MongoDB\
Step 2.2 (optional): Change, Repair or Remove Installation
Running the Installer again will allow you make changes to your installation:
"Change" takes you back to the Custom Installation screen where you can add or remove selected components. "Remove" will Uninstall MongoDB from your PC.
Step 3: Configure the environment for MongoDB
Before we can run MongoDB we need to do a bit of Windows configuration.
Firstly, start a Windows Command prompt. Find the Command Prompt icon on the Start Menu or hit R+Windows key and type "cmd" to have Windows find it. Right-click and choose Run As Administrator. When it's running it looks like this:
MongoDB needs a data directory to store files. By default this is \data\db under the current volume, so we could create this in our Command Prompt window with:
mkdir c:\data\db
We could create our data directory in a different location, but then would always need to tell MongoDB this location at runtime. Keeping to the default is easier for now.
We can add a default log directory too:
mkdir c:\data\log
To make running MongoDB executables easier, it helps to add their location to your path. Launch System Settings from the Windows Control Panel, and select Environment Variables on the Advanced tab:
Add a New entry "C:\Program Files\MongoDB\Server\3.4\bin" to your path:
Now Windows will look for executables in this directory.
Step 4: Starting MongoDB
Now we can start our MongoDB server from the Windows Command Prompt. This is done by running the mongod executable.
It can be started with various options. To see what they are we can execute with the help option:
mongod --help
This will just print the help and exit.
It can be started with various options. To see what they are we can execute with the help option:
mongod --help
This will just print the help and exit.
If the MongoDB directory has been added to the Windows path as described above then we can simply type the name of the executable we want to run:
mongod
otherwise we need to specify the full path ourselves:
C:\"Program Files"\MongoDB\Server\3.4\bin\mongod
Either way, this should start the mongod process running and writing log messages to your Command window.
Your firewall program may now detect the MongoDB network processes and ask you what to do:
I chose to disable all access here. You can always change the config later in the Firewall controls.
When you see the message:
NETWORK [thread1] waiting for connections on port 27017
then your MongoDB is up and running:
Hit CTRL-C to stop it.
This basic installation will be good enough for playing around and learning with MongoDB.
More advanced installation topics can be found here:
To connect to our MongoDB we will use the MongoDB shell, which we will look at here
Thursday, 1 December 2016
MongoDB Basics III - Querying MongoDB
MongoDB provides various methods for querying data. Note that they are case-sensitive and need to be called exactly as written below
Query Method | Description |
---|---|
db.collection.find() | Performs a query on a collection and returns a cursor object. |
db.collection.findAndModify() | Atomically modifies and returns a single document. |
db.collection.findOne() | Performs a query and returns a single document ... the first one found by natural search order. |
db.collection.findOneAndDelete() | Finds a single document and deletes it. |
db.collection.findOneAndReplace() | Finds a single document and replaces it. |
db.collection.findOneAndUpdate() | Finds a single document and updates it. |
db.collection.find( <filter>, <projection> )
This is the basic method of querying data.
Filter and Projection are optional documents that modify the query.
Filter uses key:value pairs to restrict the Documents returned and Projection uses flagged fields to restrict the Fields returned (aka projected)
db.clients.find({ name: { last: "Musterman", first: "Max" }})
This will return all the Documents from the clients Collection with first name Max and last name Musterman
db.orders.find({ value $gt 1000 }, { company: 1, _id: 0})
This will return just the company Field for all orders valued over 1000 in the orders Collection. 1 switches a column on and 0 switches it off. The _id field must be specifically excluded
Various operators can be used. ':' means equality, but we can also use $eq
Filter and Projection are optional documents that modify the query.
Filter uses key:value pairs to restrict the Documents returned and Projection uses flagged fields to restrict the Fields returned (aka projected)
db.clients.find({ name: { last: "Musterman", first: "Max" }})
This will return all the Documents from the clients Collection with first name Max and last name Musterman
db.orders.find({ value $gt 1000 }, { company: 1, _id: 0})
This will return just the company Field for all orders valued over 1000 in the orders Collection. 1 switches a column on and 0 switches it off. The _id field must be specifically excluded
Various operators can be used. ':' means equality, but we can also use $eq
Name | Description |
---|---|
$eq | Matches values that are equal to a specified value. |
$gt | Matches values that are greater than a specified value. |
$gte | Matches values that are greater than or equal to a specified value. |
$lt | Matches values that are less than a specified value. |
$lte | Matches values that are less than or equal to a specified value. |
$ne | Matches all values that are not equal to a specified value. |
$in | Matches any of the values specified in an array. |
$nin | Matches none of the values specified in an array. |
$or | Joins query clauses with a logical OR returns all documents that match the conditions of either clause. |
$and | Joins query clauses with a logical AND returns all documents that match the conditions of both clauses. |
$not | Inverts the effect of a query expression and returns documents that do not match the query expression. |
$nor | Joins query clauses with a logical NOR returns all documents that fail to match both clauses. |
$exists | Matches documents that have the specified field. |
$type | Selects documents if a field is of the specified type. |
$mod | Performs a modulo operation on the value of a field and selects documents with a specified result. |
$regex | Selects documents where values match a specified regular expression. |
$text | Performs text search. |
$where | Matches documents that satisfy a JavaScript expression. |
$geoWithin | Selects geometries within a bounding GeoJSON geometry. The 2dsphere and 2d indexes support $geoWithin. |
$geoIntersects | Selects geometries that intersect with a GeoJSON geometry. The 2dsphere index supports $geoIntersects. |
$near | Returns geospatial objects in proximity to a point. Requires a geospatial index. The 2dsphere and 2d indexes support $near. |
$nearSphere | Returns geospatial objects in proximity to a point on a sphere. Requires a geospatial index. The 2dsphere and 2d indexes support $nearSphere. |
$all | Matches arrays that contain all elements specified in the query. |
$elemMatch | Selects documents if element in the array field matches all the specified $elemMatch conditions. |
$size | Selects documents if the array field is a specified size. |
$bitsAllSet | Matches numeric or binary values in which a set of bit positions all have a value of 1. |
$bitsAnySet | Matches numeric or binary values in which any bit from a set of bit positions has a value of 1. |
$bitsAllClear | Matches numeric or binary values in which a set of bit positions all have a value of 0. |
$bitsAnyClear | Matches numeric or binary values in which any bit from a set of bit positions has a value of 0. |
$comment | Adds a comment to a query predicate. |
$ | Projects the first element in an array that matches the query condition. |
$elemMatch | Projects the first element in an array that matches the specified $elemMatch condition. |
$meta | Projects the document’s score assigned during $text operation. |
$slice | Limits the number of elements projected from an array. Supports skip and limit slices. |
Output of find is a cursor, which can be modified with various additional methods.
For instance,
db.collection.find().pretty
will return the cursor results in an easy-to-read format
Subscribe to:
Posts (Atom)