Writing an application on top of a framework on top of a driver on top of the database is a bit like playing telephone: you say “insert foo” and the database says “purple monkey dishwasher.” mongosniff lets you see exactly what the database is hearing and saying.
It comes with the binary distribution, so if you have mongod you should have mongosniff.
To try it out, first start up an instance of mongod normally:
$ ./mongod
When you start up mongosniff you have to tell it to listen on the loopback (localhost) interface. This interface is usually called “lo”, but my Mac calls it “lo0”, so run ifconfig to make sure you have the name right. Now run:
$ sudo ./mongosniff --source NET lo sniffing... 27017
Note the “sudo”: this has never worked for me from my user account, probably because of some stupid network permissions thing.
Now start up the mongo shell and try inserting something:
> db.foo.insert({x:1})
If you look at mongosniff‘s output, you’ll see:
127.0.0.1:57856 -->> 127.0.0.1:27017 test.foo 62 bytes id:430131ca 1124151754
insert: { _id: ObjectId('4c7fb007b5d697849addc650'), x: 1.0 }
127.0.0.1:57856 -->> 127.0.0.1:27017 test.$cmd 76 bytes id:430131cb 1124151755
query: { getlasterror: 1.0, w: 1.0 } ntoreturn: -1 ntoskip: 0
127.0.0.1:27017 <<-- 127.0.0.1:57856 76 bytes id:474447bf 1195657151 - 1124151755
reply n:1 cursorId: 0
{ err: null, n: 0, wtime: 0, ok: 1.0 }
There are three requests here, all for one measly insert. Dissecting the first request, we can learn:
source -->> destination- Our client, mongo in this case, is running on port 57856 and has sent a message to the database (127.0.0.1:27017).
db.collection- This request is for the “test” database’s “foo” collection.
length bytes- The length of the request is 62 bytes. This can be handy to know if your requests are edging towards the maximum request length (16 MB).
id:hex-id id- This is the request id in hexadecimal and decimal (in case you don’t have a computer handy, apparently). Every request to and from the database has a unique id associated with it for tax purposes.
op: content- This is the actual meat of the request: we’re inserting this document. Notice that it’s inserting the float value 1.0, even though we typed 1 in the shell. This is because JavaScript only has one number type, so every number typed in the shell is converted to a double.
The next request in the mongosniff output is a database command: it checks to make sure the insert succeeded (the shell always does safe inserts).
The last message sniffed is a little different: it is going from the database to the shell. It is the database response to the getlasterror command. It shows that there was only one document returned (reply n:1) and that there are no more results waiting at the database (cursorId: 0). If this were a “real” query and there was another batch of results to be sent from the database, cursorId would be non-zero.
Hopefully this will help some people decipher what the hell is going on!
Kristina Chodorow's Blog 
::::The next request in the mongosniff output is a database command: it checks to make sure the insert succeeded (the shell always does safe inserts).::::
Thats good to know, and contrary to my assumption in this question: http://stackoverflow.com/questions/3605427/safe-write-from-the-mongodb-shell
If you can clarify my SO question, I’ll try to update the wiki this weekend.
LikeLike
Yeah, that’s why the shell often seems slower to people than the drivers. Thilio is correct, since the getlasterror called automatically doesn’t fsync or have a “real” w value (w=1 is the same as vanilla safe mode). To flush to disk, you’ll need to call getlasterror again with fsync.
LikeLike
Sweet! Didn’t know about mongosniff until now. Thanks!
LikeLike
Love the Simpsons reference. 🙂
oh, and nice article too.
LikeLike
Yeah, there are a lot of little features and tools that have been implemented for some user long ago that no one else knows about. Glad to help!
LikeLike
Thanks!
LikeLike
Does mongosniff not come with the MongoDB supported Ubuntu package? Only see the binaries below.
It would be nice if it did?
mongo
mongod
mongodump
mongoexport
mongofiles
mongoimport
mongorestore
mongos
mongostat
LikeLike
Interesting, probably just an oversight because it’s new-ish. If you could file a request at http://jira.mongodb.org/browse/SERVER, I’m sure Richard would put it in.
LikeLike
Pardon me for jumping in at such a low level before I have even
learned anything about the guts of MongoDB, but I’l risk it anyway.
It seems that there is an opportunity for an optimization here. If
the message-sending level “knows” that after the insert operation, its
caller is going to send a “getlasterror”, the two could be combined
into a single message.
It would not be necessary to create a new message operation called
“insert and then getlasterror”. You can just append the two messages.
The overhead that you want to get rid of is not the overhead of
parsing the header of each message, but the network overhead. By
batching up the messages, not only can the messages both fit into the
same packet, but, more important, the overhead of calling into the
network code is amortized over the two operations.
One way to do this would be to put a sort of “peephole optimizer” at
the level of message sending. Sending the insert command could be
delayed until there is a command on this connection whose results
depend on the result of the insert. Then all pending messages could
be sent all at once. However, this means the messaging level would
have to know a lot about the semantics of the message, and keeping
track of the dependencies could get extremely complicated and
bug-prone, so I assume you’re not taking that approach.
The other way would be for the level above the message level to tell
the message level that it wants to both insert and get the error code.
This would basically be a “synchronous” API to the message level.
Then it would be easy to combine the two messages.
LikeLike
Yup, you’re absolutely correct, and they’re sent as a single message by all of the other language drivers. I don’t think it’s ever been worth it to optimize the shell because it isn’t generally used for high-performance stuff. The shell is mainly for administrative tasks, there are other tools for importing massive amounts of data.
If anyone is interested in creating a patch to optimize this, though, I’m sure we wouldn’t mind!
LikeLike
In the “Last Error Commands” section of the documentation
that’s referenced, I saw this:
If you’re writing data to MongoDB on multiple connections, then it can
sometimes be important to call getlasterror on one connection to be
certain that the data has been committed to the database. For
instance, if you’re writing to connection #1 and want those writes to
be reflected in reads from connection #2, you can assure this by
calling getlasterror after writing to connection #1.
It seems to me that this should say the last sentence should say “and
before sending the read request on connection #2.” It’s very
important to establish causality at the client side. Otherwise
a read by connection #2 could see the data previous to the
write on connection #1, despite the getlasterror.
LikeLike
Somebody knows about this error message ?
error while loading shared libraries: libpcap.so.0.9: cannot open shared object file: No such file or directory
LikeLike
Sounds like libpcap isn’t installed.
LikeLike