Changing a mongo cursor into JSON object in Python
Welcome to our tutorial on serializing PyMongo cursors into JSON. On this article, we’ll cowl learn how to correctly deal with ObjectId and datetime objects, in addition to every other object, utilizing a customized JSONEncoder.
One frequent job when working with PyMongo is the necessity to serialize information for storage or for switch over a community. On this tutorial, we’ll have a look at learn how to serialize a PyMongo cursor, which is a standard information construction used to retailer outcomes from a MongoDB question, into JSON format.
A generally reported error when doing so, is the next TyperError:
TypeError: ObjectId('') shouldn't be JSON serializable
We will even delve into learn how to correctly deal with complicated information sorts resembling ObjectId and datetime objects, which can’t be serialized straight into JSON. We’ll present you learn how to use a customized JSONEncoder to correctly deal with these objects and every other customized object sorts you might have in your PyMongo cursor.
So if you wish to discover ways to serialize PyMongo cursors into JSON and deal with complicated information sorts, preserve studying!
Making a customized JSONEncoder
The JSONEncoder — which is a member of the json module of the usual library — is an extensible JSON encoder for Python information buildings. By default, is helps the next serialisations:
+-----------------------------------------+--------+
| Python | JSON |
+-----------------------------------------+--------+
| dict | object |
| record, tuple | array |
| str | string |
| int, float, int & float-derived Enums | quantity |
| True | true |
| False | false |
| None | null |
+-----------------------------------------+--------+
Which means that every time an object of a distinct information sort is noticed (which isn’t listed on the above desk), a TypeError will likely be raised.
When working with paperwork in Mongo, by default each doc may have an assigned _id that corresponds to a singular identifier for each doc inside a set. Now everytime you question a Mongo assortment, a cursor will likely be returned containing (a pointer to) the retrieved paperwork the place every doc will even have the _id discipline of sort ObjectId.
Due to this fact, if you happen to try to serialise such paperwork utilizing the default JSONEncoder, you’ll find yourself getting the error talked about within the introduction of this tutorial:
TypeError: ObjectId('') shouldn't be JSON serializable
Due to this fact, to be able to handle to serialise such objects contained within the PyMongo cursor, we have to prolong the default JSONEncoder such that it correctly handles such information sorts the best way we’d need it to. To attain this we will even must implement the default methodology to return the mapping we want, as specified within the documentation.
To increase this to acknowledge different objects, subclass and implement a
default()methodology with one other methodology that returns a serializable object foroif doable, in any other case it ought to name the superclass implementation (to boostTypeError).
In our customized JSONEncoder, I’m about to serialise any occasion of bson.ObjectId and datetime.datetime to str. Relying on the paperwork contained in your personal Mongo cursor, you might have to specify and deal with further (or maybe much less) information sorts.
import json
from datetime import datetime
from typing import Anyfrom bson import ObjectId
class MongoJSONEncoder(json.JSONEncoder):
def default(self, o: Any) -> Any:
if isinstance(o, ObjectId):
return str(o)
if isinstance(o, datetime):
return str(o)
return json.JSONEncoder.default(self, o)
Making a Python object out of the JSON object
Lastly, in case you want to convert the newly created JSON object right into a Python object (that may be a record of dictionaries containing the key-value pairs that correspond to the doc values inside the Mongo cursor), all it’s good to name is the json.masses() operate:
data_obj = json.masses(data_json)
Closing Ideas
On this tutorial, we discovered learn how to serialize PyMongo cursors into JSON and correctly deal with complicated information sorts resembling ObjectId and datetime objects. We achieved this by making a customized JSONEncoder that prolonged the default JSONEncoder and applied a default() methodology.
We then used this practice encoder to encode the PyMongo cursor, and eventually, we transformed the ensuing JSON object right into a Python object utilizing the json.masses() operate. This tutorial demonstrated learn how to deal with ObjectId and datetime objects, however the customized JSONEncoder may also be prolonged to deal with every other customized object sorts that could be current within the PyMongo cursor.
