Docs
reem
- class reem.RedisInterface(host='localhost', marshallers=[<reem.marshalling.NumpyMarshaller object>])
- initialize()
Call before doing anything with this connection :return:
- class reem.KeyValueStore(interface='localhost')
Dictionary Like object used for get/set paradigm
The
KeyValueStore
object is one that users will frequently use. It keepsReader
andWriter
objects for each key that the user has read or written to. It does not actually handle the getting and setting of data but producesKeyAccessor
objects that assist with path construction and call the reader and writer’s write and read methods.- interface
Defines the connection to Redis this
- Type
str or RedisInterface
- reader will use. If a str, then a RedisInterface will be created and
- connected to automatically.
- get(path)
Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.
Marginally faster than nested accessors.
- set(path, value)
Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.
Marginally faster than nested accessors.
- __ensure_key_existence(key)
Ensure that the requested key has a reader and writer associated with it.
Returns: None
- track_schema_changes(set_value, keys=None)
Performance optimization for skipping schema update checks
Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.
- Parameters
set_value (according to) – True/False indicating if the keys’ schema should be tracked
keys (List[str]) – List of keys to track. If None, all present and future keys are tracked
set_value –
Returns: None
- class reem.PublishSpace(interface='localhost')
Convenience class for publishing
This class keeps track of
PublishWriter
objects for each key the user has published on.- __ensure_key_existence(key)
Identical to
KeyValueStore
method of same name but does not instantiate aReader
object
- get(path)
Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.
Marginally faster than nested accessors.
- set(path, value)
Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.
Marginally faster than nested accessors.
- track_schema_changes(set_value, keys=None)
Performance optimization for skipping schema update checks
Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.
- Parameters
set_value (according to) – True/False indicating if the keys’ schema should be tracked
keys (List[str]) – List of keys to track. If None, all present and future keys are tracked
set_value –
Returns: None
- class reem.CallbackSubscriber(channel, interface, callback_function, kwargs)
Callback Subscriber Implementation
- call_user_function(channel, message)
Wrapper callback function (wrapping user function) for this class to work with a RawSubscriber object Fits required interface for a ChannelSubscriber callback function :param channel: channel published to :param message: message that was published :return: None :rtype: None
- listen()
Makes this subscriber start listening
Returns: None
- update_local_copy(channel, message)
Update the local copy of the data stored under this channel name in redis.
- Parameters
channel – the name of the channel that was published.
message – message published on that channel
Returns: None
- value()
Get data stored at root
Where as the reader can do
server["foo"].read()
with if server is aKeyValueStore
, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.Returns: all data stored underneath a top level Redis key.
- class reem.SilentSubscriber(channel, interface)
Silent Subscriber Implementation
- update_local_copy(channel, message)
Update the local copy of the data stored under this channel name in redis.
- Parameters
channel – the name of the channel that was published.
message – message published on that channel
Returns: None
- listen()
Makes this subscriber start listening
Returns: None
- value()
Get data stored at root
Where as the reader can do
server["foo"].read()
with if server is aKeyValueStore
, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.Returns: all data stored underneath a top level Redis key.
reem.connection
- class reem.connection.RedisInterface(host='localhost', marshallers=[<reem.marshalling.NumpyMarshaller object>])
- initialize()
Call before doing anything with this connection :return:
- class reem.connection.KeyValueStore(interface='localhost')
Dictionary Like object used for get/set paradigm
The
KeyValueStore
object is one that users will frequently use. It keepsReader
andWriter
objects for each key that the user has read or written to. It does not actually handle the getting and setting of data but producesKeyAccessor
objects that assist with path construction and call the reader and writer’s write and read methods.- interface
Defines the connection to Redis this
- Type
str or RedisInterface
- reader will use. If a str, then a RedisInterface will be created and
- connected to automatically.
- get(path)
Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.
Marginally faster than nested accessors.
- set(path, value)
Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.
Marginally faster than nested accessors.
- __ensure_key_existence(key)
Ensure that the requested key has a reader and writer associated with it.
Returns: None
- track_schema_changes(set_value, keys=None)
Performance optimization for skipping schema update checks
Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.
- Parameters
set_value (according to) – True/False indicating if the keys’ schema should be tracked
keys (List[str]) – List of keys to track. If None, all present and future keys are tracked
set_value –
Returns: None
- class reem.connection.PublishSpace(interface='localhost')
Convenience class for publishing
This class keeps track of
PublishWriter
objects for each key the user has published on.- __ensure_key_existence(key)
Identical to
KeyValueStore
method of same name but does not instantiate aReader
object
- get(path)
Reads a path in the form ‘key.element1.element2’ from the redis server. If the path doesn’t exist, an error is raised.
Marginally faster than nested accessors.
- set(path, value)
Sets a value in a path in the form ‘key.element1.element2’ from the redis server. If any elements along the path, except for the last key, do not exist, an error is raised.
Marginally faster than nested accessors.
- track_schema_changes(set_value, keys=None)
Performance optimization for skipping schema update checks
Stop checking for schema updates when setting data. Use ONLY if your data’s schema is static and you are really trying to eek out every bit of optimization.
- Parameters
set_value (according to) – True/False indicating if the keys’ schema should be tracked
keys (List[str]) – List of keys to track. If None, all present and future keys are tracked
set_value –
Returns: None
- class reem.connection.SilentSubscriber(channel, interface)
Silent Subscriber Implementation
- update_local_copy(channel, message)
Update the local copy of the data stored under this channel name in redis.
- Parameters
channel – the name of the channel that was published.
message – message published on that channel
Returns: None
- listen()
Makes this subscriber start listening
Returns: None
- value()
Get data stored at root
Where as the reader can do
server["foo"].read()
with if server is aKeyValueStore
, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.Returns: all data stored underneath a top level Redis key.
- class reem.connection.CallbackSubscriber(channel, interface, callback_function, kwargs)
Callback Subscriber Implementation
- call_user_function(channel, message)
Wrapper callback function (wrapping user function) for this class to work with a RawSubscriber object Fits required interface for a ChannelSubscriber callback function :param channel: channel published to :param message: message that was published :return: None :rtype: None
- listen()
Makes this subscriber start listening
Returns: None
- update_local_copy(channel, message)
Update the local copy of the data stored under this channel name in redis.
- Parameters
channel – the name of the channel that was published.
message – message published on that channel
Returns: None
- value()
Get data stored at root
Where as the reader can do
server["foo"].read()
with if server is aKeyValueStore
, accessing the root value of a subscriber is not as easy. This method retrieves all data stored underneath a top level key.Returns: all data stored underneath a top level Redis key.
- class reem.connection.Writer(top_key_name, interface)
Responsible for setting data inside Redis
The Writer class is an internal class that is used for all data sent to Redis (not including pub/sub messages). Each key that will have nested data below requires a new instantiation of Writer
- top_key_name
The name of the Redis key under which JSON data will be stored. To Redis, this will become a ReJSON key name. It is also used to generate the Redis key name that marshallers use to store non JSON data.
- Type
str
- interface
Defines the connection to Redis this writer will use
- Type
- __initialize_metadata()
Pull metadata for this key from Redis or set a default
- Returns
None
- send_to_redis(set_path, set_value)
Execute equivalent of
JSON.SET self.top_key_name <set_path> <set_value>
From the user’s perspective, it executes
JSON.SET self.top_key_name <set_path> <set_value>
except thatset_value
can be json-incompatible. In such a case, it determines what non-serializable types are inside set_value, stores the serializable data as a JSON, and stores the non-serializable data usingself.interface
’s marshallers.- Parameters
set_path (str) – path underneath JSON key to set
set_value – value to set
- Returns
None
- delete_from_redis(del_path)
Execute equivalent of
JSON.DEL self.top_key_name <set_path>
- Parameters
del_path (str) – path underneath JSON key to delete
- Returns
None
- __process_metadata(set_path, set_value)
Handle metadata updates
Given the path and value the user would like to set, check if there are non-serializable data types and update metadata locally and in Redis. Happens without pipeline
- Parameters
set_path (str) – path underneath JSON key to set
set_value – value to set
- Returns
None
- __publish_non_serializables(set_path, set_value)
Publish JSON incompatible data to Redis
Given a set, publish the non-serializable components to redis, given that metadata has been updated already
- Parameters
set_path (str) – path underneath JSON key to set
set_value – value to set
- Returns
None
- __publish_serializables(set_path, set_value)
Publish the serializable portion of
set_value
Take out the non-serializable part of set_value and publish it at set_path
- Parameters
set_path (str) – path underneath JSON key to set
set_value – value to set
Returns: None
- class reem.connection.Reader(top_key_name, interface)
Responsible for getting data from Redis
The Reader class is an internal class that is used for all read from Redis (not including pub/sub messages). Each key that will have nested data below requires a new instantiation of Reader
- top_key_name
The name of the Redis key under which JSON data is stored
- Type
str
- interface
Defines the connection to Redis this reader will use
- Type
- read_from_redis(read_path)
Read specified path from Redis
This is the only public method of the Reader class. It will retrieve the data stored at a specified path from Redis. At a high level, it reads data stored with ReJSON and inserts non-JSON compatible data at appropriate paths using the metadata associated with this key.
- Parameters
read_path (str) – path the user wants to read
Returns: data stored at value in Redis
- update_metadata()
Update the local copy of metadata if a relevant path has been updated.
The metadata listener is a redis client subscribed to key-space notifications. If a relevant path is updated, this Reader’s
pull_metadata
flag will be turned on. Ifpull_metadata
isTrue
, then the reader will fetch metadata from the Redis server.Returns: None
- queue_reads(read_path)
Queue reads in a pipeline
Queue all redis queries necessary to read data at path into the appropriate redis pipeline. First, queue decoded pipeline with the ReJSON query Next, queue all the special path reads with the non-decoded pipeline and marshallers
- Parameters
read_path – path user wants to read
Returns: None
- build_dictionary(read_path)
Execute pipelines and consolidate data into a dictionary
- Parameters
read_path – path user wants to read
Returns: The data stored at
read_path
in Redis
- pull_special_path(read_path)
Directly pull a non-JSON path
If the user specified path is not in JSON, this will retrieve the data directly without going through ReJSON.
- Parameters
read_path – path user wants to read
Returns:
- class reem.connection.PublishWriter(top_key_name, interface)
Defines PublishWriter behavior
The PublishWriter is identical to the writer but publishes a message when it writes a value.
- send_to_redis(set_path, set_value)
PublishWriter equivalent of Writer
send_to_redis
This is an equivalent function to
Writer
’ssend_to_redis
method but also publishes a message indicating what channel has been updated.- Parameters
set_path (str) – path underneath JSON key to set
set_value – value to set
Returns: None
- delete_from_redis(del_path)
Execute equivalent of
JSON.DEL self.top_key_name <set_path>
- Parameters
del_path (str) – path underneath JSON key to delete
- Returns
None