Database interactions

Database object

class rocksdb.DB
__init__(db_name, Options opts, read_only=False)
Parameters:
  • db_name (unicode) – Name of the database to open
  • opts (rocksdb.Options) – Options for this specific database
  • read_only (bool) – If True the database is opened read-only. All DB calls which modify data will raise an Exception.
put(key, value, sync=False, disable_wal=False)

Set the database entry for “key” to “value”.

Parameters:
  • key (bytes) – Name for this entry
  • value (bytes) – Data for this entry
  • sync (bool) –

    If True, the write will be flushed from the operating system buffer cache (by calling WritableFile::Sync()) before the write is considered complete. If this flag is true, writes will be slower.

    If this flag is False, and the machine crashes, some recent writes may be lost. Note that if it is just the process that crashes (i.e., the machine does not reboot), no writes will be lost even if sync == False.

    In other words, a DB write with sync == False has similar crash semantics as the “write()” system call. A DB write with sync == True has similar crash semantics to a “write()” system call followed by “fdatasync()”.

  • disable_wal (bool) – If True, writes will not first go to the write ahead log, and the write may got lost after a crash.
delete(key, sync=False, disable_wal=False)

Remove the database entry for “key”.

Parameters:
Raises rocksdb.errors.NotFound:
 

If the key did not exists
merge(key, value, sync=False, disable_wal=False)

Merge the database entry for “key” with “value”. The semantics of this operation is determined by the user provided merge_operator when opening DB.

See rocksdb.DB.put() for the parameters

Raises:rocksdb.errors.NotSupported if this is called and no rocksdb.Options.merge_operator was set at creation
write(batch, sync=False, disable_wal=False)

Apply the specified updates to the database.

Parameters:
get(key, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")
Parameters:
  • key (bytes) – Name to get
  • verify_checksums (bool) – If True, all data read from underlying storage will be verified against corresponding checksums.
  • fill_cache (bool) – Should the “data block”, “index block” or “filter block” read for this iteration be cached in memory? Callers may wish to set this field to False for bulk scans.
  • snapshot (rocksdb.Snapshot) – If not None, read as of the supplied snapshot (which must belong to the DB that is being read and which must not have been released). Is it None a implicit snapshot of the state at the beginning of this read operation is used
  • read_tier (string) –

    Specify if this read request should process data that ALREADY resides on a particular cache. If the required data is not found at the specified cache, then rocksdb.errors.Incomplete is raised.

    Use all if a fetch from disk is allowed.
    Use cache if only data from cache is allowed.
Returns:

None if not found, else the value for this key
multi_get(keys, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")
Parameters:keys (list of bytes) – Keys to fetch

For the other params see rocksdb.DB.get()

Returns:A dict where the value is either bytes or None if not found
Raises:If the fetch for a single key fails

Note

keys will not be “de-duplicated”. Duplicate keys will return duplicate values in order.

key_may_exist(key, fetch=False, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")

If the key definitely does not exist in the database, then this method returns False, else True. If the caller wants to obtain value when the key is found in memory, fetch should be set to True. This check is potentially lighter-weight than invoking DB::get(). One way to make this lighter weight is to avoid doing any IOs.

Parameters:
  • key (bytes) – Key to check
  • fetch (bool) – Obtain also the value if found

For the other params see rocksdb.DB.get()

Returns:
  • (True, None) if key is found but value not in memory
  • (True, None) if key is found and fetch=False
  • (True, <data>) if key is found and value in memory and fetch=True
  • (False, None) if key is not found
iterkeys(fetch=False, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")

Iterate over the keys

For other params see rocksdb.DB.get()

Returns:A iterator object which is not valid yet. Call first one of the seek methods of the iterator to position it
Return type:rocksdb.BaseIterator
itervalues(fetch=False, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")

Iterate over the values

For other params see rocksdb.DB.get()

Returns:A iterator object which is not valid yet. Call first one of the seek methods of the iterator to position it
Return type:rocksdb.BaseIterator
iteritems(fetch=False, verify_checksums=False, fill_cache=True, snapshot=None, read_tier="all")

Iterate over the items

For other params see rocksdb.DB.get()

Returns:A iterator object which is not valid yet. Call first one of the seek methods of the iterator to position it
Return type:rocksdb.BaseIterator
snapshot()

Return a handle to the current DB state. Iterators created with this handle will all observe a stable snapshot of the current DB state.

Return type:rocksdb.Snapshot
get_property(prop)

DB implementations can export properties about their state via this method. If “property” is a valid property understood by this DB implementation, a byte string with its value is returned. Otherwise None

Valid property names include:

  • b"rocksdb.num-files-at-level<N>": return the number of files at level <N>,

    where <N> is an ASCII representation of a level number (e.g. “0”).

  • b"rocksdb.stats": returns a multi-line byte string that describes statistics

    about the internal operation of the DB.

  • b"rocksdb.sstables": returns a multi-line byte string that describes all

    of the sstables that make up the db contents.

  • b"rocksdb.num-immutable-mem-table": Number of immutable mem tables.

  • b"rocksdb.mem-table-flush-pending": Returns 1 if mem table flush is pending, otherwise 0.

  • b"rocksdb.compaction-pending": Returns 1 if a compaction is pending, otherweise 0.

  • b"rocksdb.background-errors": Returns accumulated background errors encountered.

  • b"rocksdb.cur-size-active-mem-table": Returns current size of the active memtable.

get_live_files_metadata()

Returns a list of all table files.

It returns a list of dict’s were each dict has the following keys.

name
Name of the file
level
Level at which this file resides
size
File size in bytes
smallestkey
Smallest user defined key in the file
largestkey
Largest user defined key in the file
smallest_seqno
smallest seqno in file
largest_seqno
largest seqno in file
compact_range(begin=None, end=None, reduce_level=False, target_level=-1)

Compact the underlying storage for the key range [begin,end]. The actual compaction interval might be superset of [begin, end]. In particular, deleted and overwritten versions are discarded, and the data is rearranged to reduce the cost of operations needed to access the data.

This operation should typically only be invoked by users who understand the underlying implementation.

begin == None is treated as a key before all keys in the database. end == None is treated as a key after all keys in the database. Therefore the following call will compact the entire database: db.compact_range().

Note that after the entire database is compacted, all data are pushed down to the last level containing any data. If the total data size after compaction is reduced, that level might not be appropriate for hosting all the files. In this case, client could set reduce_level to True, to move the files back to the minimum level capable of holding the data set or a given level (specified by non-negative target_level).

Parameters:
  • begin (bytes) – Key where to start compaction. If None start at the beginning of the database.
  • end (bytes) – Key where to end compaction. If None end at the last key of the database.
  • reduce_level (bool) – If True allow rocksdb to move the data to another level, if the current is not big enouth. If False you may end with a bigger level than configured.
  • target_level (int) – Level where to push the the range to compact.
options

Returns the associated rocksdb.Options instance.

Note

Changes to this object have no effect anymore. Consider this as read-only

Iterator

class rocksdb.BaseIterator

Base class for all iterators in this module. After creation a iterator is invalid. Call one of the seek methods first before starting iteration

seek_to_first()

Position at the first key in the source

seek_to_last()

Position at the last key in the source

seek(key)
Parameters:key (bytes) – Position at the first key in the source that at or past

Methods to support the python iterator protocol

__iter__()
__next__()
__reversed__()

Snapshot

class rocksdb.Snapshot

Opaque handler for a single Snapshot. Snapshot is released if nobody holds a reference on it. Retrieved via rocksdb.DB.snapshot()

WriteBatch

class rocksdb.WriteBatch

WriteBatch holds a collection of updates to apply atomically to a DB.

The updates are applied in the order in which they are added to the WriteBatch. For example, the value of “key” will be “v3” after the following batch is written:

batch = rocksdb.WriteBatch()
batch.put(b"key", b"v1")
batch.delete(b"key")
batch.put(b"key", b"v2")
batch.put(b"key", b"v3")
__init__(data=None)

Creates a WriteBatch.

Parameters:data (bytes) – A serialized version of a previous WriteBatch. As retrieved from a previous .data() call. If None a empty WriteBatch is generated
put(key, value)

Store the mapping “key->value” in the database.

Parameters:
  • key (bytes) – Name of the entry to store
  • value (bytes) – Data of this entry
merge(key, value)

Merge “value” with the existing value of “key” in the database.

Parameters:
  • key (bytes) – Name of the entry to merge
  • value (bytes) – Data to merge
delete(key)

If the database contains a mapping for “key”, erase it. Else do nothing.

Parameters:key (bytes) – Key to erase
clear()

Clear all updates buffered in this batch.

Note

Don’t call this method if there is an outstanding iterator. Calling rocksdb.WriteBatch.clear() with outstanding iterator, leads to SEGFAULT.

data()

Retrieve the serialized version of this batch.

Return type:bytes
count()

Returns the number of updates in the batch

Return type:int
__iter__()

Returns an iterator over the current contents of the write batch.

If you add new items to the batch, they are not visible for this iterator. Create a new one if you need to see them.

Note

Calling rocksdb.WriteBatch.clear() on the write batch invalidates the iterator. Using a iterator where its corresponding write batch has been cleared, leads to SEGFAULT.

Return type:rocksdb.WriteBatchIterator

WriteBatchIterator

class rocksdb.WriteBatchIterator
__iter__()

Returns self.

__next__()

Returns the next item inside the corresponding write batch. The return value is a tuple of always size three.

First item (Name of the operation):

  • "Put"
  • "Merge"
  • "Delete"
Second item (key):
Key for this operation.
Third item (value):
The value for this operation. Empty for "Delete".

changelog tutoro

Errors

exception rocksdb.errors.NotFound
exception rocksdb.errors.Corruption
exception rocksdb.errors.NotSupported
exception rocksdb.errors.InvalidArgument
exception rocksdb.errors.RocksIOError
exception rocksdb.errors.MergeInProgress
exception rocksdb.errors.Incomplete