org.yamcs.yarch.rocksdb

Class Tablespace

java.lang.Object

org.yamcs.yarch.rocksdb.Tablespace

public class Tablespace
extends Object

Tablespaces are rocksdb databases normally stored in the yamcs data directory (/storage/yamcs-data). Each corresponds to a directory <tablespace-name>.rdb and has a definition file tablespace-name.tbs. Tablespaces can also have time based partitions in different RocksDB databases in sub-directories such as <tablespace-name>.rdb/YYYY/ There are two column families in the main database: - the _metadata_ column family - contains metadata. - the default column family - contains data. The data is partitioned by the first 4 bytes of the key which we call tbsIndex. One tbsIndex corresponds to a so called tablespace record. For example tbsIndex=5 can correspond to all telemetry packets for packet XYZ. Except for the 4 bytes tbsIndex, the rest of the key and value are completely dependent on the data type. For example for yarch table data, the rest of key following the 4 bytes tbsIndex represents the key of the row in the table. The metadata contains two types of records, identified by the first byte of the key: - key: 0x1 value: 1 byte version number (0x1), 4 bytes max tbsIndex used to store the max tbsIndex and also stores a version number in case the format will change in the future - key: 0x2, 1 byte record type, 4 bytes tbsIndex value: protobuf encoded TablespaceRecord used to store what information corresponds to each tbsIndex the record type corresponds to the Type enumerations from tablespace.proto In tablespace.proto for a description of the possible record types. The metadata

Constructor Summary

Constructors

Constructor and Description
Tablespace(String name, byte id)

Method Summary

Modifier and Type	Method and Description
void	close()
Tablespace.TablespaceRecord	createMetadataRecord(String yamcsInstance, Tablespace.TablespaceRecord.Builder trb) Creates a new tablespace record and adds it to the metadata database
void	dispose(YRDB rdb)
List<Tablespace.TablespaceRecord>	filter(Tablespace.TablespaceRecord.Type type, String instanceName, java.util.function.Predicate<Tablespace.TablespaceRecord.Builder> p)
String	getCustomDataDir()
byte[]	getData(byte[] key)
String	getDataDir()
byte	getId()
String	getName()
YRDB	getRdb() Get the main database of the tablespace
YRDB	getRdb(String relativePath)
YRDB	getRdb(String partitionDir, boolean readOnly) (Creates) and returns a database in the given partition directory.
RDBFactory	getRdbFactory()
List<Tablespace.TablespaceRecord>	getTableHistograms(String instanceName, String tableName) Returns a list of all records of type HISTOGRAM for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.
List<Tablespace.TablespaceRecord>	getTablePartitions(String instanceName, String tableName) Returns a list of all records of type TABLE_PARTITION for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.
void	loadDb(boolean readonly)
void	putData(byte[] key, byte[] value) inserts data into the main partition
void	remove(byte[] key)
void	removeTbsIndex(Tablespace.TablespaceRecord.Type type, int tbsIndex)
void	removeTbsIndices(Tablespace.TablespaceRecord.Type type, IntArray tbsIndexArray)
void	setCustomDataDir(String dataDir)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tablespace

public Tablespace(String name,
                  byte id)

Method Detail

loadDb

public void loadDb(boolean readonly)
            throws IOException,
                   org.rocksdb.RocksDBException

Throws:

IOException

org.rocksdb.RocksDBException

getName

public String getName()

getId

public byte getId()

getTablePartitions

public List<Tablespace.TablespaceRecord> getTablePartitions(String instanceName,
                                                            String tableName)
                                                     throws org.rocksdb.RocksDBException,
                                                            IOException

Returns a list of all records of type TABLE_PARTITION for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.

Throws:

org.rocksdb.RocksDBException

IOException

getTableHistograms

public List<Tablespace.TablespaceRecord> getTableHistograms(String instanceName,
                                                            String tableName)
                                                     throws org.rocksdb.RocksDBException,
                                                            IOException

Returns a list of all records of type HISTOGRAM for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.

Throws:

org.rocksdb.RocksDBException

IOException

filter

public List<Tablespace.TablespaceRecord> filter(Tablespace.TablespaceRecord.Type type,
                                                String instanceName,
                                                java.util.function.Predicate<Tablespace.TablespaceRecord.Builder> p)
                                         throws org.rocksdb.RocksDBException,
                                                DatabaseCorruptionException

Throws:

org.rocksdb.RocksDBException

DatabaseCorruptionException

createMetadataRecord

public Tablespace.TablespaceRecord createMetadataRecord(String yamcsInstance,
                                                        Tablespace.TablespaceRecord.Builder trb)
                                                 throws org.rocksdb.RocksDBException

Creates a new tablespace record and adds it to the metadata database

Parameters:

trb - - the builder has to have all fields set except for the tbsIndex which will be assigned by this method

Returns:

a fully built

Throws:

org.rocksdb.RocksDBException

getCustomDataDir

public String getCustomDataDir()

getRdb

public YRDB getRdb(String partitionDir,
                   boolean readOnly)
            throws IOException

(Creates) and returns a database in the given partition directory. If the directory is null, return then main tablespace db

Parameters:

partitionDir -

readOnly -

Throws:

IOException

getRdb

public YRDB getRdb(String relativePath)
            throws IOException

Throws:

IOException

getRdb

public YRDB getRdb()

Get the main database of the tablespace

dispose

public void dispose(YRDB rdb)

setCustomDataDir

public void setCustomDataDir(String dataDir)

getDataDir

public String getDataDir()

removeTbsIndex

public void removeTbsIndex(Tablespace.TablespaceRecord.Type type,
                           int tbsIndex)
                    throws org.rocksdb.RocksDBException

Throws:

org.rocksdb.RocksDBException

removeTbsIndices

public void removeTbsIndices(Tablespace.TablespaceRecord.Type type,
                             IntArray tbsIndexArray)
                      throws org.rocksdb.RocksDBException

Throws:

org.rocksdb.RocksDBException

close

public void close()

getRdbFactory

public RDBFactory getRdbFactory()

putData

public void putData(byte[] key,
                    byte[] value)
             throws org.rocksdb.RocksDBException

inserts data into the main partition

Parameters:

key -

value -

Throws:

org.rocksdb.RocksDBException

getData

public byte[] getData(byte[] key)
               throws org.rocksdb.RocksDBException

Throws:

org.rocksdb.RocksDBException

remove

public void remove(byte[] key)
            throws org.rocksdb.RocksDBException

Throws:

org.rocksdb.RocksDBException