Package org.yamcs.yarch.rocksdb

Class Tablespace

  • public class Tablespace
extends Object
    Tablespaces are used to store data by the RdbStorageEngine. Each tablespace can store data from one or more Yamcs instances.

    Tablespaces are rocksdb databases normally stored in the yamcs data directory (/storage/yamcs-data). Each corresponds to a directory <tablespace-name>.rdb and has a definition file tablespace-name.tbs.

    Tablespaces can also have time based partitions in different RocksDB databases in sub-directories such as <tablespace-name>.rdb/YYYY/

    There are two column families in the main database: - the _metadata_ column family - contains metadata. - the default column family - contains data.

    The data is partitioned by the first 4 bytes of the key which we call tbsIndex.

    One tbsIndex corresponds to a so called tablespace record. For example tbsIndex=5 can correspond to all telemetry packets for packet XYZ.

    Except for the 4 bytes tbsIndex, the rest of the key and value are completely dependent on the data type. For example for yarch table data, the rest of key following the 4 bytes tbsIndex represents the key of the row in the table.

    The metadata contains three types of records, identified by the first byte of the key:

    • key: 0x01

      value: 1 byte version number (0x1), 4 bytes max tbsIndex

      used to store the max tbsIndex and also stores a version number in case the format will change in the future

    • key: 0x02, 1 byte record type, 4 bytes tbsIndex

      value: protobuf encoded TablespaceRecord

      Used to store the information corresponding to the given tbsIndex. The record type corresponds to the Type enumerations from tablespace.proto

    • key: 0x03, 1 byte record type, sequence name encoded in UTF8

      value: last sequence number 8 bytes big endian

    • Constructor Detail

      • Tablespace

        public Tablespace​(String name)

    • Method Detail

      • getName

        public String getName()

      • getTablePartitions

        public List<Tablespace.TablespaceRecord> getTablePartitions​(String instanceName,
                                                            String tableName)
                                                     throws org.rocksdb.RocksDBException,
                                                            IOException
        Returns a list of all records of type TABLE_PARTITION for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.
        Throws:
        org.rocksdb.RocksDBException
        IOException

      • getTableHistograms

        public List<Tablespace.TablespaceRecord> getTableHistograms​(String instanceName,
                                                            String tableName)
                                                     throws org.rocksdb.RocksDBException,
                                                            IOException
        Returns a list of all records of type HISTOGRAM for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.
        Throws:
        org.rocksdb.RocksDBException
        IOException

      • createMetadataRecord

        public Tablespace.TablespaceRecord createMetadataRecord​(String yamcsInstance,
                                                        Tablespace.TablespaceRecord.Builder trb)
                                                 throws org.rocksdb.RocksDBException
        Creates a new tablespace record and adds it to the metadata database
        Parameters:
        trb - the builder has to have all fields set except for the tbsIndex which will be assigned by this method
        Returns:
        a fully built
        Throws:
        org.rocksdb.RocksDBException

      • getCustomDataDir

        public String getCustomDataDir()

      • getRdb

        public YRDB getRdb​(String partitionDir,
                   boolean readOnly)
        (Creates) and returns a database in the given partition directory. If the directory is null, return then main tablespace db
        Parameters:
        partitionDir -
        readOnly -

      • getRdb

        public YRDB getRdb​(String relativePath)

      • getRdb

        public YRDB getRdb()
        Get the main database of the tablespace

      • dispose

        public void dispose​(YRDB rdb)

      • setCustomDataDir

        public void setCustomDataDir​(String dataDir)

      • getDataDir

        public String getDataDir()

      • removeTbsIndex

        public void removeTbsIndex​(Tablespace.TablespaceRecord.Type type,
                           int tbsIndex)
                    throws org.rocksdb.RocksDBException
        Removes the tbsIndex from the metadata and all the associated data from the main db (data might still be present in the partitions)
        Parameters:
        type -
        tbsIndex -
        Throws:
        org.rocksdb.RocksDBException

      • removeTbsIndices

        public void removeTbsIndices​(Tablespace.TablespaceRecord.Type type,
                             IntArray tbsIndexArray)
                      throws org.rocksdb.RocksDBException
        Removes the tbs indices with ALL the associated data from the main db (data might still be present in the partitions)
        Parameters:
        type -
        tbsIndexArray -
        Throws:
        org.rocksdb.RocksDBException

      • getRdbFactory

        public RDBFactory getRdbFactory()

      • putData

        public void putData​(byte[] key,
                    byte[] value)
             throws org.rocksdb.RocksDBException
        inserts data into the main partition
        Parameters:
        key -
        value -
        Throws:
        org.rocksdb.RocksDBException

      • getData

        public byte[] getData​(byte[] key)
               throws org.rocksdb.RocksDBException
        Throws:
        org.rocksdb.RocksDBException

      • remove

        public void remove​(byte[] key)
            throws org.rocksdb.RocksDBException
        Throws:
        org.rocksdb.RocksDBException

      • getTable

        public RdbTable getTable​(TableDefinition tblDef)
        returns the table associated to this definition or null if this table is not known.
        Parameters:
        tblDef -
        Returns:

      • close

        public void close()