Package org.kairosdb.bigqueue

Interface IFanOutQueue

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static long EARLIEST
      Constant represents earliest timestamp
      static long LATEST
      Constant represents latest timestamp

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      byte[] dequeue​(String fanoutId)
      Retrieves and removes the front of a fan out queue
      byte[] dequeue​(String fanoutId, boolean useLatest)
      Retrieves and removes the front of a fan out queue
      long enqueue​(byte[] data)
      Adds an item at the back of the queue
      long findClosestIndex​(long timestamp)
      Find an index closest to the specific timestamp when the corresponding item was enqueued.
      void flush()
      Force to persist current state of the queue, normally, you don't need to flush explicitly since: 1.)
      byte[] get​(long index)
      Retrieves data item at the specific index of the queue
      long getBackFileSize()
      Current total size of the back files of this queue
      long getFrontIndex()
      Get the queue front index, this is the earliest appended index
      long getFrontIndex​(String fanoutId)
      Get front index of specific fanout queue
      long getFrontIndex​(String fanoutId, boolean useLatest)
      Get front index of specific fanout queue
      int getLength​(long index)
      Get length of data item at specific index of the queue
      long getRearIndex()
      Get the queue rear index, this is the next to be appended index
      long getTimestamp​(long index)
      Get timestamp of data item at specific index of the queue, this is the timestamp when corresponding item was appended into the queue.
      boolean isEmpty()
      Determines whether the queue is empty
      boolean isEmpty​(String fanoutId)
      Determines whether a fan out queue is empty
      boolean isEmpty​(String fanoutId, boolean useLatest)
      Determines whether a fan out queue is empty
      void limitBackFileSize​(long sizeLmit)
      Limit the back file size of this queue, truncate back files and advance the queue front if necessary.
      byte[] peek​(String fanoutId)
      Peek the item at the front of a fanout queue, without removing it from the queue
      byte[] peek​(String fanoutId, boolean useLatest)
      Peek the item at the front of a fanout queue, without removing it from the queue
      int peekLength​(String fanoutId)
      Peek the length of the item at the front of a fan out queue
      int peekLength​(String fanoutId, boolean useLatest)
      Peek the length of the item at the front of a fan out queue
      long peekTimestamp​(String fanoutId)
      Peek the timestamp of the item at the front of a fan out queue
      long peekTimestamp​(String fanoutId, boolean useLatest)
      Peek the timestamp of the item at the front of a fan out queue
      void removeAll()
      Removes all items of a queue, this will empty the queue and delete all back data files.
      void removeBefore​(long timestamp)
      Remove all data before specific timestamp, truncate back files and advance the queue front if necessary.
      void resetQueueFrontIndex​(String fanoutId, long index)
      Reset the front index of a fanout queue.
      long size()
      Total number of items remaining in the queue.
      long size​(String fanoutId)
      Total number of items remaining in the fan out queue
      long size​(String fanoutId, boolean useLatest)
      Total number of items remaining in the fan out queue

    • Method Detail

      • isEmpty

        boolean isEmpty​(String fanoutId,
                boolean useLatest)
         throws IOException
        Determines whether a fan out queue is empty
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        true if empty, false otherwise
        Throws:
        IOException - exception thrown if IO error occurs

      • isEmpty

        boolean isEmpty​(String fanoutId)
         throws IOException
        Determines whether a fan out queue is empty
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        true if empty, false otherwise
        Throws:
        IOException - exception thrown if IO error occurs

      • isEmpty

        boolean isEmpty()
        Determines whether the queue is empty
        Returns:
        true if empty, false otherwise

      • enqueue

        long enqueue​(byte[] data)
      throws IOException
        Adds an item at the back of the queue
        Parameters:
        data - to be enqueued data
        Returns:
        index where the item was appended
        Throws:
        IOException - exception throws if there is any IO error during enqueue operation.

      • dequeue

        byte[] dequeue​(String fanoutId)
        throws IOException
        Retrieves and removes the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during dequeue operation.

      • dequeue

        byte[] dequeue​(String fanoutId,
               boolean useLatest)
        throws IOException
        Retrieves and removes the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during dequeue operation.

      • peek

        byte[] peek​(String fanoutId)
     throws IOException
        Peek the item at the front of a fanout queue, without removing it from the queue
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • peek

        byte[] peek​(String fanoutId,
            boolean useLatest)
     throws IOException
        Peek the item at the front of a fanout queue, without removing it from the queue
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • peekLength

        int peekLength​(String fanoutId)
        throws IOException
        Peek the length of the item at the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • peekLength

        int peekLength​(String fanoutId,
               boolean useLatest)
        throws IOException
        Peek the length of the item at the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • peekTimestamp

        long peekTimestamp​(String fanoutId)
            throws IOException
        Peek the timestamp of the item at the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • peekTimestamp

        long peekTimestamp​(String fanoutId,
                   boolean useLatest)
            throws IOException
        Peek the timestamp of the item at the front of a fan out queue
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        data at the front of a queue
        Throws:
        IOException - exception throws if there is any IO error during peek operation.

      • get

        byte[] get​(long index)
    throws IOException
        Retrieves data item at the specific index of the queue
        Parameters:
        index - data item index
        Returns:
        data at index
        Throws:
        IOException - exception throws if there is any IO error during fetch operation.

      • getLength

        int getLength​(long index)
       throws IOException
        Get length of data item at specific index of the queue
        Parameters:
        index - data item index
        Returns:
        length of data item
        Throws:
        IOException - exception throws if there is any IO error during fetch operation.

      • getTimestamp

        long getTimestamp​(long index)
           throws IOException
        Get timestamp of data item at specific index of the queue, this is the timestamp when corresponding item was appended into the queue.
        Parameters:
        index - data item index
        Returns:
        timestamp of data item
        Throws:
        IOException - exception throws if there is any IO error during fetch operation.

      • size

        long size​(String fanoutId)
   throws IOException
        Total number of items remaining in the fan out queue
        Parameters:
        fanoutId - the fanout identifier
        Returns:
        total number
        Throws:
        IOException - exception thrown if IO error occurs

      • size

        long size​(String fanoutId,
          boolean useLatest)
   throws IOException
        Total number of items remaining in the fan out queue
        Parameters:
        fanoutId - the fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        total number
        Throws:
        IOException - exception thrown if IO error occurs

      • size

        long size()
        Total number of items remaining in the queue.
        Returns:
        total number

      • flush

        void flush()
        Force to persist current state of the queue, normally, you don't need to flush explicitly since: 1.) FanOutQueue will automatically flush a cached page when it is replaced out, 2.) FanOutQueue uses memory mapped file technology internally, and the OS will flush the changes even your process crashes, call this periodically only if you need transactional reliability and you are aware of the cost to performance.

      • removeBefore

        void removeBefore​(long timestamp)
           throws IOException
        Remove all data before specific timestamp, truncate back files and advance the queue front if necessary.
        Parameters:
        timestamp - a timestamp
        Throws:
        IOException - exception thrown if there was any IO error during the removal operation

      • limitBackFileSize

        void limitBackFileSize​(long sizeLmit)
                throws IOException
        Limit the back file size of this queue, truncate back files and advance the queue front if necessary. Note, this is a best effort call, exact size limit can't be guaranteed
        Parameters:
        sizeLmit - size limit
        Throws:
        IOException - exception thrown if there was any IO error during the operation

      • getBackFileSize

        long getBackFileSize()
              throws IOException
        Current total size of the back files of this queue
        Returns:
        total back file size
        Throws:
        IOException - exception thrown if there was any IO error during the operation

      • findClosestIndex

        long findClosestIndex​(long timestamp)
               throws IOException
        Find an index closest to the specific timestamp when the corresponding item was enqueued. to find latest index, use LATEST as timestamp. to find earliest index, use EARLIEST as timestamp.
        Parameters:
        timestamp - when the corresponding item was appended
        Returns:
        an index
        Throws:
        IOException - exception thrown during the operation

      • resetQueueFrontIndex

        void resetQueueFrontIndex​(String fanoutId,
                          long index)
                   throws IOException
        Reset the front index of a fanout queue.
        Parameters:
        fanoutId - fanout identifier
        index - target index
        Throws:
        IOException - exception thrown during the operation

      • removeAll

        void removeAll()
        throws IOException
        Removes all items of a queue, this will empty the queue and delete all back data files.
        Throws:
        IOException - exception throws if there is any IO error during dequeue operation.

      • getFrontIndex

        long getFrontIndex()
        Get the queue front index, this is the earliest appended index
        Returns:
        an index

      • getFrontIndex

        long getFrontIndex​(String fanoutId)
            throws IOException
        Get front index of specific fanout queue
        Parameters:
        fanoutId - fanout identifier
        Returns:
        an index
        Throws:
        IOException - exception thrown if IO error occurs

      • getFrontIndex

        long getFrontIndex​(String fanoutId,
                   boolean useLatest)
            throws IOException
        Get front index of specific fanout queue
        Parameters:
        fanoutId - fanout identifier
        useLatest - if no offset has been recorded the head of the queue is used
        Returns:
        an index
        Throws:
        IOException - exception thrown if IO error occurs

      • getRearIndex

        long getRearIndex()
        Get the queue rear index, this is the next to be appended index
        Returns:
        an index