A buffer that is designed for the sole purpose of buffering UnsafeRows in PythonForeachWriter. It is designed to be used with only two threads: the QueryExecutionThread which writes data to the buffer and WriterThread thread that reads from the buffer and writes to the Python worker stdin. Adds to the buffer are non-blocking, and reads through the buffer's iterator are blocking, that is, it blocks until new data is available or all data has been added.

Internally, it uses a HybridRowQueue to buffer the rows in a practically unlimited queue across memory and local disk. However, HybridRowQueue is designed to be used only with EvalPythonExec where the buffer's consumer is always behind the buffer's populator, that is, the WriterThread does not try to read n + 1 rows if the streaming thread has only written n rows at any point of time. This assumption is not true for PythonForeachWriter where rows may be added at a different rate as they are consumed by the Python worker. Hence, to maintain the invariant of the reader being behind the writer while using HybridRowQueue, the buffer does the following: - Keeps a count of the rows in the HybridRowQueue - Blocks the buffer's consuming iterator when the count is 0 so that the reader does not try to read more rows than what has been written.

The implementation of the blocking iterator (ReentrantLock, Condition, etc.) has been borrowed from that of ArrayBlockingQueue.