Supported Engines & Query Dialects
|Name||Technology||Query Dialect||Engine Repository||Image||Notes|
||Apache Spark||Spark Streaming SQL||GitHub||Docker Hub||Also used in SQL shell. Currently the only engine that supports GIS data.|
||Apache Flink||Flink Streaming SQL||GitHub||Docker Hub||Has best support for stream-to-table joins|
||DataFusion||DataFusion Batch SQL||-||-||Experimental engine used in
* There is currently no way to express nested and GIS data types when declaring root dataset schemas, but you still can use them through pre-processing queries
** Apache Flink has known issues with Decimal type and currently relies on our patches that have not been upstreamed yet, so stability is not guaranteed FLINK-17804.
*** Arrow is lacking Decimal compute support ARROW-RS-272
|Projection / Temporal Table Joins||❌*||✔️|
* Spark Engine is capable of stream processing but temporarily we have to use it in the batch processing mode, so only row-level operations like map and filter are currently usable, as those do not require corrent stream processing and watermarking semantics.