Several real-time decision making applications rely on the computation of quantitative summaries of very large streams of data. StreamQRE is a declarative query language that simplifies the task of programming the desired logic. It combines regular expressions, quantitative aggregation, and relational features. A compilation algorithm translates the high-level query into a streaming algorithm with precise guarantees for resource usage. StreamQRE has been implemented as a Java library, and its performance has been compared to other popular streaming engines (RxJava, Esper and Flink).
The following diagram shows the performance of StreamQRE, RxJava, Esper and Flink on an adaptation of the Yahoo Streaming Benchmark. The vertical axis shows the throughput in millions of tuples per second. The experiments were run on a typical contemporary laptop, with an Intel Core i7-4710HQ CPU running at 2.5 GHz and with 12 GB of RAM.
|StreamQRE: Modular Specification and Efficient Evaluation of Quantitative Queries over Streaming Data.
Konstantinos Mamouras, Mukund Raghothaman, Rajeev Alur, Zachary G. Ives, and Sanjeev Khanna.
Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). Barcelona, Spain, June 2017.
|Automata-based Stream Processing.
Rajeev Alur, Konstantinos Mamouras, and Caleb Stanford.
Proceedings of the 44th International Colloquium on Automata, Languages, and Programming (ICALP). Warsaw, Poland, July 2017.
|Derivatives of Quantitative Regular Expressions.
Rajeev Alur, Konstantinos Mamouras, and Dogan Ulus.
In Models, Algorithms, Logics and Tools, Lecture Notes in Computer Science, Springer, 2017.
|An Introduction to the StreamQRE Language.
Rajeev Alur and Konstantinos Mamouras.
Marktoberdorf Summer School, 2017.
// Processes a single data item.
QRe<Double, Double> item = Q.atomic(x -> true, x -> x);
// Computes the running sum.
QRe<Double, Double> sum = Q.iter(item, 0.0, (x,y) -> x+y);
// Computes the running count.
QRe<Double, Long> count = Q.iter(item, 0L, (x,y) -> x+1);
// Computes the running average.
QRe<Double, Double> avg = Q.combine(sum, count, (x,y) -> x/y);