Collaborators
Brown University
Portland State
Carnegie Mellon

About

Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. In the past, these two modes of operation were found only in separate, stove-piped systems. However, with the creation of NewSQL OLTP systems, it becomes possible to perform scalable real-time operations without sacrificing transactional support. Enter S-Store, the world’s first transactional streaming database system.

Architecture

Architecture

Architecture

S-Store is built on H-Store, a distributed main-memory OLTP system, taking advantage of its ability to process transactions with low latency and high throughput. We add support for push-based processing, including two layers of triggering mechanisms. We also introduce native data structures for streams and windows.

Transaction Model

Transaction Model

Transaction Model

S-Store models its streaming workflow as DAGs of streaming transactions. A streaming transaction is considered to be a stored procedure (SP) operating on an input batch of tuples. Due to data and processing dependencies, transaction executions (TEs) cannot be performed in an arbitrary order, and additional isolation rules are necessary to preserve ACID guarantees.
Transaction Model

Applications

Applications

Applications

S-Store is designed to meet the needs of a number of transactional applications driven by high-velocity streaming inputs. As a demonstration, we have implemented a system that models a bicycle share program, known as BikeShare. S-Store is also applicable to systems such as online gaming, real-time marketing, Internet of Things, voting leaderboards, and real-time ETL frontends.

Publications

Data Ingestion for the Connected World
John Meehan, Cansu Aslantas, Jiang Du, Nesime Tatbul, Stan Zdonik
CIDR 2017, Jan 2017

Integrating Real-Time and Batch Processing in a Polystore
John Meehan, Stan Zdonik, Shaobo Tian, Yulong Tian, Nesime Tatbul, Adam Dziedzic, Aaron Elmore
IEEE-HPEC 2016, Sept 2016

Handling Shared, Mutable State in Stream Processing with Correctness Guarantees
Nesime Tatbul, Stan Zdonik, John Meehan, Cansu Aslantas, Michael Stonebraker, Kristin Tufte, Chris Giossi, Hong Quach
IEEE Data Engineering Bulletin, Special Issue on Next-Generation Stream Processing, 38(4), Dec 2015

S-Store: Streaming Meets Transaction Processing
John Meehan, Nesime Tatbul, Stan Zdonik, Cansu Aslantas, Ugur Cetintemel, Jiang Du, Tim Kraska, Samuel Madden, David Maier, Andrew Pavlo, Michael Stonebraker, Kristin Tufte, Hao Wang
PVLDB 8(13), Sept 2015

S-Store: A Streaming NewSQL System for Big Velocity Applications (Demo)
Ugur Cetintemel, Jiang Du, Tim Kraska, Samuel Madden, David Maier, John Meehan, Andrew Pavlo, Michael Stonebraker, Erik Sutherland, Nesime Tatbul, Kristin Tufte, Hao Wang, Stanley Zdonik
VLDB 2014 (Poster)

S-Store: A Big-Velocity Database System
John Meehan, ISTC Blog Post, Dec 2014

S-Store: Real-Time Analytics Meets Transaction Processing
Nesime Tatbul, ISTC Blog Post, Feb 2014