Project objectives

Persistent or non-volatile memory technologies are emerging, such as ReRAM and 3D XPoint. Servers equipped with non-volatile main memory will not loose memory content if they crash or are switched off. As computer architectures equipped with NVM become mainstream, there will be an increasing demand on revising software for data management, so that it can fully utilize the new hardware. In particular, IT industry will focus on developing a new generation of robust and highly available in-memory data store systems which both scale and are durable. This opportunity opens new avenues of research that we are exploring—on distributed synchronization, replication, and recovery algorithms for such systems.
	Contrary to SQL database systems that strive to ensure strong consistency properties but have mediocre performance, the existing in-memory NoSQL data store systems are scalable but ensure only weak consistency properties, so they are not suitable for all type of applications. They admit certain undesirable anomalies, such as lost updates, inconsistent state, out of order request processing, and split-brain syndrome. They are also unable to take full advantage of NVM, as their internal architectures are predicated on the assumption that memory is volatile.
The goal of the Persistent Datastore project is to develop cutting-edge solutions for a new generation of highly-available data store systems that ensure strong consistency and durability like traditional SQL database systems, but are efficient and scalable as the state-of-the-art NoSQL systems. As a proof of concept, we develop PersistentDS™—a toolkit of datastore protocols and replicated data types for computer systems equipped with persistent memory.

Project tasks

	Distributed synchronization
	We develop methods and algorithms that can ensure strong consistency. E.g., we propose Executive Paxos—a distributed consensus protocol which gives strong guarantees on the relative order of delivery of broadcast messages and, at the same time, allows for multiple concurrent instances as in Paxos—the most popular protocol of this sort.

	NVM-enabled replication
	We develop methods and algorithms for software-based fault-tolerant replication aimed at NVM-enabled in-memory replicated data stores. They are designed to increase system scalability and functionality without loosing strong consistency. The key idea is to explore data and command partitioning and other techniques.

	Acute Cloud Types
	We design Acute Cloud Types—a set of new conflict-free replicated data structures and types. While ensuring scalable performance, ACTs will provide new abstractions which combine the advantages of eventual consistency (efficiency and scalability) with the advantages of strong consistency (flexibility and ease of use).

	NVM-enabled recovery
	We design and evaluate recovery algorithms for the emerging NVM-enabled replicated data store systems, in particular those based on Paxos. The algorithms can provide efficient system recovery after failures, such as node crashes or network partitions, assuming no Byzantine errors.

Team

Principal investigator
	Paweł T. Wojciechowski, Associate Professor Paweł received the Habilitation degree from Poznan University of Technology (PUT) and the PhD degree in computer science from the University of Cambridge. He was a postdoctoral researcher with the School of Computer and Communication Sciences, École polytechnique fédérale de Lausanne (EPFL) and with the University of Cambridge. He is currently an associate professor with the Institute of Computing Science, PUT. His research interests span topics in concurrency, distributed computing, and programming languages.

Postdocs
	Tadeusz Kobus, Assistant Professor (part-time) Tadeusz obtained his Ph.D. in computer science from Poznan University of Technology in 2017, where he is currently an Assistant Professor (part-time) in the Institute of Computing Science. In the past, he was an intern at IBM T. J. Watson Research Center and worked on research projects for Egnyte. His research interests include fault tolerant distributed algorithms, transactional memory, and group communication systems.
	Maciej Kokociński, Assistant Professor (part-time) Maciej obtained his Ph.D. in computer science from Poznan University of Technology in 2022. He is currently a Research Assistant (part-time) in the Institute of Computing Science, Poznan University of Technology, Poland, where he also received BS and MS degrees in Computer Science, in 2009 and 2010 respectively. In the past, he worked on research projects for Egnyte. His current research interests include the theory and practice of distributed systems and transactional memory.
	Jan Kończak, Teaching Assistant Professor Jan obtained his Ph.D. in computer science from Poznan University of Technology in 2022. He is currently a Teaching Assistant Professor in the Institute of Computing Science, Poznań University of Technology, Poland, where he also received BS and MS degrees in Computer Science, in 2011 and 2012 respectively. His research interests include fault tolerant distributed algorithms, transactional memory, state machine replication and group communication systems.

Foreign collaborator
	Fernando Pedone, Full Professor Prof. Pedone received the PhD degree in computer science from the École polytechnique fédérale de Lausanne (EPFL). He was a visiting professor at Cornell University, a postdoctoral researcher at EPFL and Hewlett-Packard Laboratories (HP Labs). He is a full professor in the Faculty of Informatics at the University of Lugano (USI). His research interests include the theory and practice of dependable distributed systems and distributed data management systems.

All research work was conducted at Poznań University of Technology. We also supervised a few students who defended BS or MS diploma theses that were related to the project.

Recent News

Date	News
2024/12/3-4	Paweł gave a talk, titled KDB: A Persistent Key-Value Data Store with Batch Updates and Snapshots, and presented, with Maciej, two posters, titled KDB: A Persistent Key-Value Data Store with Batch Updates and Snapshots and Creek: A Mixed-Consistency Replication Scheme, at Middleware ‘24: the 25th International Middleware Conference: Demos, Posters and Doctoral Symposium (Hong Kong, China), the best conference on middleware systems (CORE A, MEiN 140). [Poster session]
2024/06/17	Paweł gave a talk, titled Jiffy: A Lock-free Skip List with Batch Updates and Snapshots and presented a poster based on the talk at the Highlights of Parallel Computing workshop, held in conjunction with the SPAA ’24: 36th ACM Symposium on Parallelism in Algorithms and Architectures (Nantes, France), one of the top conferences on parallel algorithms (CORE A, MEiN 140).
2024/05/14	Paweł gave an invited seminar talk, titled A selective and biased choice of techniques for building a distributed data store in the Department of Computer Science and Technology, the University of Cambridge, held as part of the Logic and Semantics Seminar (Computer Laboratory) series. Here is the abstract of the talk.
2023/09/22	Hubert Cwajda defended a MS thesis, titled Comparison of concurrent index data structures using different concurrency control mechanisms, whose supervisor was Maciej.
2023/06/30	The TEAM project has officially come to an end. However, there are still unfinished research tasks and papers on which we intend to continue working.
2023/06/27	Paweł gave an invited keynote speech, titled A selective and biased choice of techniques for building a distributed data store, at the 17th ACM International Conference on Distributed and Event-Based Systems (DEBS ‘23), a premier venue for academia and industry to discuss cutting-edge research of event-based computing related to Big Data, AI/ML, IoT, and Distributed Systems (27–30 June 2023, Neuchâtel, Switzerland). Here is the abstract of the keynote.
2023/06/22	Paweł visited the Department of Computer Science at ETH Zurich (Swiss Federal Institute of Technology in Zürich), invited by prof. Ana Klimović who leads the Efficient Architectures and Systems Lab (part of Systems Group). During a meeting we talked about our research interests and current work.
2023/06/21	Paweł gave an invited talk, titled A selective and biased choice of techniques for building a distributed data store, at Google Zürich headquarters. The visit was also a good opportunity to discuss the results of our Project with people in the IT industry during a number of impromptu meetings with Google engineers (21-23 June 2023, Zürich, Switzerland).
2023/05/23	We presented the results of the Project during a poster session at the 1st Science Day of Poznan University of Technology. As part of the event, presentations were made by individual Faculties, Institutes, Departments, EUNICE and units related to the scientific activity of PUT.
2023/05/16	Paweł gave a talk summarizing the results of the Project during the Conference Seminars, a one-day event organized by our institute to promote research work done for the last year by research groups forming our institute.
2023/04/21	Our paper, titled On the correctness of highly available systems in the presence of failures, has been accepted for publication in the Elsevier Journal of Parallel and Distributed Computing (JPDC) (CORE A, MEiN 100).
2023/03/28	The Discipline Council at our faculty voted to award Maciej and Jan’s PhDs a distinction. Congratulations!
2023/02/07	A remote meeting was held, which was a continuation of Paweł’s and Jan’s talks at the one-day workshop in the headquarters of Intel Poland in Gdańsk. At this meeting, we answered a number of questions that arose after listening to our talks and presented our plans for the future. In addition, we agreed to extend the server rental we received from Intel during the global pandemic until February 28, 2028.
2023/01/31	Unfortunately, Intel officially cancelled the Intel® Optane™ Persistent Memory. This decision was first revealed in mid-2022 and was a surprise to us. Luckily, our research results to date are largely orthogonal to a concrete technology. However, our prototype implementations that use Intel Optane DC Persistent Memory chips have been affected by Intel’s decision. Currently, there is no alternative product and it is unclear if there will ever be. However, a new kind of storage devices will appear that use persistent memory within the CXL (Compute Express Link), a new open standard for high-speed, high capacity central processing unit (CPU)-to-device and CPU-to-memory connections, designed for high performance data center computers. Intel announced that they are committed to continue to support development of CXL on Intel platforms as they believe it will be the future and standard of tiered-memory solutions.
2023/01/09	Paweł and Jan gave two invited talks during a one-day workshop organized by Intel Poland in its headquarters in Gdańsk. They presented in detail the recent results of the Project. The workshop was a good opportunity to learn about various projects at Intel, discuss avenues for any future collaboration, meet people from different R&D groups, including the group that developed PMDK (a library that we used), and thank again Intel for lending us servers that are equipped with persistent memory.
	History

Funding

The project is financed within the TEAM programme of the Foundation for Polish Science co-financed by the European Union under the European Regional Development Fund.

Contact us

If you have any questions about the Persistent Datastore project, please don’t hesitate to contact us.

Institute of Computing Science, Poznań University of Technology, Piotrowo 2, Poznań 60-965, Poland
mailto: Pawel . T . Wojciechowski @ put.edu.pl
phone: +(48) 61 665 3021