Prolog programming with a map-reduce parallel construct

Abstract

Map-Reduce is a programming model that has its roots in early functional programming. In addition to producing short and elegant code for problems involving lists or collections, this model has proven very useful for large-scale highly parallel data processing. In this work, we present the design and implementation of a high-level parallel construct that makes the Map-Reduce programming model available for Prolog programmers. To the best of our knowledge, there is no Map-Reduce framework native to Prolog, and so the aim of this work is to offer data processing features from which several applications can greatly benefit; the Inductive Logic Programming field, for instance, can take advantage of a Map-Reduce predicate when proving newly created rules against sets of examples. Our Map-Reduce model was comprehensively tested with different applications. Our experiments, using the Yap Prolog system, show that: (i) the model scales linearly up to 24 processors; (ii) a dynamic distributed scheduling strategy performs better than centralized or static scheduling strategies; and (iii) the performance varies significantly with the number of items being sent to each processor at a time. Overall, our Map-Reduce framework presents as a good alternative for both taking advantage of the currently available low cost multi-core architectures and developing scalable data processing applications, native to the Prolog programming language. © 2013 ACM.