This project has retired. For details please refer to its
Attic page
Apache Crunch - Apache Crunch
Apache
Crunch
Apache Crunch
Overview
Getting Started
User Guide
API Documentation
Development
Source Code
Mailing Lists
Issue Tracking
Wiki
Project
About
Bylaws
License
Apache Crunch
Simple and Efficient MapReduce Pipelines
The
Apache Crunch
Java library provides a framework for writing, testing,
and running MapReduce pipelines. Its goal is to make pipelines that are
composed of many user-defined functions simple to write, easy to test, and
efficient to run.
Apache Crunch moved to the Apache Attic in July of 2020 and is no longer
actively developed.
Running on top of
Hadoop
MapReduce and
Apache Spark
, the Apache
Crunch
library is a simple Java API for tasks like joining and data aggregation
that are tedious to implement on plain MapReduce. The APIs are especially useful when
processing data that does not fit naturally into relational model, such as time series,
serialized object formats like protocol buffers or Avro records, and HBase rows and columns.
For Scala users, there is the Scrunch API, which is built on top of the Java APIs and
includes a REPL (read-eval-print loop) for creating MapReduce pipelines.