Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

我们为许可用户提供了部分翻译好的中文文档。您只需登录便可查阅这些文档

MapReduce

Programming technique for analyzing data sets that do not fit in memory

mapreduce is a programming technique which is suitable for analyzing large data sets that otherwise cannot fit in your computer’s memory. Using a datastore to process the data in small chunks, the technique is composed of a Map phase, which formats the data or performs a precursory calculation, and a Reduce phase, which aggregates all of the results from the Map phase. For more information, see Getting Started with MapReduce.

For information about using other products with mapreduce, see Speed Up and Deploy MapReduce Using Other Products.

Functions

mapreduceProgramming technique for analyzing data sets that do not fit in memory
datastoreCreate datastore for large collections of data
addAdd single key-value pair to KeyValueStore
addmultiAdd multiple key-value pairs to KeyValueStore
hasnextDetermine if ValueIterator has one or more values available
getnextGet next value from ValueIterator
mapreducerDefine execution environment for mapreduce or tall arrays
gcmrGet current mapreducer configuration

Using Objects

KeyValueStoreStore key-value pairs for use with mapreduce
ValueIteratorAn iterator over intermediate values for use with mapreduce

Topics

Getting Started with MapReduce

Learn about the MapReduce programming technique and run an example calculation.

Write a Map Function

Create a map function for use in a mapreduce algorithm.

Write a Reduce Function

Create a reduce function for use in a mapreduce algorithm.

Build Effective Algorithms with MapReduce

Summary of mapreduce example files.

Speed Up and Deploy MapReduce Using Other Products

Capabilities of other products to speed up and share mapreduce algorithms.

Find Maximum Value with MapReduce

This example shows how to find the maximum value of a single variable in a data set using mapreduce.

Compute Mean Value with MapReduce

This example shows how to compute the mean of a single variable in a data set using mapreduce.

Create Histograms Using MapReduce

This example shows how to visualize patterns in a large data set without having to load all of the observations into memory simultaneously.

Compute Mean by Group Using MapReduce

This example shows how to compute the mean by group in a data set using mapreduce.

Simple Data Subsetting Using MapReduce

This example shows how to extract a subset of a large data set.

Using MapReduce to Compute Covariance and Related Quantities

This example shows how to compute the mean and covariance for several variables in a large data set using mapreduce.

Compute Summary Statistics by Group Using MapReduce

This example shows how to compute summary statistics organized by group using mapreduce.

Using MapReduce to Fit a Logistic Regression Model

This example shows how to use mapreduce to carry out simple logistic regression using a single predictor.

Tall Skinny QR (TSQR) Matrix Factorization Using MapReduce

This example shows how to compute a tall skinny QR (TSQR) factorization using mapreduce.

Compute Maximum Average HSV of Images with MapReduce

This example shows how to use ImageDatastore and mapreduce to find images with maximum hue, saturation and brightness values in an image collection.

Troubleshooting

Debug MapReduce Algorithms

This example shows how to debug your mapreduce algorithms in MATLAB® using a simple example file, MaxMapReduceExample.m. Debugging enables you to follow the movement of data between the different phases of mapreduce execution and inspect the state of all intermediate variables.

Was this topic helpful?