Save Variables in parfor-Loops

Since R2024a

This example shows how to use the save function in a parfor-loop to save one or more variables to a file.

To use the save function in the body of a parfor-loop, you must use the "-fromstruct" option. If you do not use this option, MATLAB® produces an error due to a transparency violation.

You must also ensure that each worker or parfor iteration has unique access to any files it writes or saves data to. When multiple workers attempt to write to the same file, MATLAB can produce an error or corrupted data, or one worker might overwrite the data from another worker. These issues are more likely to occur when:

• There is more than one worker per machine, and the workers attempt to write to the same file.

• The workers have a shared file system and use the same path to identify a file for writing.

Start a parallel pool of process workers.

pool = parpool("Processes");

Starting parallel pool (parpool) using the 'Processes' profile ...
Connected to parallel pool with 6 workers.

Use a parfor-loop to generate data and save it to separate MAT-files in each iteration. Temporarily store the results variable in a scalar structure. Use the "-fromstruct" option of the save function to save the field and value from the structure to a file as a variable.

parfor idx = 1:pool.NumWorkers
    x = rand(1000*idx);
    s = struct("x",x);
    save(sprintf("output_%d.mat",idx),"-fromstruct",s);
end

By default, parpool sets the working folder on the workers to match that of the MATLAB client session. So, the client current working folder becomes the default folder where the workers save the files.

View the contents of one of the files using the whos function.

whos("-file","output_1.mat");

  Name         Size                Bytes  Class     Attributes

  x         1000x1000            8000000  double              

To save multiple variables, create a structure with multiple fields and then save the fields and values as individual variables.

parfor idx = 1:pool.NumWorkers
   x = rand(1000*idx);
   y = eye(idx);
   z = magic(idx);
   s = struct("x",x,"y",y,"z",z);
   save(sprintf("output_%d.mat",idx),"-fromstruct",s);
end

View the contents of one of the files.

whos("-file","output_2.mat");

  Name         Size                 Bytes  Class     Attributes

  x         2000x2000            32000000  double              
  y            2x2                     32  double              
  z            2x2                     32  double              

Delete the files and shut down the parallel pool.

delete output*
delete(pool);

Parallel pool using the 'Processes' profile is shutting down.