[FieldTrip] Run ft_prepare_vol_sens on multiple cores

Frans Nord fnorden at gmail.com
Tue Apr 19 14:02:21 CEST 2022

Thank you John, that helped a lot!

Using a parfor in the sb_transfer function seems to provide the main
reduction in computation time as far as I can see. Have you looked into
adding a parfor in the for loops calling ft_compute_leadfield as well?
Seems like it could possibly reduce the computation time even further.

Best regards

On Thu, Apr 14, 2022 at 6:56 PM Richards, John via fieldtrip <
fieldtrip at science.ru.nl> wrote:

> Frans.
> I was able to do this.  There are two places where there is an iterative
> part of the ft_prepare_leadfield.  I think one of these is the main
> program, and another is the sb_transfer.  I was able to recode that section
> and use parfor for the parallel processing.  We do 3mm hex source volumes
> for adults, or dense tetra source/head models, both which have something
> like 100K source voxels; and 2mm hex source models for infants, or dense
> tetra source/head models.  We run this on 48-core machines and it takes 90
> min or so to do the LF.  I used to do these with the single-threaded matlab
> and it took something like 18 hrs.
> I don't remember offhand exactly which sections I did this in.  I have
> customized versions for the files below.  I would be glad to give you a
> further description if you are interested.
> John
> ./JERLab_ft_prepare_leadfield.m
> ./external/simbio/JERLab_sb_transfer.m
> ./forward/JERLab_ft_prepare_vol_sens_notransfer.m
> ./forward/JERLab_ft_prepare_vol_sens.m
> ./forward/private/JERLab_leadfield_simbio.m
> Also in sb_solve I commented out the disp because they were distracting on
> the matlab output, but I don't think I changed this module.
> It takes a few steps to upgrade to a new fieldtrip version because I have
> these extra programs.  The last time I upgraded was june 2021 to the
> fieldtrip-20210330 version.  These are the notes I used to upgrade, so they
> might be a good hint where to look for parallelization with MATLAB.
> ERLab_ft_prepare_leadfield
>   add computeparallel to the cfg file
>   will do the parallel, OR, use the original code if computeparallel is
> not set
> private/prepare_headmodel
>  adds parfor based on computeparallel in the cfg
> JERLab_ft_prepare_vol_sens, substitute JERLab_sb_transfer
> external/simbio/JERLab_sb_transfer
>   adds the parfor for the parallel pool
> external/simbio/sb_solve
>   removed the disp's, so don't print during the parallel
>   clears are ok in subroutines, but not in external parfor loop
> This used to be ft_prepare_vol_sens_notransfer, but is a JERLab program,
> so rename.
> it is used in private/prepare_headmodel
> x forward/JERLab_ft_prepare_vol_sens_notransfer.m
> ***********************************************
> John E. Richards
> Carolina Distinguished Professor
> Department of Psychology
> University of South Carolina
> Columbia, SC  29208
> Dept Phone: 803 777 2079
> Fax: 803 777 9558
> Email: richards-john at sc.edu
> https://jerlab.sc.edu
> *************************************************
> ----------------------------------------------------------------------
> Message: 1
> Date: Wed, 13 Apr 2022 10:48:41 +0000
> From: "Schoffelen, J.M. (Jan Mathijs)"
>         <janmathijs.schoffelen at donders.ru.nl>
> To: FieldTrip discussion list <fieldtrip at science.ru.nl>
> Subject: Re: [FieldTrip] Run ft_prepare_vol_sens on multiple cores
> Message-ID: <34E8671E-BDFF-40B0-A8FC-51B6DE832330 at donders.ru.nl>
> Content-Type: text/plain; charset="utf-8"
> Hi Frans,
> Perhaps the simbio folks who are reading along can comment on this?
> Best wishes,
> Jan-Mathijs
> On 12 Apr 2022, at 17:51, Frans Nord via fieldtrip <
> fieldtrip at science.ru.nl<mailto:fieldtrip at science.ru.nl>> wrote:
> Hello all,
> In the article "The FieldTrip-SimBio pipeline for EEG forward modelling" (
> https://pubmed.ncbi.nlm.nih.gov/29580236/<
> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-31327df5-454455534531-2af725325d4aef4b&q=1&e=57cfd234-1f5c-425d-a911-de4097b63ba0&u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F29580236%2F__%3B%21%21HJOPV4FYYWzcc1jazlU%214B2vOR7eEquYu23rZZqYB7aXimMxJhGdqndb2U4xGpnQc91i_jchiCZIl946JbymXnDbqWfKxg16WtwFDXZTXNI-VKqad34BCpR5Yg%24>)
> it is stated that they were able to reduce the computational time
> substantially when calling ft_prepare_vol_sens and ft_calculate_leadfield
> through running it on all 16 cores.
> Looking at the code I would assume that it's the call to sb_transfer that
> takes a long time to compute but I don't know how to attempt any
> parallelizing there. Anyone who has any idea on how to do that?
> Best regards and thank you for developing a great toolbox!
> //Frans
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-31327df5-454455534531-6da7080247d95362&q=1&e=57cfd234-1f5c-425d-a911-de4097b63ba0&u=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pcbi.1002202__%3B%21%21HJOPV4FYYWzcc1jazlU%214B2vOR7eEquYu23rZZqYB7aXimMxJhGdqndb2U4xGpnQc91i_jchiCZIl946JbymXnDbqWfKxg16WtwFDXZTXNI-VKqad34U9MFCNg%24
> _______________________________________________
> fieldtrip mailing list
> https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
> https://doi.org/10.1371/journal.pcbi.1002202
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20220419/3701c05d/attachment.htm>

More information about the fieldtrip mailing list