[FieldTrip] Question using fieldtrip on a HPC cluster

Mon Feb 28 12:46:12 CET 2022

Hi April,

We have recently released automaticanalysis 5.8.0 <https://zenodo.org/record/6047630> , a MATLAB-based pipeline for neuroimaging including xMRI and M/EEG.

I support the compilation and execution of workflows on various scheduled clusters, including SoGE, SLURM, Torque, and LSF. 

Please, let me know if you would like to learn more about it. 

Kind regards,

Tibor 

Auer, Tibor M.D. Ph.D.

Research Fellow

School of Psychology, Faculty of Health and Medical Sciences

University of Surrey, Guildford GU2 7XH

 <mailto:T.Auer at surrey.ac.uk> T.Auer at surrey.ac.uk

 <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FTiborAuer&data=02%7C01%7Ct.auer%40surrey.ac.uk%7Cdb32da458c424eedef2908d7d4bd1421%7C6b902693107440aa9e21d89446a2ebb5%7C0%7C0%7C637211780868086968&sdata=zrT5%2FnGGsar14C3WartuU99tzsfLu30Peh9fuaqrAUg%3D&reserved=0> @TiborAuer

From: fieldtrip <fieldtrip-bounces at science.ru.nl> On Behalf Of Schoffelen, J.M. (Jan Mathijs) via fieldtrip
Sent: Monday, February 14, 2022 7:47 AM
To: FieldTrip discussion list <fieldtrip at science.ru.nl>
Cc: Schoffelen, J.M. (Jan Mathijs) <janmathijs.schoffelen at donders.ru.nl>
Subject: Re: [FieldTrip] Question using fieldtrip on a HPC cluster

Hi April, 

I am privy to the details of the job management on your HPC cluster, so I can only guess a bit about what’s going on on your end.

My frame if reference is our local compute cluster. When I send off a matlab job to the cluster, this consists of an instruction to the job scheduler to start a (non-interactive) matlab session on one of the compute nodes, followed by the execution of a few matlab commands, the most relevant of which obviously is the analysis script/function that you want to have executed. Naively, the compute node on which the matlab session is started, does not necessarily ‘know’ about the path settings that the user may assume to have been specified. In other words, you may want to investigate whether this is the case for you. 

Again, not inhibited by any knowledge on how you interact with the HPC-cluster, I think you might need to add some addpath(<path-to-fieldtrip>), followed by ft_defaults to you matlab m-file. Also, I would check with your local IT-support to get some idea how the job deployment works at your place. It could even be that the functions in fieldtrip/qsub might be a way forward for you. Please have a look at https://www.fieldtriptoolbox.org/faq/what_are_the_different_approaches_i_can_take_for_distributed_computing/ , https://www.fieldtriptoolbox.org/faq/how_to_get_started_with_distributed_computing_using_qsub/ and links therein

Best wishes,

Jan-Mathijs

On 12 Feb 2022, at 01:03, April Luo via fieldtrip <fieldtrip at science.ru.nl <mailto:fieldtrip at science.ru.nl> > wrote:

Hello, 

I'm trying to run fieldtrip package on sEEG data on a HPC cluster, but when I was running ft_preprocessing function, it gave an error for ft_preamble init, saying 'Could not run %s - does not seem to exist, full_cmd,' I checked further and it seems like it's looking for ft_preamble_init.m file, I checked to make sure it's under /utilities/private, but it still gave this error. Running on a local matlab works. but I wanted to run on a HPC cluster because of the large datafile. Could you help me solve this issue? Thank you so much!

My matlab version on both my local computer and HPC cluster is 2021a and my fieldtrip version is 20210501.

Best,

April Luo

_______________________________________________
fieldtrip mailing list
https://mailman.science.ru.nl/mailman/listinfo/fieldtrip
https://urldefense.com/v3/__https://doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-p_UjC2cVtPAz1G7aUnveOAZ9nB1AC2hVt4uXhzmBZlumXV7jJKoLY0D5_dSOkzr-zZH3l5P7OGKqbKeMNVTD0OHySj0jBuD_D2ehA$ <https://urldefense.com/v3/__https:/doi.org/10.1371/journal.pcbi.1002202__;!!HJOPV4FYYWzcc1jazlU!-p_UjC2cVtPAz1G7aUnveOAZ9nB1AC2hVt4uXhzmBZlumXV7jJKoLY0D5_dSOkzr-zZH3l5P7OGKqbKeMNVTD0OHySj0jBuD_D2ehA$>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.ru.nl/pipermail/fieldtrip/attachments/20220228/81e3fa3e/attachment.htm>