[FieldTrip] Debian packaging of fieldtrip

Robert Oostenveld r.oostenveld at donders.ru.nl
Mon Dec 6 15:13:11 CET 2010


Dear Michael

As you may know, I am in charge of the fieldtrip project. Since you  
asked the questions on the email list, let me also reply through this  
channel. There are a number of techical issues that you raised, some  
of which have already been addressed by email lists subscribers. Let  
me just respond to some of the remaining points.

Regarding octave: It would of course be nice if it ran both on octave  
and matlab. However, the active developers at the moment do not have a  
need for octave as alternative for Matlab. At the donders we have  
enough matlab licenses and I believe this applies to the other  
developers as well. The consequence is that maintaining compatibility  
with octave is not high on the priority list of any of these  
developers. However, I certainly welcome suggestions for improving the  
octave compatibility. Suggestions for improvements can be posted on http://bugzilla.fcdonders.nl 
. If someone wants to take up this responsibility, that person could  
get direct access to the svn server. Please note that we have external  
people already contributing to the project source code and  
documentation, it is not only a Donders project.

Regarding test suite: there is the branches/testing directory in which  
you can find test code. It is not a complete suite, and most  
importantly is that the data files (which amount to many GBs are not  
in there. I am trying to encourage contributors to add test scripts  
there, but at the moment we do not have an automated testing  
procedure. We used to have one, but at a certain point it ran too long  
(more than one full night) and was not sufficiently sensitive in  
picking up the actual bugs, so that is why we abandoned the automatic  
(regression) testing. We now rely on our users to report problems and  
take quick action in fixing the bugs. The automatic testing remains a  
desireable feature, but currently we lack manpower and expertise for  
properly implementing this.

Regarding openmeeg: where do you in general envision the "glue code"  
between project A and project B to go? In this particular example, I  
think that the openmeeg glue code should go into the fieldtrip  
package. The reason for this is that it is maintained by the whole  
fieldtrip developer team, and hosted on the fieldtrip svn. Part of the  
glue is actually even in the fieldtrip/forward directory, so taking  
away fieldtrip/external/openmeeg might make it more confusing for  
users. Note that the openmeeg package in fieldtrip does automatic  
checking of the required binaries and provides instructions if the  
binaries are missing, so towards end-users it is friendlier to rely on  
that checking mechanism.

Regarding externals in general: the externals are more than just  
providing a version of some external software for easy download. The  
externals also provide a specific version (that is tested and known to  
work), a subset of the toolbox (e.g. part of spm2 instead of the  
complete version) and bug fixes if the external parter does not  
provide them in time (e.g. the isfinite function which fails on recent  
matlabs and therefore had to be replaced by the finite function on  
afni and spm2). SPM8 works with a particular version of fieldtrip, and  
the SPM developers are responsible for keeping their fieldtrip version  
in synch with our developments. Vice versa, the fieldtrip developers  
are responsible for keeping the fieldtrip/external. The challenge in  
package management on linux is of course also dependencies. With the  
matlab toolboxes we are joinly trying to work towards better  
dependency management, but are clearly not there yet. How do you  
envision that your debian packaging ensures correct versions to be co- 
installed?

Regarding mex files: not all users have compilers on their system by  
default (esp. on windows 64 bit and osx in general), therefore we  
provide the packaged fieldtrip version  including binaries. This also  
facilitates the fieldtrip release to be installed and maintained by a  
single user in a larger research group, e.g. on a shared network drive  
that is read only for all users except the maintainer. Recompiling by  
individual users requires individual users to have write access. Of  
course if you consider every user to be super user on his own linux  
box, that is not a problem. But in reality many researchers have  
limited rights to the computers on which they are working. Recompiling  
the mex files is of course always an option, but it would be  
cumbersome to force this on the users. Why would you want to exclude  
the mex binaries from the release version?

Regarding your observation of files being replicated: fieldtrip (as  
all matlab toolboxes) consists of interpreted code and not compield  
code. To ensure a consistent and long-term stable user interface to  
the functions, part of the functions are hidden (in the private  
directories). Those hidden functions are also hidden from each other,  
i.e. fieldtrip/fileio/private and fieldtrip/forward/private functions  
cannot access each other. Hence some of the low-level functions (and  
also mex files) have to be replicated. That is why we use svn file  
externals. Checking out so many svn file externals is very slow due to  
the suboptimal implementation of the svn client software. However, the  
regular users (95% of all users) don't notice and don't care, because  
they are just getting a packaged zip file, unzip it and can get to work.

Then some general questions: which part of the FieldTrip audience do  
you expect to benefit from the packaged version? I presume that it  
would be limited to the linux users, with root access to their own  
computers, and debian based releases. Actually I would expect those  
people to benefit more from the read-only SVN version of fieldtrip  
that we make available to all users. Working with SVN should be simple  
for this particular sub-class of fieldtrip users. And using svn allows  
the user to roll back to older versions, and to make changes in his  
copy of the code without those changes being overwritten with the next  
"svn update".

If you want to discuss this in more detail, feel free to drop me an  
email to schedule a phone or skype call. Probably that is more  
efficient for both of us in getting things done.

best regards,
Robert


-----------------------------------------------------------
Robert Oostenveld, PhD
Senior Researcher & MEG Physicist
Donders Institute for Brain, Cognition and Behaviour
Centre for Cognitive Neuroimaging
Radboud University Nijmegen
tel.: +31 (0)24 3619695
e-mail: r.oostenveld at donders.ru.nl
web: http://www.ru.nl/neuroimaging
skype: r.oostenveld
-----------------------------------------------------------






On 4 Dec 2010, at 14:27, Michael Hanke wrote:

> Hi,
>
> I'm one of the NeuroDebian (http://neuro.debian.net) developers and  
> I am
> looking into packaging fieldtrip for the Debian operating system.  
> I'd be
> glad if you could provide me with some information concerning the
> following questions:
>
> 1. I cloned the public SVN repository. There are no tags in it. Does
>   fieldtrip do actual releases, or is every revision intended to be
>   used by users?
>
> 2. The repository contains a large number of binaries for various
>   platforms. A Debian package cannot ship any of these, as all  
> binaries
>   have to be buildable from source. Therefore I stripped all MEX
>   extension and third party binaries. This raises the question of how
>   to get them rebuilt ;-)
>
>   There are no Makefiles, but there is ft_compile_mex.m. However, that
>   doesn't seem to build everything that is shipped with the fieldtrip
>   distribution.
>
> 3. My initial goal was simply to get enough of fieldtrip running to  
> be able to
>   serve as a dependency for our package of the Matlab version of SPM8.
>   However, I now discovered that there is a "port" of fieldtrip to
>   octave: http://kurage.nimh.nih.gov/meglab/Meg/Software which makes  
> it
>   mandatory for me to try to integrate it properly into Debian octave
>   infrastructure.
>
>   Are you aware of this octave port? Is there any plan to incorporate
>   this effort?  To what degree is it complete or usable? Or maybe
>   current fieldtrip code already runs (partially) on octave?
>
> 4. Distribution packaging can sometimes cause problems, because the
>   maintainer is not aware of some peculiarities. Do you have a test
>   suite that can be used to assess proper functioning of fieldtrip?
>
>   I saw the tutorial and associated data that could serve as a test
>   suite, but some tutorial parts only mention compatibility with
>   versions of the past, and I wasn't able to locate the actual  
> tutorial
>   _code_ for download.
>
>   I also saw http://fieldtrip.fcdonders.nl/development/infrastructure_for_testing
>   but it looks more like a plan for the future, correct?
>
> 5. I had to strip all 'external' software due to various
>   licensing/distribution problems. I wonder to what degree the  
> presence
>   of these pieces is critical for fieldtrip functionality -- beside
>   obvious lack of support for some file formats?
>
> Thanks in advance for your answers to this long list,
>
> Michael
>
> -- 
> GPG key: 4096R/7FFB9E9B Michael Hanke
> http://mih.voxindeserto.de
> _______________________________________________
> Fieldtrip mailing list
> Fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip









More information about the fieldtrip mailing list