[FieldTrip] Debian packaging of fieldtrip
Robert Oostenveld
r.oostenveld at donders.ru.nl
Mon Dec 6 15:13:11 CET 2010
Dear Michael
As you may know, I am in charge of the fieldtrip project. Since you
asked the questions on the email list, let me also reply through this
channel. There are a number of techical issues that you raised, some
of which have already been addressed by email lists subscribers. Let
me just respond to some of the remaining points.
Regarding octave: It would of course be nice if it ran both on octave
and matlab. However, the active developers at the moment do not have a
need for octave as alternative for Matlab. At the donders we have
enough matlab licenses and I believe this applies to the other
developers as well. The consequence is that maintaining compatibility
with octave is not high on the priority list of any of these
developers. However, I certainly welcome suggestions for improving the
octave compatibility. Suggestions for improvements can be posted on http://bugzilla.fcdonders.nl
. If someone wants to take up this responsibility, that person could
get direct access to the svn server. Please note that we have external
people already contributing to the project source code and
documentation, it is not only a Donders project.
Regarding test suite: there is the branches/testing directory in which
you can find test code. It is not a complete suite, and most
importantly is that the data files (which amount to many GBs are not
in there. I am trying to encourage contributors to add test scripts
there, but at the moment we do not have an automated testing
procedure. We used to have one, but at a certain point it ran too long
(more than one full night) and was not sufficiently sensitive in
picking up the actual bugs, so that is why we abandoned the automatic
(regression) testing. We now rely on our users to report problems and
take quick action in fixing the bugs. The automatic testing remains a
desireable feature, but currently we lack manpower and expertise for
properly implementing this.
Regarding openmeeg: where do you in general envision the "glue code"
between project A and project B to go? In this particular example, I
think that the openmeeg glue code should go into the fieldtrip
package. The reason for this is that it is maintained by the whole
fieldtrip developer team, and hosted on the fieldtrip svn. Part of the
glue is actually even in the fieldtrip/forward directory, so taking
away fieldtrip/external/openmeeg might make it more confusing for
users. Note that the openmeeg package in fieldtrip does automatic
checking of the required binaries and provides instructions if the
binaries are missing, so towards end-users it is friendlier to rely on
that checking mechanism.
Regarding externals in general: the externals are more than just
providing a version of some external software for easy download. The
externals also provide a specific version (that is tested and known to
work), a subset of the toolbox (e.g. part of spm2 instead of the
complete version) and bug fixes if the external parter does not
provide them in time (e.g. the isfinite function which fails on recent
matlabs and therefore had to be replaced by the finite function on
afni and spm2). SPM8 works with a particular version of fieldtrip, and
the SPM developers are responsible for keeping their fieldtrip version
in synch with our developments. Vice versa, the fieldtrip developers
are responsible for keeping the fieldtrip/external. The challenge in
package management on linux is of course also dependencies. With the
matlab toolboxes we are joinly trying to work towards better
dependency management, but are clearly not there yet. How do you
envision that your debian packaging ensures correct versions to be co-
installed?
Regarding mex files: not all users have compilers on their system by
default (esp. on windows 64 bit and osx in general), therefore we
provide the packaged fieldtrip version including binaries. This also
facilitates the fieldtrip release to be installed and maintained by a
single user in a larger research group, e.g. on a shared network drive
that is read only for all users except the maintainer. Recompiling by
individual users requires individual users to have write access. Of
course if you consider every user to be super user on his own linux
box, that is not a problem. But in reality many researchers have
limited rights to the computers on which they are working. Recompiling
the mex files is of course always an option, but it would be
cumbersome to force this on the users. Why would you want to exclude
the mex binaries from the release version?
Regarding your observation of files being replicated: fieldtrip (as
all matlab toolboxes) consists of interpreted code and not compield
code. To ensure a consistent and long-term stable user interface to
the functions, part of the functions are hidden (in the private
directories). Those hidden functions are also hidden from each other,
i.e. fieldtrip/fileio/private and fieldtrip/forward/private functions
cannot access each other. Hence some of the low-level functions (and
also mex files) have to be replicated. That is why we use svn file
externals. Checking out so many svn file externals is very slow due to
the suboptimal implementation of the svn client software. However, the
regular users (95% of all users) don't notice and don't care, because
they are just getting a packaged zip file, unzip it and can get to work.
Then some general questions: which part of the FieldTrip audience do
you expect to benefit from the packaged version? I presume that it
would be limited to the linux users, with root access to their own
computers, and debian based releases. Actually I would expect those
people to benefit more from the read-only SVN version of fieldtrip
that we make available to all users. Working with SVN should be simple
for this particular sub-class of fieldtrip users. And using svn allows
the user to roll back to older versions, and to make changes in his
copy of the code without those changes being overwritten with the next
"svn update".
If you want to discuss this in more detail, feel free to drop me an
email to schedule a phone or skype call. Probably that is more
efficient for both of us in getting things done.
best regards,
Robert
-----------------------------------------------------------
Robert Oostenveld, PhD
Senior Researcher & MEG Physicist
Donders Institute for Brain, Cognition and Behaviour
Centre for Cognitive Neuroimaging
Radboud University Nijmegen
tel.: +31 (0)24 3619695
e-mail: r.oostenveld at donders.ru.nl
web: http://www.ru.nl/neuroimaging
skype: r.oostenveld
-----------------------------------------------------------
On 4 Dec 2010, at 14:27, Michael Hanke wrote:
> Hi,
>
> I'm one of the NeuroDebian (http://neuro.debian.net) developers and
> I am
> looking into packaging fieldtrip for the Debian operating system.
> I'd be
> glad if you could provide me with some information concerning the
> following questions:
>
> 1. I cloned the public SVN repository. There are no tags in it. Does
> fieldtrip do actual releases, or is every revision intended to be
> used by users?
>
> 2. The repository contains a large number of binaries for various
> platforms. A Debian package cannot ship any of these, as all
> binaries
> have to be buildable from source. Therefore I stripped all MEX
> extension and third party binaries. This raises the question of how
> to get them rebuilt ;-)
>
> There are no Makefiles, but there is ft_compile_mex.m. However, that
> doesn't seem to build everything that is shipped with the fieldtrip
> distribution.
>
> 3. My initial goal was simply to get enough of fieldtrip running to
> be able to
> serve as a dependency for our package of the Matlab version of SPM8.
> However, I now discovered that there is a "port" of fieldtrip to
> octave: http://kurage.nimh.nih.gov/meglab/Meg/Software which makes
> it
> mandatory for me to try to integrate it properly into Debian octave
> infrastructure.
>
> Are you aware of this octave port? Is there any plan to incorporate
> this effort? To what degree is it complete or usable? Or maybe
> current fieldtrip code already runs (partially) on octave?
>
> 4. Distribution packaging can sometimes cause problems, because the
> maintainer is not aware of some peculiarities. Do you have a test
> suite that can be used to assess proper functioning of fieldtrip?
>
> I saw the tutorial and associated data that could serve as a test
> suite, but some tutorial parts only mention compatibility with
> versions of the past, and I wasn't able to locate the actual
> tutorial
> _code_ for download.
>
> I also saw http://fieldtrip.fcdonders.nl/development/infrastructure_for_testing
> but it looks more like a plan for the future, correct?
>
> 5. I had to strip all 'external' software due to various
> licensing/distribution problems. I wonder to what degree the
> presence
> of these pieces is critical for fieldtrip functionality -- beside
> obvious lack of support for some file formats?
>
> Thanks in advance for your answers to this long list,
>
> Michael
>
> --
> GPG key: 4096R/7FFB9E9B Michael Hanke
> http://mih.voxindeserto.de
> _______________________________________________
> Fieldtrip mailing list
> Fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip
More information about the fieldtrip
mailing list