[FieldTrip] Debian packaging of fieldtrip

[Michael Hanke]

Hi Robert,

thanks for your detailed explanations. Please see my in-line responses
below:

On Mon, Dec 06, 2010 at 03:13:11PM +0100, Robert Oostenveld wrote:
> Regarding octave: It would of course be nice if it ran both on
> octave and matlab. However, the active developers at the moment do
> not have a need for octave as alternative for Matlab. At the donders
> we have enough matlab licenses and I believe this applies to the
> other developers as well. The consequence is that maintaining
> compatibility with octave is not high on the priority list of any of
> these developers.

Octave compatibility has many advantages (not talking about abstract
philosophical ones now). Let say I want to run a simulation that
involves fieldtrip code on our cluster. I'd suddenly need 500! matlab
licenses for me alone. Money is not a severe problem here, but I bet my
institution would not appreciate having to double the number of licenses
if everybody wants to do like me.  Moreover, cloud-computing becomes
really accessible these days. Few lines of code allow me to send jobs to
e.g. Amazon's EC.  However, not for Matlab-based computing...

BUT: if fieldtrip works fine with octave you can quickly get an army of
users that help with maintaining compatibility, just expose it to
people. If there is code to perform an automatic functionality check it
would facilitate things significantly.

> The automatic testing remains a desireable feature, but currently we
> lack manpower and expertise for properly implementing this.

At NeuroDebian we are working with many neuroscience projects to
implement test suites and cross-toolkit interoperability tests. We'd be
happy to contribute.

> Regarding openmeeg: where do you in general envision the "glue code"
> between project A and project B to go? In this particular example, I
> think that the openmeeg glue code should go into the fieldtrip
> package. The reason for this is that it is maintained by the whole
> fieldtrip developer team, and hosted on the fieldtrip svn. Part of
> the glue is actually even in the fieldtrip/forward directory, so
> taking away fieldtrip/external/openmeeg might make it more confusing
> for users. Note that the openmeeg package in fieldtrip does
> automatic checking of the required binaries and provides
> instructions if the binaries are missing, so towards end-users it is
> friendlier to rely on that checking mechanism.

I feared that, and that is why I asked. I did not see that the externals
are more than just externals. Ideally you could make a clear distinction
between 3rd-party code and fieldtrip-specific glue. I agree that such
code belongs to fieldtrip, but it should not be labeled 'external', because
it is not really that.

< snip reasoning about tested dependencies >
> How do you envision that your debian packaging ensures correct
> versions to be co-installed?

This is a common problem that we have to deal with every day. The short
answer is: Debian packaging allows me to specify arbitrarily detailed
versioned dependencies. If fieldtrip of yesterday works with SPM2 older
than 10 years OR SPM8 of the very last release, I can say that and the
Debian package manager ensures that these dependencies become available
on the system -- if not possible it would refuse to install fieldtrip.
(softer dependencies are also possible, of course).

The issue of "tested" dependencies is more tricky. Your approach (and
the one of many others for that matter) is to "fork" 3rd-party code and
include it into fieldtrip. However, this entitles you to be responsible
for all bugs that affect the code that ships with fieldtrip. If some
external doesn't see support from its original developers anymore
(SPM2?) you even inherit all responsibility. But do you have enough
manpower to support all that additional code. Typically projects simply
assume that their forked dependencies work and do not monitor other
people's bugs (also because having public bugtrackers isn't common in
neuroscience -- fieldtrip being one of the noble exceptions!). However,
we believe this trust only works because so few people use the code and
so few bugs get reported -- the true health state of research software
code is, sadly, much worse.

We believe that the only way to ensure proper functioning is proper
testing. This exposes bugs, makes people aware, attracts contributions,
increases quality, make software more portable and eventually simply
better. I could talk at length about the benefits of
free-and-open-source-software development models, but I'll skip that.
We have seen a number of serious bugs in core neuro-imaging libraries and
applications in the past that affected research results, but only became
apparent by built/run-time tests of Debian packages on hardware
architectures that the original developers had no access to -- but
nevertheless affected common platforms like amd64.

That being said, I do not want to blame anyone for having manually
tested and frozen dependencies to ensure proper functioning. This is
often the only reasonable strategy that can be done with limited
manpower. What I'm arguing for is simply to enable more people/machines
to do testing and by that get access to "unlimited" manpower. Having
your software in Debian causes even non-neuroscientist developers to
check your code, run tests, send patches -- without you having to
motivate/pay/ask them at all.

> Of course if you consider every user to be super user on his own linux
> box, that is not a problem. But in reality many researchers have
> limited rights to the computers on which they are working. Recompiling
> the mex files is of course always an option, but it would be
> cumbersome to force this on the users.

Debian packaging does not enforce compilation on users. Everything is
performed automatically without any required interaction. If you do not
have the permissions to install things on a system you cannot install
packages. However, this issue is completely orthogonal to fieldtrip
providing a build system to allow (not enforce) recompiling MEX files.

> Why would you want to exclude the mex binaries from the
> release version?

The Debian policy mandates it. Every Debian source package must be
compilable by anybody (given access to all build-dependencies). There is
a trust path between Debian and its users that requires anybody to be
able to verify what is shipped in Debian's binary package. Compiling
EVERYTHING from source is the only way to ensure this. We don't even
ship pre-generated PDF, but rebuild them to make sure we can fix typos
in the manuals whenever we find them (over-simplified example ;-).

> Then some general questions: which part of the FieldTrip audience do
> you expect to benefit from the packaged version?

Almost everybody -- especially those you do not use fieldtrip yet.
Almost every single neuroscience software project that we packaged in
the past 6 years has seen an enormous increase in its user base after it
became available in the official Debian/ubuntu archives.

> I presume that it would be limited to the linux users, with root
> access to their own computers, and debian based releases.

Somewhat, but root access is only necessary for installing the package,
which is typically done by a sysadmin. The primary customer of a package
is a user that is served with readily usable software -- not a hacker.
Debian packaging also makes arbitrary software deployment very simple.
It doesn't matter whether you deploy Matlab-code, C-code, Python scripts
or data -- all is done with the same infrastructure and the same
command. This kind of deployment system is used in some large research
institutions to be able to handle complex systems with affordable
administration. If you want to get some first-hand reports you could
consider contacting some people listed in the 'research institutions'
section of our testimonials page:

http://neuro.debian.net/testimonials.html

> Actually I would expect those people to benefit more from the
> read-only SVN version of fieldtrip that we make available to all
> users. Working with SVN should be simple for this particular sub-class
> of fieldtrip users.  And using svn allows the user to roll back to
> older versions, and to make changes in his copy of the code without
> those changes being overwritten with the next "svn update".

Acknowledged. But as I said, Debian packaging is a global integration
effort that abstracts development and deployment workflows beyond the
level of a single project. What looks simple when working with fieldtrip
alone, becomes tedious when working with fieldtrip, AFNI, FSL, SPM,
openmeeg, ITK-SNAP, ANTS, ...

http://neuro.debian.net/pkgs.html

> If you want to discuss this in more detail, feel free to drop me an
> email to schedule a phone or skype call. Probably that is more
> efficient for both of us in getting things done.

Replied in a private message with my contact details.


Thanks again,

Michael

-- 
Michael Hanke
http://mih.voxindeserto.de