[FieldTrip] Debian packaging of fieldtrip

[Robert Oostenveld]

Hi Michael

Thanks for your detailled response. I don't think that we have big  
differences in how we would like to see things develop, only in how we  
are personally and with our respective teams able to further that  
development. Working together and combining our complementary  
expertise is certainly a good move. Let's continue this specific  
collaboration discussion off-line.

Regarding octave: that is certainly something that I see the value in.  
But the practical situation at our Donders Centre where ~150 people  
work is that we have ~100 MATLAB floating network licenses, and a 50  
node linux cluster. That number of licenses is enough to give people a  
comfortable number of data analysis sessions on their desktop  
workstations and on the linux cluster. We don't have the scientific  
manpower nor comuting hardware to keep many more than these 100 MATLAB  
sessions busy at any given point in time. The MATLAB maintenance  
contract (which entitles us to MATLAB updates) costs us ~6000 euro/ 
year. This includes a small increase in the number of licenses every  
year. This 6000 euros is peanuts compared to the scanner and personel  
costs, and compares more to the amount of money we spend on our coffee  
machine (yes, we do have a nice machine!).

So from a scientific budget point of view I currently cannot warrant  
investing effort from people of _our_ centre (i.e. salary costs) into  
the octave compatibility. But the prospect of outsourcing part of the  
computations to large clusters, grids or the cloud is of course  
interesting. For that reason I also started a distributed computing  
sub-project to fieldtrip (see http://fieldtrip.fcdonders.nl/development/peer) 
  and in that project the licenses do become more of an issue.

If I were able to get an "army of octave users" (as you write) to help  
me out with improving the compatibility of the code, I certainly would  
welcome that and would be willing to coordinate those efforts. If  
there are people on this mailing list interested in this, please do  
contact me! Also people that are willing to contribute to improving  
the testing infrastructure are very welcome.

best regards,
Robert




On 6 Dec 2010, at 16:32, Michael Hanke wrote:

> Hi Robert,
>
> thanks for your detailed explanations. Please see my in-line responses
> below:
>
> On Mon, Dec 06, 2010 at 03:13:11PM +0100, Robert Oostenveld wrote:
>> Regarding octave: It would of course be nice if it ran both on
>> octave and matlab. However, the active developers at the moment do
>> not have a need for octave as alternative for Matlab. At the donders
>> we have enough matlab licenses and I believe this applies to the
>> other developers as well. The consequence is that maintaining
>> compatibility with octave is not high on the priority list of any of
>> these developers.
>
> Octave compatibility has many advantages (not talking about abstract
> philosophical ones now). Let say I want to run a simulation that
> involves fieldtrip code on our cluster. I'd suddenly need 500! matlab
> licenses for me alone. Money is not a severe problem here, but I bet  
> my
> institution would not appreciate having to double the number of  
> licenses
> if everybody wants to do like me.  Moreover, cloud-computing becomes
> really accessible these days. Few lines of code allow me to send  
> jobs to
> e.g. Amazon's EC.  However, not for Matlab-based computing...
>
> BUT: if fieldtrip works fine with octave you can quickly get an army  
> of
> users that help with maintaining compatibility, just expose it to
> people. If there is code to perform an automatic functionality check  
> it
> would facilitate things significantly.
>
>> The automatic testing remains a desireable feature, but currently we
>> lack manpower and expertise for properly implementing this.
>
> At NeuroDebian we are working with many neuroscience projects to
> implement test suites and cross-toolkit interoperability tests. We'd  
> be
> happy to contribute.
>
>> Regarding openmeeg: where do you in general envision the "glue code"
>> between project A and project B to go? In this particular example, I
>> think that the openmeeg glue code should go into the fieldtrip
>> package. The reason for this is that it is maintained by the whole
>> fieldtrip developer team, and hosted on the fieldtrip svn. Part of
>> the glue is actually even in the fieldtrip/forward directory, so
>> taking away fieldtrip/external/openmeeg might make it more confusing
>> for users. Note that the openmeeg package in fieldtrip does
>> automatic checking of the required binaries and provides
>> instructions if the binaries are missing, so towards end-users it is
>> friendlier to rely on that checking mechanism.
>
> I feared that, and that is why I asked. I did not see that the  
> externals
> are more than just externals. Ideally you could make a clear  
> distinction
> between 3rd-party code and fieldtrip-specific glue. I agree that such
> code belongs to fieldtrip, but it should not be labeled 'external',  
> because
> it is not really that.
>
> < snip reasoning about tested dependencies >
>> How do you envision that your debian packaging ensures correct
>> versions to be co-installed?
>
> This is a common problem that we have to deal with every day. The  
> short
> answer is: Debian packaging allows me to specify arbitrarily detailed
> versioned dependencies. If fieldtrip of yesterday works with SPM2  
> older
> than 10 years OR SPM8 of the very last release, I can say that and the
> Debian package manager ensures that these dependencies become  
> available
> on the system -- if not possible it would refuse to install fieldtrip.
> (softer dependencies are also possible, of course).
>
> The issue of "tested" dependencies is more tricky. Your approach (and
> the one of many others for that matter) is to "fork" 3rd-party code  
> and
> include it into fieldtrip. However, this entitles you to be  
> responsible
> for all bugs that affect the code that ships with fieldtrip. If some
> external doesn't see support from its original developers anymore
> (SPM2?) you even inherit all responsibility. But do you have enough
> manpower to support all that additional code. Typically projects  
> simply
> assume that their forked dependencies work and do not monitor other
> people's bugs (also because having public bugtrackers isn't common in
> neuroscience -- fieldtrip being one of the noble exceptions!).  
> However,
> we believe this trust only works because so few people use the code  
> and
> so few bugs get reported -- the true health state of research software
> code is, sadly, much worse.
>
> We believe that the only way to ensure proper functioning is proper
> testing. This exposes bugs, makes people aware, attracts  
> contributions,
> increases quality, make software more portable and eventually simply
> better. I could talk at length about the benefits of
> free-and-open-source-software development models, but I'll skip that.
> We have seen a number of serious bugs in core neuro-imaging  
> libraries and
> applications in the past that affected research results, but only  
> became
> apparent by built/run-time tests of Debian packages on hardware
> architectures that the original developers had no access to -- but
> nevertheless affected common platforms like amd64.
>
> That being said, I do not want to blame anyone for having manually
> tested and frozen dependencies to ensure proper functioning. This is
> often the only reasonable strategy that can be done with limited
> manpower. What I'm arguing for is simply to enable more people/ 
> machines
> to do testing and by that get access to "unlimited" manpower. Having
> your software in Debian causes even non-neuroscientist developers to
> check your code, run tests, send patches -- without you having to
> motivate/pay/ask them at all.
>
>> Of course if you consider every user to be super user on his own  
>> linux
>> box, that is not a problem. But in reality many researchers have
>> limited rights to the computers on which they are working.  
>> Recompiling
>> the mex files is of course always an option, but it would be
>> cumbersome to force this on the users.
>
> Debian packaging does not enforce compilation on users. Everything is
> performed automatically without any required interaction. If you do  
> not
> have the permissions to install things on a system you cannot install
> packages. However, this issue is completely orthogonal to fieldtrip
> providing a build system to allow (not enforce) recompiling MEX files.
>
>> Why would you want to exclude the mex binaries from the
>> release version?
>
> The Debian policy mandates it. Every Debian source package must be
> compilable by anybody (given access to all build-dependencies).  
> There is
> a trust path between Debian and its users that requires anybody to be
> able to verify what is shipped in Debian's binary package. Compiling
> EVERYTHING from source is the only way to ensure this. We don't even
> ship pre-generated PDF, but rebuild them to make sure we can fix typos
> in the manuals whenever we find them (over-simplified example ;-).
>
>> Then some general questions: which part of the FieldTrip audience do
>> you expect to benefit from the packaged version?
>
> Almost everybody -- especially those you do not use fieldtrip yet.
> Almost every single neuroscience software project that we packaged in
> the past 6 years has seen an enormous increase in its user base  
> after it
> became available in the official Debian/ubuntu archives.
>
>> I presume that it would be limited to the linux users, with root
>> access to their own computers, and debian based releases.
>
> Somewhat, but root access is only necessary for installing the  
> package,
> which is typically done by a sysadmin. The primary customer of a  
> package
> is a user that is served with readily usable software -- not a hacker.
> Debian packaging also makes arbitrary software deployment very simple.
> It doesn't matter whether you deploy Matlab-code, C-code, Python  
> scripts
> or data -- all is done with the same infrastructure and the same
> command. This kind of deployment system is used in some large research
> institutions to be able to handle complex systems with affordable
> administration. If you want to get some first-hand reports you could
> consider contacting some people listed in the 'research institutions'
> section of our testimonials page:
>
> http://neuro.debian.net/testimonials.html
>
>> Actually I would expect those people to benefit more from the
>> read-only SVN version of fieldtrip that we make available to all
>> users. Working with SVN should be simple for this particular sub- 
>> class
>> of fieldtrip users.  And using svn allows the user to roll back to
>> older versions, and to make changes in his copy of the code without
>> those changes being overwritten with the next "svn update".
>
> Acknowledged. But as I said, Debian packaging is a global integration
> effort that abstracts development and deployment workflows beyond the
> level of a single project. What looks simple when working with  
> fieldtrip
> alone, becomes tedious when working with fieldtrip, AFNI, FSL, SPM,
> openmeeg, ITK-SNAP, ANTS, ...
>
> http://neuro.debian.net/pkgs.html
>
>> If you want to discuss this in more detail, feel free to drop me an
>> email to schedule a phone or skype call. Probably that is more
>> efficient for both of us in getting things done.
>
> Replied in a private message with my contact details.
>
>
> Thanks again,
>
> Michael
>
> -- 
> Michael Hanke
> http://mih.voxindeserto.de
> _______________________________________________
> Fieldtrip mailing list
> Fieldtrip at donders.ru.nl
> http://mailman.science.ru.nl/mailman/listinfo/fieldtrip