Subject: Re: CVNet - Visual psychophysics, Window/*unix-based OS
From: Mario Kleiner <[email-masked]>
To: "Etienne B. Roesch" <[email-masked]>
Cc: Stephane Rainville <[email-masked]>, [email-masked], Psychtoolbox Forum <[email-masked]>
Date: Sat, 10 Dec 2011 06:18:57 +0100

Dear all,

as the primary developer of Psychtoolbox for the last couple of
years, i should probably clarify a few things from my experience and
point of view. I'm cc'ing our user forum, as some people responded
there to Etienne as well.

For Psychtoolbox, Linux is now the recommended operating system
choice if you need the most robust and precise timing and a maximum
of flexibility. During benchmarking Linux showed higher out-of-the-
box timing precision than either OSX or Windows. Testing method and
results for different operating systems, as well as some practical
tips for optimization, are documented in a pdf file in Psychtoolboxs
PsychDocumentation subfolder. Linux also provides a couple of unique
optional features which allow further optimization for realtime
behaviour if the default behaviour is not good enough. Linux used to
be a low priority for PTB development in the past, but that has
changed during the last two years, especially now that we have over
700 users on Linux. At least for me, making sure Psychtoolbox works
best on Linux is a very high priority. I think Stephane's information
is a bit outdated here, referring to the status of maybe two years
ago, and the unfortunately always slightly outdated info on our Wiki.
In terms of functionality the Linux PTB is pretty much on par with
the OSX PTB, in some areas it is already ahead of it. A huge
advantage of Linux being open source and the development process for
most components being carried out openly as well, is that we can
often work directly with the developers of the operating system to
further improve its suitability for our purpose and to diagnose and
fix bugs in the system itself, instead of being forced to implement
sometimes problematic workarounds for bugs in proprietary operating
systems inside PTB. That doesn't mean everything on Linux is perfect
or free of bugs and quirks, or "just works", but for very demanding
paradigms i'd at least consider it the best starting point.

The NeuroDebian project <http://neuro.debian.net> mentioned by
Diederick provides a very convenient way to install and maintain PTB
and a large number of other open source neuroscience software
packages on Linux. It is basically an "App Store" for neuroscience
software. The recommended distribution "Ubuntu Linux" provides an
installer ("WUBI") that allows to safely install Linux inside an
existing MS-Windows installation for people who want to give it a
try, but are worried about accidentally reformatting their hard
discs. I encourage people to give Linux a try for demanding
experimental setups.

In my experience OSX on Apple hardware is still a good enough choice
for many use cases, with ok timing for most purposes, a compromise
between Linux and MS-Windows. However, the quality of low-level
features of the operating system relevant to us has degraded quite a
bit in my experience, at least for all versions after 10.4. Examples
of especially painful bugs i remember on certain combinations of
operating system and graphics cards: Broken stereoscopic stimulus
presentation (frame-sequential and multi-display), broken
presentation of high precision stimuli with more than 8 bit color or
luminance resolution, broken timestamping of visual stimulus onset.
The quality of Apple's developer support has degraded even more,
since they are now more focused on selling phones than computers. I
had to spent much more time during the last three years finding
workarounds for osx bugs than for windows and linux combined, and
Apple was less than helpful in resolving problems caused by their
bugs. Given Apple's attitude and the general pain of working with
buggy black boxes, piling workarounds on top of workarounds, i'm
nowadays much less inclined to "aggressively eradicate bugs in osx".
I'm usually much more inclined to recommend use of a more well suited
operating system for complex tasks.

MS-Windows is even more shaky, as others said. A well configured
system with the proper choice of hardware can provide acceptable
timing precision, but the proper choice of hardware and configuration
is mostly a matter of luck. As far as my experience goes, while
Windows XP had many bugs and deficiencies we had to work around in
ptb, but can be sort of ok timing wise, the overall quality of Vista
and Windows-7 has improved, while their timing got worse. Vista and
Windows-7 are almost unuseable wrt visual timing precision or
artifact free stimulus presentation as soon as one connects multiple
displays. This is mostly due to design decisions Microsoft made when
they redesigned the graphics system of Vista. There are a couple of
design limitations we can't even work around in the multi-display case.

All that said, for many tasks even a Windows system can be good
enough. Installing Linux on top of such a Windows system is easy if
you are unhappy with Windows performance or just want to give Linux a
try. If you are concerned with saving some money, i personally
wouldn't spend extra money for Apple hardware, because the "It just
works(tm)" advantage which was true just a few years ago is now gone
for demanding setups, sometimes it is quite the opposite.

Oh, and wrt. lookup table limitations on Windows, see <http://
docs.psychtoolbox.org/PsychImaging> subsection "EnableCLUTMapping". A
solution that works around the problem and different, but equally
annoying timing problems on other operating systems.

Wrt. to this statement of Wei Bi:

"In the old days, DirectDraw and OpenGL provided their own APIs to give
programmers opportunity to control the screen VSync. But as far as I
know,
the lastest OpenGL and Direct3D do not provide these functions any
more. We
have to make sure that our programs to update the algorithms and
graphics
fast enough for the available VSync to avoid Jitters."

I'm not aware of any relevant change to those APIs over the years?
There was and is functionality to enable or disable synchronization
of bufferswaps to vsync, that's it. If that doesn't work it is simply
due to graphics driver bugs or misconfiguration, but not by design.
PTB has extensive checks to find out if the mechanism is broken and
to enable workarounds for such problems. On most setups it just works
as expected. 3D desktop compositors, as used on all modern operating
systems to generate the stylish user interface, can cause large
trouble for visual stimulus timing, which is why PTB automatically
disables them during a session. One must often manually disable them
when using other toolkits.

Best,
-mario

On Dec 9, 2011, at 9:54 PM, Stephane Rainville wrote:

> Dear Etienne,
>
> From experience, I concur with others in this thread so far, but I
> do have my two cents to add.
>
> There is nothing wrong with using a PC to issue commands to
> specialized hardware that bypasses Windows graphics management
> (e.g., a VSG card), but you'll find that working with Windows
> graphics (e.g., PsychoToolbox, or 'PTB') is fraught with peril as
> the PTB discussion thread will attest. For instance, look-up tables
> on Windows are strictly limited to values that increase in a
> monotonic  fashion (even that may fail), and synchronizing look-up
> table changes to the vertical retrace can be unreliable.
>
> On the other hand, the Mac platform is designed from sound
> fundamental principles, and the Mac + Matlab + PTB combination has
> been extensively refined and debugged over the years. You're right
> that a PC linux box is an option, but you then expose yourself to
> potential  idiosyncracies of the Linux-on-PC + Matlab + PTB
> combination for which 1 - there are few users to help you out, and
> 2 - is low priority on the PTB development roadmap.
>
> For the price of buying a Mac (and the option to convert your
> Matlab license from PC to Mac rather than buy a new one), you're
> looking at a pretty powerful system whose few bugs have been (and
> continue to be) aggressively eradicated. Even a Mac mini or an iMac
> would likely do the trick, as I've recently determined that the
> Mini-Display port one of the recent iMac 27" supports 10-bit
> resolution.
>
> Hope this helps,
>
> Stephane
>
> ----------------------------------------------------------------------
> ---------------
> Stephane Rainville, PhD
>
> Founder and Senior Consultant
> VizirLabs - Accelerate Your Vision Research
>
> www.vizirlabs.com
> ----------------------------------------------------------------------
> ---------------
>
> On 12-08-2011, at 12:03 PM, Etienne B. Roesch wrote:
>
>> Dear all,
>>
>> I have been happily using the combo MacOS-MatLab-PsychToolbox for
>> a long time. I have had some experience with E-Prime and
>> Presentation, in the past, but my experience showed that Windows
>> is not to be trusted with extremely short display durations (say
>> 10ms, i.e. 1 refresh on a 100Hz CRT monitor).
>>
>> I am now faced with a decision to make about a new setup, and
>> because I haven't followed exactly what happened in the sphere of
>> the recent releases of Windows, I would like to query the
>> community on the setups they use, in terms of robustness,
>> reliability and accuracy for extremely short presentation timing.
>>
>> Apart from my favourite combo, I am also considering running
>> Matlab/PsychToolbox on a dedicated linux box.
>>
>> Thanks in advance to all of you,
>>
>> Etienne

*********************************************************************
Mario Kleiner
Max Planck Institute for Biological Cybernetics
Spemannstr. 38
72076 Tuebingen
Germany

e-mail: [email-masked]
office: +49 (0)7071/601-1623
fax:    +49 (0)7071/601-616
www:    http://www.kyb.tuebingen.mpg.de/~kleinerm
*********************************************************************
"For a successful technology, reality must take precedence
over public relations, for Nature cannot be fooled."
(Richard Feynman)

-------------------------------------------------------------------
To get information on using CVNet, send a note to: [email-masked]
In the body of the message, enter: info cvnet