[visionlist] signal detection query

[Todd S. Horowitz]

Daniel, Joseph

I think we're all agreed now on point (1) :)

As to point (2), I don't think Daniel's objection is a problem for Joseph's study, since the point is not to compare old stimuli+TMS to new stimuli without TMS, but to compare old TMS and old non-TMS stimuli; the new stimuli are there simply to measure the false alarm rate.

Similarly, I think this dispenses with Daniel's other objection. It's true that the Gaussian equal-variance assumptions probably do not apply, so that d' is not independent of criterion. However, since all of the stimuli are being tested in the same block of trial, criterion should be constant, so the d's will be comparable.

However, this makes me wonder why bother to compute SDT measures at all. Since the false alarm rate should be constant for both classes of stimuli, why not just compare % correct?

thanks
Todd

On Jul 21, 2010, at 4:26 AM, Daniel Oberfeld wrote:

> Hi Joseph,
> 
> Re (1) : If you use the correct formula for calculating d', then it will automatically correct for unequal numbers of old and new pictures.
> 
> Re (2): I think this is no problem for calculating the SDT statistics, but rather for the interpretation of your results - does it make sense to compare responses to old stimuli+TMS and responses to new stimuli without TMS...?
> 
> There is one very serious issue with calculating d' for your data, however. In case you collected binary responses ("Is the picture old or new?"), then for calculating d' you will have to assume that the internal distributions for "signal" and "noise" have identical standard deviations (cf. Macmillan & Creelman, 2005). It is known for a long time that this assumption is frequently incorrect for experimental data (e.g., Swets, 1986). And thus d' is not a valid measure of sensitivity because it is strongly influenced by response bias (Verde, MacMillan, & Rotello, 2006).
> 
> The simple solution (at least for future experiments) is to obtain rating responses rather than binary responses - with these responses, you can caculate for example the area under the ROC curve, which is a valid index of sensitivity even if the SDs of the internal distributions are unequal (Swets, 1986).  Again, Macmillan & Creelman (2005) explain in detail how to conduct such an experiment.
> 
> Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide (2. ed.). Mahwah, NJ [et al.]: Lawrence Erlbaum Associates.
> Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: their ROCs and implied models. Psychological Bulletin, 99(1), 100-117.
> Verde, M. F., MacMillan, N. A., & Rotello, C. M. (2006). Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of d ', A(z), and A '. Perception and Psychophysics, 68(4), 643-654.
> 
> 
> Best,
> 
> Daniel
> 
> 
> -- 
> Dr. Daniel Oberfeld-Twistel
> Johannes Gutenberg - Universitaet Mainz
> Department of Psychology
> Experimental Psychology
> Wallstrasse 3
> 55122 Mainz
> Germany
> 
> Phone ++49 (0) 6131 39 39274
> Fax   ++49 (0) 6131 39 39268
> http://www.staff.uni-mainz.de/oberfeld/
> _______________________________________________
> visionlist mailing list
> visionlist at visionscience.com
> http://visionscience.com/mailman/listinfo/visionlist

Todd S. Horowitz, PhD
Assistant Professor of Ophthalmology
	Harvard Medical School
Associate Director
	Visual Attention Lab
	Brigham & Women's Hospital
64 Sidney Street, Suite 170 
Cambridge, MA 02139
phone:  (617) 768-8813
fax:    (617) 768-8816
http://search.bwh.harvard.edu/



The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist/attachments/20100721/3e47198d/attachment.htm>