[visionlist] signal detection query

[Kornbrot, Diana]

HI Joseph,
As Todd & others have clearly described there is no difficulty with problem 1.

As for problem 2, unfortunately the outside TMS fa rate CANNOT provide any information whatsoever as to whether differences in hit rate under strong and weak TMS are due to differences in d' or differences in bias.
Simple algebra shows why:
d'(strong TMS) = z(hit, strong TMS) - z(fa, no TMS)
d'(weak TMS) = z(hit, weak TMS) - z(fa, no TMS)
Hence
d'(strong TMS) - d'(weak TMS) = z(hit, strong TMS) = z(hit, weak TMS),
the contribution form the fa rate is eliminated
If z(hit, strong TMS) IS different from z(hit, weak TMS) this could be due to better discriminability, or to people in the strong TMS state being more  [or less] biased towards 'yes'. The fa rate outside TMS cannot provide a solution. It is necessary to have separate FA rates under the two TMS stimulation conditions in order to draw any conclusion about d'
Don't shoot the messenger
Best
Diana


On 21/07/2010 16:13, "Todd S. Horowitz" <toddh at search.bwh.harvard.edu> wrote:

Daniel, Joseph

I think we're all agreed now on point (1) :)

As to point (2), I don't think Daniel's objection is a problem for Joseph's study, since the point is not to compare old stimuli+TMS to new stimuli without TMS, but to compare old TMS and old non-TMS stimuli; the new stimuli are there simply to measure the false alarm rate.

Similarly, I think this dispenses with Daniel's other objection. It's true that the Gaussian equal-variance assumptions probably do not apply, so that d' is not independent of criterion. However, since all of the stimuli are being tested in the same block of trial, criterion should be constant, so the d's will be comparable.

However, this makes me wonder why bother to compute SDT measures at all. Since the false alarm rate should be constant for both classes of stimuli, why not just compare % correct?

thanks
Todd

On Jul 21, 2010, at 4:26 AM, Daniel Oberfeld wrote:

Hi Joseph,

Re (1) : If you use the correct formula for calculating d', then it will automatically correct for unequal numbers of old and new pictures.

Re (2): I think this is no problem for calculating the SDT statistics, but rather for the interpretation of your results - does it make sense to compare responses to old stimuli+TMS and responses to new stimuli without TMS...?

There is one very serious issue with calculating d' for your data, however. In case you collected binary responses ("Is the picture old or new?"), then for calculating d' you will have to assume that the internal distributions for "signal" and "noise" have identical standard deviations (cf. Macmillan & Creelman, 2005). It is known for a long time that this assumption is frequently incorrect for experimental data (e.g., Swets, 1986). And thus d' is not a valid measure of sensitivity because it is strongly influenced by response bias (Verde, MacMillan, & Rotello, 2006).

The simple solution (at least for future experiments) is to obtain rating responses rather than binary responses - with these responses, you can caculate for example the area under the ROC curve, which is a valid index of sensitivity even if the SDs of the internal distributions are unequal (Swets, 1986).  Again, Macmillan & Creelman (2005) explain in detail how to conduct such an experiment.

Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide (2. ed.). Mahwah, NJ [et al.]: Lawrence Erlbaum Associates.
Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: their ROCs and implied models. Psychological Bulletin, 99(1), 100-117.
Verde, M. F., MacMillan, N. A., & Rotello, C. M. (2006). Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of d ', A(z), and A '. Perception and Psychophysics, 68(4), 643-654.


Best,

Daniel



________________________________
Professor Diana Kornbrot
  email:  d.e.kornbrot at herts.ac.uk
   web:    http://web.mac.com/kornbrot/iweb/KornbrotHome.html
Work
School of Psychology
University of Hertfordshire
College Lane, Hatfield, Hertfordshire AL10 9AB, UK
    voice:     +44 (0) 170 728 4626
    mobile:   +44 (0) 796 890 2102
    fax          +44 (0) 170 728 5073
Home
19 Elmhurst Avenue
London N2 0LT, UK
   landline: +44 (0) 208 883 3657
   mobile:   +44 (0) 796 890 2102
   fax:         +44 (0) 870 706 4997
________________________________




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist/attachments/20100722/1b9cbeca/attachment-0001.htm>