std(x)/mean(x) in QC4 and Bessel correction

[MichaelClerx]

In QC4 we take std(x)/mean(x) of two values (see #94) using nump's standard version of std which doesn't use the n/(n+1) correction https://en.wikipedia.org/wiki/Bessel%27s_correction (in numpy terminology, ddof=0)

For the other QCs using RMSD, the series are so long that it doesn't make a big difference, but for n=2 it makes a difference of sqrt(2)

What do we do to make this obvious / easy to report in papers?

Do we change it to be to be the sample std (ddof=1 in numpy terms) and adjust the threshold accordingly? (just move the sqrt(2) into the threshold)

Do we keep it as it is but reformulate as |a-b|/(a+b) explicitly to avoid this confusion?

@mirams @chonlei @joeyshuttleworth ?

[joeyshuttleworth]

Yes, this is a bit ambiguous.

I think we can just make sure to call it the "population standard deviation" to be clearer.

I think you want to use the sample std (ddof=1) when you take a sample and are trying to estimate the population standard deviation., and we're not really doing any statistics here, so I don't think we need to account for the N-1 or worry about including a factor of sqrt(2).

|a-b|/(a+b) would be fine too, and would be much clearer in the code (don't need to remember that np defaults to ddof=0). But for the sake of documentation/publications, I think pop. std / mean is a nice way of explaining what we're trying to do.

[mirams]

I think 'standard deviation' on its own normally means 1/N version, and I would use 'corrected' or 'unbiased estimate of population' or something for 1/(N+1). Agree with Joey that we should stick to 1/N here as we are just talking about the variance of the signals we have, not the projected variance of more signals if we cared to measure more.

I'm not sure "population standard deviation" as a standalone phrase helps particularly (without saying whether you have the whole population or just a sample of it that you are calculating it from!).

[MichaelClerx]

Hmmm.
For future QC's, would it make for sense to just do |a - b|/c < threshold, where c is some sensible scale for each. E.g. 1MOhm for Rseries and 1pF for Cm ?

I don't really understand how Rseal could change during an experiment (I get how the physical R would change, but where would we get that measurement?) so that sounds a dodgy one to me

[mirams]

OK, yeah for QC4 it does sound a bit odd. With resistance effects having a 'one over' relationship with bad things, I think the scaling is intended to get the magnitudes 'about right' before doing the comparison. We don't want anything absolute (1MOhm to 2MOhm seal resistance change is not the same as 100MOhm to 101 MOhm).

[MichaelClerx]

Is that not exactly why you'd want an absolute scale? 1 to 2 is fine. 10 to 20 is not