Rokenn Swiftsong wrote:
> From: "Rokenn Swiftsong" <rokenn@...>
>
> On a monk board I have been reading lately (started playing my monk again) I
> found a link to an interesting discussion on the benefits (or lack there of)
> of stat buffs. The Shaman that ran the test came to the conclusion that
> that buffing Str had a negative impact on damage. Here is a link to the
> thread:
>
> http://www.soerbaird.com/forums/Forum16/HTML/001893.html
>
> I would love to read John's comments on this :)
Well, different John here... but I'll comment nonetheless.
BTW, I really dislike that John Kim has usurped the name John here. :)
Anyway, speaking as someone who does this sort of testing professionally, I can tell you that
this person did a lot of work into the test, but personally, I wouldn't take the results to the
bank.
The main reason is that he just didn't have a big enough sample size, in my opinion. His
smallest test case had only 71 attempted hits, and of those, only 55% hit. That combined with
the variability of the possible results (1 damage to 39 damage) makes it quite possible for
random variation to infuence the results more than they should.
For example: We all know that the numeric mean of rolling a six sided die is 3.5. If you rolled
a 6 sided die an infinite number of times, assuming the die is perfectly balanced, you should get
a mean of 3.5 -- but you don't need to do it that many times. The way randomness works means
that you can roll it a lesser number of times, and depending on how many times you roll that die,
be PRETTY sure that the result you get will accurately reflect the true result. That's what
statistics and probabability is all about.
Now, using the 6 sided die, if you rolled it 71 times, would you get 3.5 as the mean? I'd guess
probably. 71 times is a lot of rolls when there are only 6 possible results.
On the other hand, if you roll it 71 times and it's a 40 sided die, do you think you'd get the
true mean of 20.5? Maybe.... but I'd expect a result something in the neighbourhood of 18-24.
That's the effect of variability. And this test didn't even get 71 hits, he got 39. Do you
think you could roll a 40 sided die and get an accurate reading of the average roll with 39
tries? Try rolling a six sided die 5 times and seeing if you get an average of 3.5.
If you've taken a lot of stats courses, you'll know what I mean by the concept of "power". Power
determines the size of sample you'll need to get within a certain allowable variability. I could
whip out my stats book and figure out the power needed to get an acceptable result for this
test... but to be honest, I hate stats, so I'll wimp out. :)
These tests MAY be valid, but given the weird results, I would take them with a grain of salt.
The person may be on to something here, I won't deny it, but I'd like to see the test repeated
and compare the results.
Jyzan