SD formula for $A/B$ testing

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite












I'm trying to understand the statistic behind $A/B$ testing and link it to a statistic I know from school.
I'm struggling to translate binominal distribution and sampling to A/B test.
I found online (VWO) formula for SD (standard deviation) as
Standard Error (SE) = $sqrt(p cdot (1-p))/ n$ where $p$ is conversion and $n$ is number of trials.



One answer here What is the difference and relationship between the binomial and Bernoulli distributions?
gives SD formula as $sqrt(kp(1-p))/n $ where $k$ is a number of trials
and $n$ is a sample size.



How does it translate to $A/B$ testing?
I assume $p$ is a conversion rate/probability of success
$K$ is the number of people in the test group
$n$ is it a number of people in the test size too?



I also read somewhere that for samples you divide by $n-1$, not $n$? Why that's not the case here?







share|cite|improve this question

























    up vote
    0
    down vote

    favorite












    I'm trying to understand the statistic behind $A/B$ testing and link it to a statistic I know from school.
    I'm struggling to translate binominal distribution and sampling to A/B test.
    I found online (VWO) formula for SD (standard deviation) as
    Standard Error (SE) = $sqrt(p cdot (1-p))/ n$ where $p$ is conversion and $n$ is number of trials.



    One answer here What is the difference and relationship between the binomial and Bernoulli distributions?
    gives SD formula as $sqrt(kp(1-p))/n $ where $k$ is a number of trials
    and $n$ is a sample size.



    How does it translate to $A/B$ testing?
    I assume $p$ is a conversion rate/probability of success
    $K$ is the number of people in the test group
    $n$ is it a number of people in the test size too?



    I also read somewhere that for samples you divide by $n-1$, not $n$? Why that's not the case here?







    share|cite|improve this question























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I'm trying to understand the statistic behind $A/B$ testing and link it to a statistic I know from school.
      I'm struggling to translate binominal distribution and sampling to A/B test.
      I found online (VWO) formula for SD (standard deviation) as
      Standard Error (SE) = $sqrt(p cdot (1-p))/ n$ where $p$ is conversion and $n$ is number of trials.



      One answer here What is the difference and relationship between the binomial and Bernoulli distributions?
      gives SD formula as $sqrt(kp(1-p))/n $ where $k$ is a number of trials
      and $n$ is a sample size.



      How does it translate to $A/B$ testing?
      I assume $p$ is a conversion rate/probability of success
      $K$ is the number of people in the test group
      $n$ is it a number of people in the test size too?



      I also read somewhere that for samples you divide by $n-1$, not $n$? Why that's not the case here?







      share|cite|improve this question













      I'm trying to understand the statistic behind $A/B$ testing and link it to a statistic I know from school.
      I'm struggling to translate binominal distribution and sampling to A/B test.
      I found online (VWO) formula for SD (standard deviation) as
      Standard Error (SE) = $sqrt(p cdot (1-p))/ n$ where $p$ is conversion and $n$ is number of trials.



      One answer here What is the difference and relationship between the binomial and Bernoulli distributions?
      gives SD formula as $sqrt(kp(1-p))/n $ where $k$ is a number of trials
      and $n$ is a sample size.



      How does it translate to $A/B$ testing?
      I assume $p$ is a conversion rate/probability of success
      $K$ is the number of people in the test group
      $n$ is it a number of people in the test size too?



      I also read somewhere that for samples you divide by $n-1$, not $n$? Why that's not the case here?









      share|cite|improve this question












      share|cite|improve this question




      share|cite|improve this question








      edited Jul 14 at 20:45







      user401938
















      asked Jul 14 at 20:13









      twopens

      11




      11

























          active

          oldest

          votes











          Your Answer




          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "69"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );








           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2851918%2fsd-formula-for-a-b-testing%23new-answer', 'question_page');

          );

          Post as a guest



































          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes










           

          draft saved


          draft discarded


























           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2851918%2fsd-formula-for-a-b-testing%23new-answer', 'question_page');

          );

          Post as a guest













































































          Comments

          Popular posts from this blog

          What is the equation of a 3D cone with generalised tilt?

          Relationship between determinant of matrix and determinant of adjoint?

          Color the edges and diagonals of a regular polygon