Negative Binomial Distribution Puns

I have found one package (NBZIMM) for zero inflated distributions, however when I run his option for NB distributions it gives me different results than my previous package (lme4) for NBs distributions. Since NBZIMM is a package from an outside source at github, I am in doubt if it could be a problem with the package. Also, it does not display AIC or PIC values (NA), so its kinda hard for me to contrast my models.

Does anyone have a suggestion of a package I could use?

ps: I did find a seccond package (GLMMadaptive), but everytime I try to run my models on it, R aborts.

Thank you in advance

I'm trying to understand which variables (sex, age, weight, etc.) may be influencing the number of days a migratory species stay at my study site and for that I'm doing GLMs. The problem is that I still don't know which distribution to use.

I know Poisson assumes that my variable is normally distributed but the majority of individuals stayed at the study site for 1 or 2 days, with just a few staying for more days, so I have lots of small numbers and a few big ones (and days can't assume negative values). This doesn't seem normally distributed, however, I don't understand well the overdispersion concept the negative binomial distribution uses and when I tried to make an analysis (I'm using R software, if this helps), it keeps telling me "iteration limit reached". The Poisson analysis don't tell me any of this and works fine.

Is there a way I can test which distribution fits better? And on negative binomial's case, what does exactly mean "iteration limit reached"? I'm fairly new to statistical analysis so I appreciate any kind soul who takes time to answer this (probably) very basic question.

(It’s not a terribly complicated distribution and I probably should have recognized it sooner, but bear with me!)

For those who haven’t played Minecraft, there are tools like pickaxes and shovels you use regularly, and they have limited uses before they break. There is an enchantment that can be given to such tools—Unbreaking (with three tiers)—whose effect is essentially to make tools last longer. Unbreaking doesn’t add a set number of uses, however. Instead it changes the probability of the tool damaging each use from 100% to 50%, 33.3%, and 25% (for the three tiers of the enchantment).

This got me thinking: if my unenchanted tool would normally last 100 uses, how many uses should I expect to get from the tool if it’s enchanted with Unbreaking? In order to compute the expected value of a discrete variable like this, you need to know its probability mass function, so I tried to derive what it would be in this case.

Suppose the tool has Unbreaking 3. What’s the probability the tool breaks after 100 uses, as though it weren’t enchanted? Well, since the tool has a 25% chance of getting damaged each use, this amounts to that event being realized 100 uses straight: (0.25)^(100). There’s only one way for this to happen, so the calculation is complete. What about the probability it takes 101 uses? This amounts to the event being realized 99 times in the first 100 trials, and realized in the 101^st trial. At first I thought the binomial coefficient needed here was nCk(101, 100), but that would also count the case where the 101^st trial is the one without damage—that is, the case where we wouldn’t even see a 101^st trial.

That’s the point. The last trial is taken for granted as being a damaging event, so rather than looking at the nCk(101, 100) ways that 100 damaging events can be spread over 101 trials, we instead need the nCk(100, 99) ways that 99 damaging events can be spread over 100 trials, since we need exactly 99 damaging events before the last trial breaks the tool finally.

I hope the train of thought is clear here. I thought I was onto something fresh and original, but all I did was rediscover the negative binomial distribution. Still, it’s cool to see probability/statistics at work in a video game

Hello, I have been learning about different probability distribution recently. I see it stated that the cumulative distribution function for the negative binomial distribution is given in terms of the regularized incomplete beta function. (This is true for the regular binomial distribution as well, I have just been looking at the negative binomial distribution.)

However I can't find the cumulative distribution function actually derived anywhere. After playing around with some specific examples I have some intuition for why it makes sense, but how is it proven in general?

Thanks

I was thinking, with all of the election talk in the last couple of weeks, that you could estimate the probability of a state flipping in the next n votes by treating votes as samples from a negative binomial distribution where successes are votes for the candidate in question and failures would be votes for the opposition. And you can set these numbers based on the current vote tallies.

The issue with this is that with a modest number of votes remaining, tens to hundreds of thousands, this becomes intractable because of the binomial coefficient/choose operator taking factorials of these large numbers. For example, 25,000! works out to something like 10^100,000, a far larger number than any computer I have access to is able to store.

So I guess what I am wondering is if there is some trick to enable this sort of thing? Is there some approximation you can do with the choose operator to make this calculation more manageable? Even if my thought process regarding elections and the negative binomial is misguided, and something like Bayes Theorem would be more appropriate, surely there are some problems that need to be solved with the negative binomial such that the issue of large factorials needs to be addressed.

Hello everyone and Merry Christmas!
I am new to python and I try to develop a negative binomial distribution for a data frame that only has one column. I am not quite sure how to do it, as I thought I needed more than one parameter to make it work. This is what I have so far and I get an error at the last line.
Thank you in advance

import pandas as pd

df_sentence_length = pd.read_csv("QuietDonSentenceLenghts.csv")

import numpy as np

import statsmodels.api as sm

import scipy.stats

import matplotlib.pyplot as plt

from statsmodels.discrete.discrete_model import NegativeBinomial

negativebi = sm.NegativeBinomial(df_sentence_length[0]).fit()

https://imgur.com/a/nzZp9Gp - This is introduced in one paragraph, then a seemingly impossible question follows. The books "answer" is a joke. Can anyone explain how i'm supposed to know what literally any of the parameters are with "activity 2.6"? What is k? x? p? q? I mean i guess x=7 since "success must occur at the xth trial". Does that make k= 4 since there would be 7-3 trials where he misses?

As I understand it, the negative-binomial distribution is about "in a Bernoulli process (e.g. coin flip) with a given probability p, how many failures will you get until you get n successes."

So I'm looking at scipy's documentation here: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.nbinom.html and it confirms my understanding:

> Negative binomial distribution describes a sequence of i.i.d. Bernoulli trials, repeated until a predefined, non-random number of successes occurs. ... nbinom takes and as shape parameters where n is the number of successes, whereas p is the probability of a single success.

Yet the sample code shows an example of the "number of sucesses" being n = 0.4:

from scipy.stats import nbinom

# Calculate a few first moments:
n, p = 0.4, 0.4        # << how can n be 0.4?

mean, var, skew, kurt = nbinom.stats(n, p, moments='mvsk')

But I'm really confused by n = 0.4. So maybe I do some Python I get these results:

p = 0.5
n = 3
nbinom.pmf(np.arange(0,11), n, p)
array([0.125     , 0.1875    , 0.1875    , 0.15625   , 0.1171875 ,
   0.08203125, 0.0546875 , 0.03515625, 0.02197266, 0.01342773,
   0.00805664])

And I explain that with something like, "with a series of fair coin flips, you have a 0.125 chance of getting the targeted 3 success right away (e.g. HHH), and 0.1875 chance of getting one failure and 3 successes (e.g. HTHH, or HHTH...).

But how do I interpret/describe this? What does it mean to get "0.4 successes?" in a discrete distribution?:

p = 0.5
n = 0.4
nbinom.pmf(np.arange(0,11), n, p)
array([7.57858283e-01, 1.51571657e-01, 5.30500798e-02, 2.12200319e-02,
   9.01851357e-03, 3.96814597e-03, 1.78566569e-03, 8.16304314e-04,
   3.77540745e-04, 1.76185681e-04, 8.28072701e-05])

And R does the same thing:

p = .5
r = 0.4
dnbinom(x = 0:10, prob = p, size=r)
[1] 7.578583e-01 1.515717e-01 5.305008e-02 2.122003e-02 9.018514e-03 3.968146e-03 1.785666e-03 8.163043e-04 3.775407e-04 1.761857e-04
[11] 8.280727e-05

If n is a count of the number of successes, how can it be 0.4... or what does this even mean?

Hello everyone,

Just a quick question on the derivation of the negative binomial variance. Coaching actuaries has the following, which makes sense:

https://preview.redd.it/8rmdpdg5zi731.png?width=481&format=png&auto=webp&s=e3c38c769ff800c823069b0bdf28bf294b37aa02

But I am failing to see why doing this isn't valid:

https://preview.redd.it/g7f39d37zi731.png?width=717&format=png&auto=webp&s=5e453ae591de6345183ef9a005b310955b5e39e5

Thanks for your help!

Without getting too technical, a negative binomial distribution allows you to calculate the odds of a given number of failures before attaining a fixed number of successes in a varying number of independent trials, assuming that you stop upon reaching the desired number of successes.

In other words, what are the odds of getting 3 skill ups at 2.5x rates if I feed 6 times? Or 7? Or 8?

Excel has a built-in function for this (NEGBINOM.DIST), so I tossed this together for a quick way to break it down numerically and visually.

Fill in the two green fields at the top. "Probability of success" should be the probability of an individual success, so 0.25 if you're doing skill-ups at 2.5x rates (i.e. 25%). Or 1/3 if you're trying to farm a specific dragon fruit from the Thursday Mythical (assuming all three have an equal chance to drop). Enter the number you need in "Number of successes".

The two output columns (and associated chart series) "odds" and "cumulative" show the chance of needing exactly a given number of attempts or at most that number of attempts respectively. So for my Urd that needs 3 more skill levels, you can see there's about an 80% chance of getting it within 16 feeds at 2.5x rates.

Click the Download button on the toolbar to grab a copy.

https://onedrive.live.com/redir?resid=E7D536CAB8019F21!1269&authkey=!AEp3rhl37b72ahs&ithint=file%2cxlsx

Also, remember gambler's fallacy. If you've made three attempts already and they all failed, you're still starting from zero, and your odds going forward are the same as they were before the three failures.

Reference for further nerding: http://stattrek.com/probability-distributions/negative-binomial.aspx

Here's the problem - https://imgur.com/pXp3o3v

I need help with all the parts a), b) and c). I derived the equation to solve for in part a), but the equation has a variable with a factorial in the denominator and a constant in the numerator for which I have no technique to evaluate or solve.

Please help. Thanks in advance :)

I'm in charge of running an analysis on some count-data and I want to compared my negative binomial GEE model to a zero-inflated negative binomial GEE model. However, SAS doesn't want to let me do that. Or Proc Genmod won't let me do that and I can't figure out what Proc is the right one.

Any tips? Keep in mind that I'll have to defend whatever I do to my superiors, so citations would be greatly appreciated.

question

Ten percent of the engines manufactured on an assembly line are defective. If engines are randomly selected one at a time and tested, and given that the first two engines tested were defective, what is the probability that at least two more engines must be tested before the first nondefective is found?

my attempt So letting Y be the number of engines tested before first nondefective is found and that p=0.9,

I suppose I need to find P(Y=>4|Y>2). The solution says that this in turn equals to P(Y>3|Y>2) which in turn equals to P(Y>1) and so 1-P(Y=0) is the answer.

Can someone enlighten me on why P(Y=>4) equates to P(Y>3).. and why it equates to P(Y>1) and not P(Y>2) at the next step? Thanks.

Hello all,

I have been reading that where we have count data, fitting a simple Poisson model is often seen as inappropriate due to over-dispersion. That is, the variance of the raw data is often greater than its mean, whereas in the Poisson model the variance is equal to the mean.

So one alternative approach is to consider the data to have a Negative Binomial distribution. This gives the probability of having seen f failiures before s successes in repeated independant trials with prob of success p.

I'm just having a bit of trouble seeing how the two scenarios are equivalent. Poisson gives the number of occurrences in some fixed time period, given that they occur at rate lambda. That makes sense to me with count data - say, the number of deaths observed in a hospital over a given length of time.

NegBin gives the number of failures seen until some fixed number of successes. So I take it here my "failures" would be the number of deaths observed. And my "successes" would therefore be the number of patients who survived. But where I get confused is that this is not fixed, which the NegBin requires.

Had the number of observed deaths been different, the number surviving would have changed too (since it is the total number of patients, s+f, that is fixed - which is the setting for the binomial distribution).

Have I just confused myself? Is there an intuitive way to see the parallel between these two distributions?

I've got two populations of patients, one normal and one cancer. I've got gene expression values (~20,000 genes) for each patient. I was told that the gene expression values are counts and that t-test is not the best test to use to see which genes are statistically significant between populations. Supposedly, because the data consists of counts from a sequencer, the count data follows either a Poisson or negative binomial distribution which should be what I use for hypothesis testing. I was told to look at http://www.biomedcentral.com/1752-0509/5/S3/S1 , http://www.ncbi.nlm.nih.gov/pubmed/19910308 , and http://www.bioconductor.org/packages/release/bioc/manuals/edgeR/man/edgeR.pdf , but it is not clear to me at all from these links (a paper and the manual for the edgeR package in bioconductor/R) how I would perform this kind of analysis on my data.

Before I knew about the above, I would just read in the two populations into two separate data frames in R and then do a t-test on the rows of gene values (rows are genes and columns are patients). Is it as simple as doing some sort of Poisson-based or negative binomial-based test on my two data frames, or do I need to use the R/bioconductor package which I'm not sure how to use (plus I'm not sure if my data is properly formatted for use with this package based on the documentation).

Any help or guidance would be greatly appreciated. Thank you!

I have 5 estimates of the mean of a negative binomial distributed random variable. I know the sizes of the samples used to generate each estimate, but not the individual data points. I am trying to come up with a likelihood estimator for population parameters p and r given the 5 sample means. I am really at a loss...

Hi guys, so I already know that the negative binomial distribution can extend up to an infinite number of trials. But I was just wondering if there was a way to confine this to a finite number of trials. Lets say I have biased coin that flips with P(heads)=0.7 and P(tails)=0.3 on any bernoulli trial. If I keep flipping, how do I find the probability that I will get my 40th head on the nth trial, provided I want a maximum of 100 trials? Clearly, the lower limit here would be trial number 40, and an upper limit of trial number 100.

I can't just chop off the distribution at a finite number because then adding everything up would not give up a value of 1. I would thus assume it is a scaling problem, when I need to find the sum of all the probabilities that are above the upper limit that I have set, and then scale the rest to add up to a probability of 1? (Seems too simple here - then again, just because I cut off my maximum number of trials at 100 doesn't mean I necessarily have to obtain 40 heads within those 100 trials, so the total probability isn't actually 1?) Or is there another way to do it?

Hopefully someone can explain this to me. Thanks! :)

Is there anyone good at stats that can help me understand what I have to do?