p - CiteSeerX

The Genetic Algorithm Estimates for the Parameters of
Order p Normal Distributions (1)
Un Algoritmo Genetico per la Stima dei Parametri di Distribuzioni Normali
di Ordine p
Salvatore Vitrano
Dipartimento di Sociologia, Universita’ degli Studi “La Sapienza”
Via Salaria 113, I-00198 Roma, e-mail: [email protected]
Roberto Baragona
Dipartimento di Sociologia, Universita’ degli Studi “La Sapienza”
Via Salaria 113, I-00198 Roma, e-mail: [email protected]
Riassunto: Le distribuzioni normali di ordine p, anche note come distribuzioni esponenziali
potenziate, costituiscono una naturale generalizzazione della distribuzione normale. La
stima dei parametri presenta delle difficoltà non trascurabili sia per quanto riguarda la
distorsione che la variabilità. Gli algoritmi genetici, sperimentati con successo in molte
applicazioni dell’inferenza statistica, possono costituire in questo senso una valida
alternativa nell’ambito dei metodi di stima proposti. Sono qui considerate la stima del
parametro di locazione per piccoli campioni, e dell’ordine p per campioni di numerosità
moderata. Un esperimento di simulazione illustra i risultati e i vantaggi che possono
derivare dall’uso degli algoritmi genetici e permette un confronto con alcuni metodi
precedentemente introdotti e largamente sperimentati.
Keywords: Exponential power functions, Genetic algorithms, Jackknife estimators,
Maximum-likelihood, Normal distributions of order p.
1. Introduction
th
Subbotin (1923), by generalizing the 2 Schiaparelli axiom, introduced a family of error
distributions named Exponential Power Functions (EPF). Several formulations of the EPF
were extensively analyzed in the literature, but here we refer to the model known as Normal
Distribution of Order p, defined on the whole real axis and characterized by the location µ,
the scale σp, and the shape parameter p (p>0). A family of unimodal symmetric curves from
this density function arises with different shapes for different values of p. The Laplace
(p=1), the Normal (p=2), and the Uniform (p → ∞ ) distributions arise as special cases. Let
x=(x1, x2, …, xn) be a random sample drawn from this probability density function. The
maximum-likelihood (ML) estimator of the parameter vector θ =(µ, σp, p) is any value θ̂
which maximizes the likelihood function:
n
Lθ (x) = {2 p1/pσp Γ(1+1/p)}–n exp{–(pσ pp ) −1 ∑ | x i
i =1
(1)
- µ | p }.
(1)
The present paper is financially supported by the University of Rome “La Sapienza” and by M.U.R.S.T.
(Italy) as a part of the grant “Non linear models and new computational methods in time series analysis and
forecasting,“ 2000.
Properties and parameter estimation methods for the distributions in this class were
extensively studied (see Chiodi, 2000, for a comprehensive review).
We propose a genetic algorithm (GA) approach for estimating θ . Holland (1975) studied
the evolution of a population of individuals in a given environment by means of a class of
analytical models called GA. Each individual in the population, which has not to be given a
strictly statistical meaning, needs to be characterized by a chromosome, i.e. a string of
characters. Each character in the string possesses a specific meaning that may be decoded
from both its value, called allele, and position, called locus. The population evolves through
several generations towards the best adaptation to the environment. A basic assumption is
that a function may be defined that maps the genetic pool (the set of all admissible strings)
into the positive real axis. This function is called the fitness function, and it has to increase
as soon as the adaptation to the environment increases. The special applications of the GA
for function optimization assume the objective function as the fitness function, and the
string owned by the best fit individual as the solution to the optimization problem.
Chatterjee and Laudatto (1997) investigated using GA for statistical inference, and we
closely followed the guidelines provided therein.
The plan of the paper is as follows. The next section is devoted to the illustration of the GA
we implemented for assessing estimates and standard errors. Comparison with alternative
approaches by means of a simulation experiment is provided in Section 3. The estimation
of the location parameter µ in case of small samples, for given order p, and the estimation
of all three parameters, for rather large sample sizes, is considered.
2. Genetic algorithm implementation
The main features of our design of the GA are the tournament selection for reproduction,
and the use of three operators, crossover, inversion, and mutation. Unlike many GA
applications, inversion proved to constitute a valid support to increase the speed of
convergence to the solution. Further, coding the potential solutions, that are real numbers in
a given interval (a,b), say, was done by using the following device. Let us choose a positive
integer as the binary string length. Then, 2 different numbers x may be coded according
to
x = a + c (b – a) / (2 – 1),
(2)
where c represents the real value corresponding to the given binary string. As c may vary
from 0 (the null string 000…000) through 2–1 (the all-one string 111…111), then (2)
encodes a finite subset of real numbers in the interval (a,b), at equispaced intervals of
length (b–a)/(2–1) each.
As far as the GA parameters choice is concerned, for the present problem, we assumed the
selection pressure ps=0.75, the crossover, mutation, and inversion probability pc=0.7,
pi=0.65, and pm=0.01 respectively, and 50 as the number of generations. We assumed the
population size rather small, s=50, as no substantial improvement was observed by taking
larger values. The likelihood (1) provided the fitness function.
Table 1: Order-p Normal distribution parameter estimates, all parameters unknown
µ
p=1
n=50
n=100
p=1.25
n=50
n=100
p=1.50
n=50
n=100
p=1.75
n=50
n=100
p=2
n=50
n=100
p=2.25
n=50
n=100
p=2.5
n=50
n=100
p=2.75
n=50
n=100
p=3
n=50
n=100
p=3.25
n=50
n=100
Maximum likelihood
p
σp
µ
Genetic algorithm
σp
p
3.0193
2.1429
1.2462
3.0267
2.1341
(.2868)
(.3198)
(.3390)
(.2796)
(.3171)
(.3279)
2.9781
2.0606
1.1144
3.0074
2.0403
1.1099
(.2004)
(.1917)
(.1602)
(.2169)
(.1988)
(.1762)
2.9765
2.0486
1.5072
3.0235
2.0528
1.4813
(.3578)
(.3436)
(.7016)
(.3176)
(.3237)
(.6211)
3.0447
2.0468
1.3691
3.0066
2.0593
1.3774
(.2181)
(.2495)
(.2954)
(.2063)
(.2820)
(.3766)
2.9694
1.9805
1.7014
2.9812
2.0096
1.8719
(.3106)
(.3483)
(.7612)
(.3167)
(.3783)
(.9849)
2.9924
1.9949
1.5935
2.9922
2.0094
1.5980
(.1959)
(.2634)
(.3806)
(.2009)
(.2324)
(.4399)
3.0454
2.1079
2.2741
2.9633
2.0760
2.1561
(.3220)
(.4431)
(1.0201)
(.2962)
(.3908)
(.9725)
2.9570
1.9861
1.8479
2.9894
1.9606
1.8190
(.1987)
(.2150)
(.3964)
(.2111)
(.2353)
(.4464)
3.0064
2.0554
2.5580
3.0510
2.0858
2.5986
(.2908)
(.3666)
(1.1405)
(.2963)
(.3090)
(1.0214)
2.9938
2.0654
2.2843
3.0127
2.024
2.2229
(.2196)
(.2362)
(.7219)
(.1815)
(.2246)
(.6650)
3.0160
2.0807
3.0267
2.9586
2.0637
2.8037
(.2985)
(.3049)
(1.1726)
(.2848)
(.3257)
(1.1711)
3.0314
1.9904
2.4970
2.9944
2.0054
2.4778
(.2077)
(.2345)
(.6559)
(.1883)
(.2125)
(.7259)
2.9395
2.0682
3.2349
2.9985
1.9875
3.0510
(.2562)
(.2919)
(1.1717)
(.2670)
(.3054)
(1.2151)
3.0771
2.0807
2.9473
3.0098
2.0395
2.7787
(.1718)
(.2123)
(.8330)
(.1661)
(.2171)
(.7440)
2.9752
2.0459
3.2150
3.0213
1.9974
3.2069
(.2416)
(.3026)
(1.1268)
(.2671)
(.3146)
(1.1257)
2.9778
2.0149
3.0505
2.9904
1.9498
2.8491
(.1846)
(.2213)
(.8972)
(.1846)
(.1966)
(.7361)
2.9970
2.0584
3.6761
3.0009
2.0152
3.5714
(.2448)
(.2661)
(1.1710)
(.2509)
(.3022)
(1.2258)
3.0188
2.0110
3.4036
3.0051
2.0365
3.3715
(.1733)
(.2124)
(.9240)
(.1624)
(.1970)
(.8667)
3.0054
1.9841
4.0412
3.0103
1.9782
3.7364
(.2199)
(.2248)
(1.0605)
(.2260)
(.2782)
(1.2226)
2.9912
2.0242
3.6140
3.0063
2.0174
3.5910
(0.1609)
(.1932)
(.9203)
(.1639)
(.2161)
(.9309)
Note: the estimated standard errors are given in parentheses.
K
1.2250
10
5
5
6
2
1
2
1
3. Simulation and results
To compare the parameter estimates obtained by the GA with the ML estimates obtained by
the common search numerical algorithms, two distinct Monte Carlo simulations were
designed. The numerical inspection of the sample mean and standard deviation of the
parameter estimates was made by drawing 100 samples using of size n= 10,20,50, when p
was supposed known, and n=50, 100, when p was unknown, from a Normal distribution of
order p (µ=3; σp=2) with p = 1-3.25 (0.25), through a generator algorithm (Chiodi, 1986).
Some simulation results are given in Table 1. The number K of samples, for which the ML
procedure has not reached any optimum point of (1), was also reported along with the
parameter estimates by the two approaches. When p>2, a less bias was observed using the
GA than the ML approach. Opposite results occur if p≤2. In this case, however, some
samples were discarded by the ML procedure. Similar findings were obtained, except some
particular cases, for the estimates of the two other parameters. When p was assumed
known, the GA and two different scale parameter estimators (Chiodi, 1988; La Tona, 1994)
were considered. The estimators gave results that were found to differ less than 1/1000,
with best results by using the GA, when p>2. Finally, in both types of Monte Carlo
simulations (p known and unknown), the standard deviations of the estimates from the two
approaches were obtained that had the same order of magnitude.
References
Chatterjee, S., Laudatto, M. (1997) Genetic algorithms in statistics: procedures and
applications, Communications in Statistics - Simulation and Computation, 26(4), 16171630.
Chiodi, M. (1986) Procedures for generating pseudo-random numbers from a normal
distribution of order p (p>1), Statistica Applicata, 1, 7-26.
Chiodi, M. (1988) Sulle distribuzioni di campionamento delle stime di massima
verosimiglianza dei parametri delle curve normali di ordine p, Istituto di Statistica
Università di Palermo.
Chiodi, M. (2000) Le curve normali di ordine p nell’ambito delle distribuzioni di errori
accidentali: una rassegna dei risultati e problemi aperti per il caso univariato e per quello
multivariato, in: Atti della XL Riunione Scientifica della SIS, ISTAT Centro Stampa, 5960. Extended and revised version available at http://statistica.economia.unipa.it/
Holland, J. H. (1975) Adaptation in Natural and Artificial Systems. University of Michigan
Press (Second Edition: MIT Press, 1992)
La Tona, L. (1994) Uno stimatore jackknife del parametro di scala di una curva normale di
norma p, Metron, 52, 181-188.
Subbotin, M. T. (1923) On the law of frequency of errors, Matematicheskii Sbornik, 31,
296-300.