Semiparametric tail-index estimation for randomly right-truncated heavy-tailed data

Mancer, Saida; Necir, Abdelhakim; Benchaira, Souad

doi:10.1108/AJMS-02-2022-0033

Purpose

The purpose of this paper is to propose a semiparametric estimator for the tail index of Pareto-type random truncated data that improves the existing ones in terms of mean square error. Moreover, we establish its consistency and asymptotic normality.

Design/methodology/approach

To construct a root mean squared error (RMSE)-reduced estimator of the tail index, the authors used the semiparametric estimator of the underlying distribution function given by Wang (1989). This allows us to define the corresponding tail process and provide a weak approximation to this one. By means of a functional representation of the given estimator of the tail index and by using this weak approximation, the authors establish the asymptotic normality of the aforementioned RMSE-reduced estimator.

Findings

In basis on a semiparametric estimator of the underlying distribution function, the authors proposed a new estimation method to the tail index of Pareto-type distributions for randomly right-truncated data. Compared with the existing ones, this estimator behaves well both in terms of bias and RMSE. A useful weak approximation of the corresponding tail empirical process allowed us to establish both the consistency and asymptotic normality of the proposed estimator.

Originality/value

A new tail semiparametric (empirical) process for truncated data is introduced, a new estimator for the tail index of Pareto-type truncated data is introduced and asymptotic normality of the proposed estimator is established.

1. Introduction

Let $(X_{i}, Y_{i})$ ⁠, i = 1, …, N ≥ 1 be a sample from a couple $(X, Y)$ of independent positive random variables (rv’s) defined over a probability space $(Ω, A, P)$ ⁠, with continuous distribution functions (df’s) F and G, respectively. Suppose that X is right-truncated by Y, in the sense that X_i is only observed when X_i ≤ Y_i. Thus, let us denote $(X_{i}, Y_{i})$ ⁠, i = 1, …, n to be the observed data, as copies of a couple of dependent rv’s $(X, Y)$ corresponding to the truncated sample $(X_{i}, Y_{i})$ ⁠, i = 1, …, N, where n = n_N is a random sequence of discrete rv’s. By the weak law of large numbers, we have

n / N \overset{P}{\to} p ≔ P (X \leq Y) = \int_{0}^{\infty} F (w) d G (w), as N \to \infty,

(1.1)

where the notation $\overset{P}{\to}$ stands for the convergence in probability. The constant p corresponds to the probability of observed sample which is supposed to be non-null, otherwise nothing is observed. The truncation phenomena frequently occurs in medical studies, when one wants to study the length of survival after the start of the disease: if Y denotes the elapsed time between the onset of the disease and death, and if the follow-up period starts X units of time after the onset of the disease then, clearly, X is right-truncated by Y. For concrete examples of truncated data in medical treatments one refers, among others, to Refs. [1, 2]. Truncated data schemes may also occur in many other fields, namely actuarial sciences, astronomy, demography and epidemiology, see for instance the textbook of [3].

From [4] the marginal df’s F^* and G^* corresponding to the joint df of $(X, Y)$ are given by

F^{*} (x) ≔ p^{- 1} \int_{0}^{x} \bar{G} (w) d F (w) and G^{*} (x) ≔ p^{- 1} \int_{0}^{x} F (w) d G (w) .

By the previous first equation, we derive a representation of the underlying df F as follows:

F (x) = p \int_{0}^{x} \frac{d F^{*} (w)}{\bar{G} (w)},

(1.2)

which will be for a great interest thereafter. In the sequel, we are dealing with the concept of regular variation. A function φ is said to be regularly varying at infinity with negative index − 1/η, notation $φ \in R V (- 1 / η)$ ⁠, if

φ (s t) / φ (t) \to s^{- 1 / η}, as t \to \infty,

(1.3)

for s > 0. This relation is known as the first-order condition of regular variation and the corresponding uniform convergence is formulated in terms of “Potter’s inequalities” as follows: for any small ϵ > 0, there exists t₀ > 0 such that for any t ≥ t₀ and s ≥ 1, we have

(1 - ϵ) s^{- 1 / η - ϵ} < φ (s t) / φ (t) < (1 + ϵ) s^{- 1 / η + ϵ} .

(1.4)

See for instance Proposition B.1.9 (assertion 5, page 367) in Ref. [5]. The second-order condition (see Ref. [6] expresses the rate of the convergence $(1.3)$ above. For any x > 0, we have

\frac{φ (t x) / φ (t) - x^{- 1 / η}}{A (t)} \to x^{- 1 / η} \frac{x^{τ / η} - 1}{τ η}, as t \to \infty,

(1.5)

where τ < 0 denotes the second-order parameter and A is a function tending to zero and not changing signs near infinity with regularly varying absolute value with positive index τ/η. A function φ that satisfies assumption $(1.5)$ is denoted $φ \in {R V}_{2} (- 1 / η; τ, A)$ ⁠. We now have enough material to tackle the main goal of the paper. To begin, let us assume that the tails of both df’s F and G are regularly varying. That is

\bar{F} \in R V (- 1 / γ_{1}) and \bar{G} \in R V (- 1 / γ_{2}), with γ_{1}, γ_{2} > 0 .

(1.6)

Under this assumption, [4] showed that

{\bar{F}}^{*} \in R V (- 1 / γ_{1}) and {\bar{G}}^{*} \in R V (- 1 / γ),

(1.7)

where

γ ≔ \frac{γ_{1} γ_{2}}{γ_{1} + γ_{2}} .

(1.8)

For further details on the proof of this statement one refers to Ref. [7] (Lemma A1). The estimation of the tail index γ₁ was recently addressed for the first time in Ref. [4] where the authors used equation $(1.8)$ to propose an estimator to γ₁ as a ratio of Hill estimators [8] of the tail indices γ and γ₂. These estimators are based on the top order statistics X_n−k:n ≤ … ≤ X_n:n and Y_n−k:n ≤ … ≤ Y_n:n pertaining to the samples $(X_{1}, \dots, X_{n})$ and $(Y_{1}, \dots, Y_{n})$ respectively. The sample fraction k = k_n being a sequence of integers such that, k_n → ∞ and k_n/n → 0 as n → ∞. The asymptotic normality of the given estimator is established in Ref. [9]. By using a Lynden-Bell integral, [10] proposed the following estimator for the tail index γ₁:

{\hat{γ}}_{1}^{(W)} (u) ≔ \frac{1}{{\bar{F}}_{n}^{(1)} (u)} \sum_{i = 1}^{n} 1 (X_{i} > u) \frac{F_{n}^{(1)} (X_{i})}{C_{n} (X_{i})} \log \frac{X_{i}}{u},

for a given deterministic threshold u > 0, where

F_{n}^{(1)} (x) ≔ \prod_{X_{i} > x} [1 - \frac{1}{n C_{n} (X_{i})}],

is the popular nonparametric maximum likelihood estimator of cdf F introduced in the well-known work [11]; with

C_{n} (x) ≔ \frac{1}{n} \sum_{i = 1}^{n} 1 (X_{i} \leq x \leq Y_{i}) .

Independently, [7] used a Woodroofe integral with a random threshold, to derive the following estimator

{\hat{γ}}_{1}^{(B M N)} ≔ \frac{1}{{\bar{F}}_{n}^{(2)} (X_{n - k : n})} \sum_{i = 1}^{k} \frac{F_{n}^{(2)} (X_{n - i + 1 : n})}{C_{n} (X_{n - i + 1 : n})} \log \frac{X_{n - i + 1 : n}}{X_{n - k : n}},

(1.9)

where

F_{n}^{(2)} (x) ≔ \prod_{X_{i} > x} \exp \{- \frac{1}{n C_{n} (X_{i})}\},

is the so-called Woodroofe’s nonparametric estimator [12] of df F. To improve the performance of ${\hat{γ}}_{1}^{(B M N)}$ ⁠, [13, 14], respectively, proposed a Kernel-smoothed and a reduced-bias versions of this estimator and established their consistency and asymptotic normality. It is worth mentioning that Lynden-Bell integral estimator ${\hat{γ}}_{1}^{(W)} (u)$ with a random threshold u = X_n−k:n becomes

{\hat{γ}}_{1}^{(W)} ≔ \frac{1}{{\bar{F}}_{n}^{(1)} (X_{n - k : n})} \sum_{i = 1}^{k} \frac{F_{n}^{(1)} (X_{n - i + 1 : n})}{C_{n} (X_{n - i + 1 : n})} \log \frac{X_{n - i + 1 : n}}{X_{n - k : n}} .

(1.10)

In a simulation study, [15] compared this estimator with ${\hat{γ}}_{1}^{(B M N)}$ ⁠. They pointed out that both estimators have similar behaviors in terms of biases and mean squared errors.

Recall that the nonparametric Lynden-Bell estimator $F_{n}^{(1)}$ was constructed on the basis of the fact that F and G are both unknown. In this paper, we are dealing with the situation when F is unknown but G is parametrized by a known model G_θ, $θ \in Θ \subset R^{d}$ ⁠, d ≥ 1 having a density g_θ with respect to Lebesgue measure. [2] considered this assumption and introduced a semiparametric estimator for df F defined by

F_{n} (x; {\hat{θ}}_{n}) ≔ P_{n} ({\hat{θ}}_{n}) \frac{1}{n} \sum_{i = 1}^{n} \frac{1 (X_{i} \leq x)}{{\bar{G}}_{{\hat{θ}}_{n}} (X_{i})},

(1.11)

where $1 / P_{n} ({\hat{θ}}_{n}) ≔ n^{- 1} \sum_{i = 1}^{n} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{i})$ and

{\hat{θ}}_{n} ≔ \arg \max_{θ \in Θ} \prod_{i = 1}^{n} g_{θ} (Y_{i}) / {\bar{G}}_{θ} (X_{i}),

(1.12)

denoting the conditional maximum likelihood estimator (CMLE) of θ, which is consistent and asymptotically normal, see for instance Ref. [16]. On the other hand, [2] showed that $F_{n} (x; {\hat{θ}}_{n})$ is an uniformly consistent estimator over the x-axis and established, under suitable regularity assumptions, its asymptotic normality. [2, 17] pointed out that the semiparametric estimate has greater efficiency uniformly over the x-axis. In the light of a simulation study, the authors suggest that the semiparametric estimate is a better choice when parametric information of the truncation distribution is available. Since the apparition of this estimation method many papers are devoted to the statistical inference with truncation data, see for instance Refs. [18–22] and [23].

Motivated by the features of the semiparametric estimation, we next propose a new estimator for γ₁ by means of a suitable functional of $F_{n} (x; {\hat{θ}}_{n})$ ⁠. We start our construction by noting that from Theorem 1.2.2 in de [5]; the first-order condition $(1.6)$ (for F) implies that

\lim_{t \to \infty} \frac{1}{\bar{F} (t)} \int_{t}^{\infty} \log (x / t) d F (x) = γ_{1} .

(1.13)

In other words, γ₁ may viewed as a functional $ψ_{t} (F)$ ⁠, for a large t, where

ψ_{t} (F) ≔ \frac{1}{\bar{F} (t)} \int_{t}^{\infty} \log (x / t) d F (x) .

Replacing F by $F_{n} (\cdot; {\hat{θ}}_{n})$ and letting t = X_n−k:n yield

\begin{align} {\hat{γ}}_{1} & = ψ_{X_{n - k : n}} (F_{n} (\cdot; {\hat{θ}}_{n})) \\ = \frac{1}{{\bar{F}}_{n} (X_{n - k : n}; {\hat{θ}}_{n})} \int_{X_{n - k : n}}^{\infty} \log (x / X_{n - k : n}) d F_{n} (x; {\hat{θ}}_{n}), \end{align}

(1.14)

as new estimator for γ₁. Observe that

\begin{array}{l} \int_{t}^{\infty} \log (x / t) d F_{n} (x; {\hat{θ}}_{n}) \\ = P_{n} (\hat{θ}) \int_{X_{n - k : n}}^{\infty} \log (x / X_{n - k : n}) 1 (x \geq X_{n - k}) d F_{n} (x; {\hat{θ}}_{n}), \end{array}

which may be rewritten into

\begin{array}{l} \frac{P_{n} ({\hat{θ}}_{n}) 1}{n} \sum_{i = 1}^{n} \int_{X_{n - k : n}}^{\infty} \frac{\log (x / X_{n - k : n}) 1 (x \geq X_{n - k})}{{\bar{G}}_{{\hat{θ}}_{n}} (X_{i})} d 1 (X_{i} \leq x) \\ = P_{n} ({\hat{θ}}_{n}) \frac{1}{n} \sum_{i = 1}^{k} \frac{\log (X_{n - i + 1} / X_{n - k : n})}{{\bar{G}}_{{\hat{θ}}_{n}} (X_{n - i + 1 : n})} . \end{array}

On the other hand, $F (X_{n - k : n}; {\hat{θ}}_{n})$ equals

P_{n} ({\hat{θ}}_{n}) \frac{1}{n} \sum_{i = 1}^{n} \frac{1 (X_{i : n} \leq X_{n - k : n})}{{\bar{G}}_{{\hat{θ}}_{n}} (X_{i : n})} = P_{n} ({\hat{θ}}_{n}) \frac{1}{n} \sum_{i = 1}^{n - k} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{i : n}) .

Hence,

\begin{array}{l} \bar{F} (X_{n - k : n}; {\hat{θ}}_{n}) & = \frac{\frac{1}{n} \sum_{i = 1}^{n} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{i : n}) - \frac{1}{n} \sum_{i = 1}^{n - k} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{i : n})}{1 n \sum_{i = 1}^{n} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{i : n})} \\ = P_{n} ({\hat{θ}}_{n}) \frac{1}{n} \sum_{i = 1}^{k} 1 / {\bar{G}}_{{\hat{θ}}_{n}} (X_{n - i + 1 : n}) . \end{array}

Thereby, the form of our new estimator is

{\hat{γ}}_{1} = \frac{\sum_{i = 1}^{k} {({\bar{G}}_{{\hat{θ}}_{n}} (X_{n - i + 1 : n}))}^{- 1} \log (X_{n - i + 1} / X_{n - k : n})}{\sum_{i = 1}^{k} {({\bar{G}}_{{\hat{θ}}_{n}} (X_{n - i + 1 : n}))}^{- 1}} .

(1.15)

The asymptotic behavior of ${\hat{γ}}_{1}$ will be established by means of the following tail empirical process

D_{n} (x; {\hat{θ}}_{n}; γ_{1}) ≔ \sqrt{k} (\frac{{\bar{F}}_{n} (x X_{n - k : n}; {\hat{θ}}_{n})}{{\bar{F}}_{n} (X_{n - k : n}; {\hat{θ}}_{n})} - x^{- 1 / γ_{1}}), for x > 1 .

This method was already used to establish the asymptotic behavior of Hill’s estimator for complete data [5]; page 162) that we will adapt to the truncation case. Indeed, by using an integration by parts and a change of variables of the integral $(1.14)$ ⁠, one gets

{\hat{γ}}_{1} = \int_{1}^{\infty} x^{- 1} \frac{{\bar{F}}_{n} (x X_{n - k : n}; {\hat{θ}}_{n})}{{\bar{F}}_{n} (X_{n - k : n}; {\hat{θ}}_{n})} d x,

and therefore

\sqrt{k} ({\hat{γ}}_{1} - γ_{1}) = \int_{1}^{\infty} x^{- 1} D_{n} (x; {\hat{θ}}_{n}; γ_{1}) d x .

(1.16)

Thus, for a suitable weighted weak approximation to $D_{n} (\cdot; {\hat{θ}}_{n}; γ_{1})$ ⁠, we may easily deduce the consistency and asymptotic normality of ${\hat{γ}}_{1}$ ⁠. This process may also contribute to the goodness-of-fit test to fitting heavy-tailed distributions via, among others, the Kolmogorov–Smirnov and Cramér–von Mises type statistics

\sup_{x > 1} |D_{n} (x; {\hat{θ}}_{n}, {\hat{γ}}_{1})| and \int_{1}^{\infty} D_{n}^{2} (x; {\hat{θ}}_{n}, {\hat{γ}}_{1}) d x^{- 1 / {\hat{γ}}_{1}} .

More precisely, these statistics are used when testing the null hypothesis H₀: “both F and G are heavy-tailed” versus the alternative one H₁: “at least one of F and G is not heavy-tailed”, that is H₀: “ $(1.6)$ holds” versus H₁: “ $(1.6)$ does not hold”. This problem has been already addressed by Refs. [24, 25] in the case of complete data. The (uniform) weighted weak convergence of $D_{n} (x; {\hat{θ}}_{n}, γ_{1})$ and the asymptotic normality of ${\hat{γ}}_{1}$ ⁠, stated below, will be of great interest to establish the limit distributions of the aforementioned test statistics. This is out of the scope of this paper whose remainder is structured as follows. In Section 2, we present our main results which consist in the consistency and asymptotic normality of estimator ${\hat{γ}}_{1}$ ⁠. The performance of the proposed estimator is checked by simulation in Section 3. An application to a real dataset composed of induction times of AIDS diseases is given in Section 4. The proofs are gathered in Section 5. A useful lemma and its proof are postponed to Appendix.

2. Main results

The regularity assumptions, denoted $[A 0]$ ⁠, concerning the existence, consistency and asymptotic normality of the CLME estimator ${\hat{θ}}_{n}$ ⁠, given in $(1.12)$ ⁠, are discussed in Ref. [16]. Here, we only state additional conditions on df G_θ corresponding to Pareto-type models which are required to establish the asymptotic behavior of our newly estimator ${\hat{γ}}_{1}$ ⁠.

$[A 1]$ For each fixed y, the function $θ \to G_{θ} (y)$ is continuously differentiable of partial derivatives $G_{θ}^{(j)} = : \partial G_{θ} / \partial θ_{j}$ ⁠, j = 1, …, d.
$[A 2] {\bar{G}}_{θ}^{(j)} \in R V (- 1 / γ_{2})$ ⁠.
$[A 3] y^{- ϵ} {\bar{G}}_{θ}^{(j)} (y) / {\bar{G}}_{θ} (y) \to 0$ ⁠, as y → ∞, for any ϵ > 0.

For common Pareto-type models, one may easily check that there exist some constants a_j ≥ 0, c_j and d_j, such that ${\bar{G}}_{θ}^{(j)} (y) \sim c_{j} (y^{- 1 / γ_{2}} + d_{j}) \log y$ ⁠, for all large x. Then one may consider that the assumptions $[A 1] - [A 3]$ are not very restrictive and they may be acceptable in the extreme value theory.

Theorem 2.1.

Assume that $\bar{F} \in {R V}_{2} (- 1 / γ_{1}; ρ_{1}, A)$ and $G_{θ} \in R V (- 1 / γ_{2})$ satisfying the assumptions $[A 0] - [A 3]$ , and suppose that γ₁ < γ₂. Then on the probability space $(Ω, A, P)$ , there exists a standard Wiener process $\{W (s), 0 \leq s \leq 1\}$ such that, for any small 0 < ϵ < 1/2, we have

\sup_{x > 1} x^{ϵ} |D_{n} (x; {\hat{θ}}_{n}, γ_{1}) - Γ (x; W) - x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k})| \overset{P}{\to} 0,

provided that

\sqrt{k} A (a_{k}) = O (1)

⁠, where

\begin{array}{l} Γ (x; W) \begin{matrix} ≔ \end{matrix} \frac{γ}{γ_{1}} x^{- 1 / γ_{1}} \{x^{1 / γ} W (x^{- 1 / γ}) - W (1)\} \\ + \frac{γ}{γ_{1} + γ_{2}} x^{- 1 / γ_{1}} \int_{0}^{1} s^{- γ / γ_{2} - 1} \{x^{1 / γ} W (x^{- 1 / γ} s) - W (s)\} d s, \end{array}

is a centered Gaussian process and

a_{k} ≔ F^{* \leftarrow} (1 - k / n)

⁠, where

F^{* \leftarrow} (s) ≔ \inf \{x : F^{*} (x) \geq s\}, 0 < s < 1,

denotes the quantile (or the generalized inverse) function pertaining to df F*.

Applying this weak approximation, we establish both consistency and asymptotic normality of our new estimator ${\hat{γ}}_{1}$ ⁠, that we state in the following Theorem.

Theorem 2.2.

Under the assumptions of Theorem 2.1, we have

\begin{array}{l} {\hat{γ}}_{1} - γ_{1} \\ = k^{- 1 / 2} \int_{1}^{\infty} x^{- 1} Γ (x; W) d x + A (a_{k}) \int_{1}^{\infty} x^{- 1 / γ_{1} - 1} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} d x + o_{P} (k^{- 1 / 2}), \end{array}

this implies that

{\hat{γ}}_{1} \overset{P}{\to} γ_{1}

⁠. Whenever

\sqrt{k} A (a_{k}) \to λ < \infty

⁠, we get

\sqrt{k} ({\hat{γ}}_{1} - γ_{1}) \overset{D}{\to} N (\frac{λ}{1 - ρ_{1}}, σ^{2}),

where

σ^{2} ≔ γ^{2} (1 + γ_{1} / γ_{2}) (1 + {(γ_{1} / γ_{2})}^{2}) {(1 - γ_{1} / γ_{2})}^{3}

⁠, and

1 (A)

stands for the indicator function pertaining to a set

A

⁠.

3. Simulation study

In this section, we will perform a simulation study in order to compare the finite sample behavior of our new semiparametric estimator ${\hat{γ}}_{1}$ ⁠, given in $(1.15)$ ⁠, with Woodrofee and Lynden-Bell integral estimators ${\hat{γ}}_{1}^{(B M N)}$ and ${\hat{γ}}_{1}^{(W)}$ ⁠, given respectively in $(1.9)$ and $(1.10)$ ⁠. The truncation and truncated distributions functions F and G will be chosen among the following two models:

Burr $(γ, δ)$ distribution with right-tail function:

\bar{H} (x) = {(1 + x^{1 / δ})}^{- δ / γ}, x \geq 0, δ > 0, γ > 0;

Fréchet $(γ)$ distribution with right-tail function:

\bar{H} (x) = 1 - \exp (- x^{- 1 / γ}), x > 0, γ > 0 .

The simulation study is being made in fours scenarios following to the choice of the underlying df’s F and G_θ:

$[S 1]$ Burr $(γ_{1}, δ)$ truncated by Burr $(γ_{2}, δ)$ ⁠; with $θ = (γ_{2}, δ)$
$[S 2]$ Fréchet $(γ_{1})$ truncated by Fréchet $(γ_{2})$ ⁠; with θ = γ₂
$[S 3]$ Fréchet $(γ_{1})$ truncated by Burr $(γ_{2}, δ)$ ⁠; with $θ = (γ_{2}, δ)$
$[S 4]$ Burr $(γ_{1}, δ)$ truncated by Fréchet $(γ_{2})$ ⁠; with θ = γ₂

To this end, we fix δ = 1/4 and choose the values 0.6 and 0.8 for γ₁ and 55% and 90% for the portions of observed truncated data given in $(1.1)$ so that the assumption γ₁ < γ₂ stated in Theorem 2.1 holds. In other words, the values of p have to be greater than 50%. For each couple $(γ_{1}, p)$ ⁠, we solve the equation $(1.1)$ to get the pertaining γ₂-value, which we summarize as follows:

(p, γ_{1}, γ_{2}) = (55 %, 0.6, 1.4), (90 %, 0.6, 5.4), (55 %, 0.8, 1.9), (90 %, 0.8, 7.2) .

(3.17)

For each scenario, we simulate 1000 random samples of size N = 300 and compute the root mean squared error (RMSE) and the absolute bias (ABIAS) corresponding to each estimator ${\hat{γ}}_{1}$ ⁠, ${\hat{γ}}_{1}^{(B M N)}$ and ${\hat{γ}}_{1}^{(W)}$ ⁠. The comparison is done by plotting the ABIAS and RMSE as functions of the sample fraction k which varies from 2 to 120. This range is chosen so that it contains the optimal number of upper extremes k* used in the computation of the tail index estimate. There are many heuristic methods to select k*, see for instance Ref. [26]; here we use the algorithm proposed by Ref. [27] in page 137, which is incorporated in the R software “Xtremes” package. Note that the computation the CMLE of θ is made by means of the syntax ”maxLik” of the MaxLik R software package. The optimal sample fraction k* is defined, in this procedure, by

k^{*} ≔ \arg \min_{1 < k < n} \frac{1}{k} \sum_{i = 1}^{k} i^{ω} |\hat{γ} (i) - median \{\hat{γ} (1), \dots, \hat{γ} (k)\}|,

for suitable constant 0 ≤ ω ≤ 1/2, where $\hat{γ} (i)$ corresponds to an estimator of tail index γ, based on the i upper order statistics, of a Pareto-type model. We observed, in our simulation study, that ω = 0.3 allows better results both in terms of bias and RMSE. It is worth mentioning that making N vary did not provide notable findings; therefore, we kept the size N fixed. The finite sample behaviors of the above-mentioned estimators are illustrated in Figures 1–8. The overall conclusion is that the biases of three estimators are almost equal, however, in the case of medium truncation $(p \approx 50 %)$ ⁠, the RMSE of our new semiparametric ${\hat{γ}}_{1}$ is clearly the smallest compared that of ${\hat{γ}}_{1}^{(B M N)}$ and ${\hat{γ}}_{1}^{(W)}$ ⁠. Actually, the medium truncation situation is the most frequently encountered in real data, while the strong truncation $(p ≫ 50 %)$ remains, up to our knowledge, theoretical. In this sense, we may consider that the semiparametric estimator is more efficient than the two other ones. We point out that the two estimators ${\hat{γ}}_{1}^{(B M N)}$ and ${\hat{γ}}_{1}^{(W)}$ have almost the same behavior which actually was noticed before by Ref. [15]. The optimal sample fractions and estimate values of the tail index obtained through the three estimators are given in Tables 1–4.

Figure 1

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S1:γ1=0.6,p=55% (top two panels) and γ1=0.6,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{1} : (γ_{1} = 0.6, p = 55 %)$ (top two panels) and $(γ_{1} = 0.6, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 2

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S1:γ1=0.8,p=55% (top two panels) and γ1=0.8,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{1} : (γ_{1} = 0.8, p = 55 %)$ (top two panels) and $(γ_{1} = 0.8, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 3

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S2:γ1=0.6,p=55% (top two panels) and γ1=0.6,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{2} : (γ_{1} = 0.6, p = 55 %)$ (top two panels) and $(γ_{1} = 0.6, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 4

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S2:γ1=0.8,p=55% (top two panels) and γ1=0.8,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{2} : (γ_{1} = 0.8, p = 55 %)$ (top two panels) and $(γ_{1} = 0.8, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 5

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1MBN (red) and γ^1W(blue), corresponding to two situations of scenario S3:γ1=0.6,p=55% (top two panels) and γ1=0.6,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(M B N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{3} : (γ_{1} = 0.6, p = 55 %)$ (top two panels) and $(γ_{1} = 0.6, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 6

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S3:γ1=0.8,p=55% (top two panels) and γ1=0.8,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{3} : (γ_{1} = 0.8, p = 55 %)$ (top two panels) and $(γ_{1} = 0.8, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 7

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S4:γ1=0.6,p=55% (top two panels) and γ1=0.6,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{4} : (γ_{1} = 0.6, p = 55 %)$ (top two panels) and $(γ_{1} = 0.6, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Figure 8

Absolute bias (left two panels) and RMSE (right two panels) of γ^1 (black) and γ^1BMN (red) and γ^1W(blue), corresponding to two situations of scenario S4:γ1=0.8,p=55% (top two panels) and γ1=0.8,p=90% (bottom two panels) based on 1000 samples of size 300 Refer to the image caption for details.

View large Download slide

Absolute bias (left two panels) and RMSE (right two panels) of ${\hat{γ}}_{1}$ (black) and ${\hat{γ}}_{1}^{(B M N)}$ (red) and ${\hat{γ}}_{1}^{(W)}$ (blue), corresponding to two situations of scenario $S_{4} : (γ_{1} = 0.8, p = 55 %)$ (top two panels) and $(γ_{1} = 0.8, p = 90 %)$ (bottom two panels) based on 1000 samples of size 300

Table 1

Optimal sample fractions and estimate values of the tail index γ₁ = 0.6 based on 1,000 samples of size 300 for the four scenarios with p = 0.55

	k*	${\hat{γ}}_{1}$	k*	${\hat{γ}}_{1}^{(B M N)}$	k*	${\hat{γ}}_{1}^{(W)}$
S1	44	0.600	41	0.599	40	0.600
S2	18	0.601	17	0.600	16	0.597
S3	21	0.601	20	0.601	19	0.599
S4	30	0.603	27	0.600	25	0.598

Table 2

Optimal sample fractions and estimate values of the tail index γ₁ = 0.6 based on 1,000 samples of size 300 for the four scenarios with p = 0.9

	k*	${\hat{γ}}_{1}$	k*	${\hat{γ}}_{1}^{(B M N)}$	k*	${\hat{γ}}_{1}^{(W)}$
S1	82	0.610	82	0.611	82	0.611
S2	37	0.640	37	0.640	37	0.640
S3	46	0.633	37	0.625	37	0.625
S4	52	0.610	52	0.610	52	0.610

Table 3

Optimal sample fractions and estimate values of the tail index γ₁ = 0.8 based on 1,000 samples of size 300 for the four scenarios with p = 0.55

	k*	${\hat{γ}}_{1}$	k*	${\hat{γ}}_{1}^{(B M N)}$	k*	${\hat{γ}}_{1}^{(W)}$
S1	59	0.799	57	0.800	54	0.799
S2	21	0.803	21	0.803	20	0.799
S3	24	0.802	22	0.798	22	0.801
S4	51	0.799	52	0.800	50	0.801

Table 4

Optimal sample fractions and estimate values of the tail index γ₁ = 0.8 based on 1,000 samples of size 300 for the four scenarios with p = 0.9

	k*	${\hat{γ}}_{1}$	k*	${\hat{γ}}_{1}^{(B M N)}$	k*	${\hat{γ}}_{1}^{(W)}$
S1	90	0.804	90	0.806	90	0.807
S2	34	0.845	34	0.846	34	0.846
S3	40	0.831	40	0.831	40	0.831
S4	71	0.814	71	0.814	71	0.815

4. Real data example

In this section, we give an application to the AIDS data set, available in the “DTDA” R package and the textbook of [28] (page 19) and already used by Ref. [1]. The data present the infection and induction times for n = 258 adults who were infected with HIV virus and developed AIDS by June 30, 1986. The variable of interest here is the time of induction T of the disease duration which elapses between the date of infection M and the date M + T of the declaration of the disease. The sample (T₁, M₁), …, (T_n, M_n) are taken between two fixed dates: “0” and “8”, i.e. between April 1, 1978, and June 30, 1986. The initial date “0” denotes an infection occurring in the three months: from April 1, 1978, to June 30, 1978. Let us assume that M and T are the observed rv’s, corresponding to the underlying rv’s M and T, given by the truncation scheme 0 ≤ M + T ≤ 8, which in turn may be rewritten into

0 \leq M \leq S,

(4.18)

where S≔8 − T. To work within the framework of the present paper, let us make the following transformations:

X ≔ \frac{1}{S + ϵ} and Y ≔ \frac{1}{M + ϵ},

(4.19)

where ϵ = 0.05 so that the two denominators be non-null. Thus, in view of $(4.18)$ ⁠, we have X ≤ Y, which means that X is randomly right-truncated by Y. Thereby, for the given sample (T₁, M₁), …, (T_n, M_n), from $(T, M)$ ⁠, the previous transformations produce a new one (X₁, Y₁), …, (X_n, Y_n) from $(X, Y)$ ⁠.

Let us now denote by F and G the df’s of the underling rv’s X and Y corresponding to the truncated rv’s X and Y, respectively. By using parametric likelihood methods, [29] fits both df’s of M and S by the two-parameter Weibull model, this implies that the df’s of F and G by may be fitted by two-parameter Fréchet model, namely $H_{(a . r)} (x) = \exp (- a^{r} x^{- r})$ ⁠, x > 0, a > 0, r > 0, hence both F and G are heavy-tailed. The estimated parameters corresponding to the fitting of df G are a₀ = 0.004 and r₀ = 2.1, see also [1] page 520. Thus, one may consider that df G is known and equals $G_{θ} = H_{(a_{0}, r_{0})}$ ⁠, where $θ = (a_{0}, r_{0})$ ⁠. By using the Thomas and Reiss algorithm, given above, we compute the optimal sample fraction k* corresponds to the tail index estimator ${\hat{γ}}_{1}$ of df F is γ₁. We find

k^{*} = 19, X_{n - k : n} = 0.356 and {\hat{γ}}_{1} = 0.917 .

(4.20)

The well-known Weissman estimator [30] of the high quantile, $q_{v} ≔ F^{- 1} (1 - v_{n})$ ⁠, corresponding to the underling df F is given by

{\hat{q}}_{v} ≔ X_{n - k : n} {(\frac{v}{{\bar{F}}_{n} (X_{n - k : n})})}^{- {\hat{γ}}_{1}},

where $v = 1 / (2 n)$ and F_n is the semiparametric estimator of df F of X given in $(1.11)$ ⁠. From the values $(4.20)$ ⁠, we get ${\hat{q}}_{v} = 0.061$ ⁠. Let us now compute the high quantile of T based on the original data, T₁, …, T_n. Recall that $P (X \geq q_{v}) = v$ and $X = 1 / (8 - T + ϵ)$ ⁠, this implies that $P (T \geq 1 / q_{v} - 8 + ϵ) = v$ ⁠, this means that 1/q_v − 8 + ϵ is the high quantile of T, which corresponds to the end-time t_end that we want to estimate. Thereby ${\hat{t}}_{end} = 1 / {\hat{q}}_{v} - 8 + 1 0^{- 2} = 1 / 0.061 - 8 + 1 0^{- 2} = 8.40$ ⁠, the value the end time of induction of AIDS is: 8 years, 4 months and 24 days.

5. Proofs

5.1 Proof of Theorem 2.1

Let us first notice that the semiparametric estimator of df F given in $(1.12)$ may be rewritten into

F_{n} (x; {\hat{θ}}_{n}) = P_{n} ({\hat{θ}}_{n}) \int_{0}^{x} \frac{d F_{n}^{*} (w)}{{\bar{G}}_{{\hat{θ}}_{n}} (w)},

(5.21)

and $1 / P_{n} (\hat{θ}) = \int_{0}^{\infty} d F_{n}^{*} (w) / {\bar{G}}_{{\hat{θ}}_{n}} (w)$ ⁠, where $F_{n}^{*} (w) ≔ n^{- 1} \sum_{i = 1}^{n} 1 (X_{i} \leq w)$ denotes the usual empirical df pertaining to the observed sample X₁, …, X_n. It is worth mentioning that by using the strong law of large numbers $P_{n} ({\hat{θ}}_{n}) \to P (θ)$ (almost surely) as n → ∞, where $P (θ) = 1 / \int_{0}^{\infty} d F^{*} (w) / {\bar{G}}_{θ} (w)$ (see e.g. Lemma 3.2 in Ref. [2]. On the other hand from equation $(1.2)$ ⁠, we deduce that $p = 1 / \int_{0}^{\infty} d F^{*} (w) / \bar{G} (w)$ ⁠, it follows that $p \equiv P (θ)$ because we already assumed that G ≡ G_θ. Next we use the distribution tail

\bar{F} (x) = P (θ) \int_{x}^{\infty} \frac{d F^{*} (w)}{{\bar{G}}_{θ} (w)},

(5.22)

and its empirical counterpart

{\bar{F}}_{n} (x; {\hat{θ}}_{n}) = P_{n} ({\hat{θ}}_{n}) \int_{x}^{\infty} \frac{d F_{n}^{*} (w)}{{\bar{G}}_{{\hat{θ}}_{n}} (w)} .

We begin by decomposing $k^{- 1 / 2} D_{n} (x; {\hat{θ}}_{n})$ ⁠, for x > 1, into the sum of

M_{n 1} (x) ≔ x^{- 1 / γ_{1}} \frac{{\bar{F}}_{n} (x X_{n - k : n}; {\hat{θ}}_{n}) - {\bar{F}}_{n} (x X_{n - k : n}; θ)}{\bar{F} (x X_{n - k : n})},

M_{n 2} (x) ≔ x^{- 1 / γ_{1}} \frac{{\bar{F}}_{n} (x X_{n - k : n}; θ) - \bar{F} (x X_{n - k : n})}{\bar{F} (x X_{n - k : n})},

M_{n 3} (x) ≔ - \frac{\bar{F} (x X_{n - k : n})}{{\bar{F}}_{n} (X_{n - k : n}; θ)} \frac{{\bar{F}}_{n} (X_{n - k : n}; θ) - \bar{F} (X_{n - k : n})}{\bar{F} (X_{n - k : n})},

M_{n 4} (x) ≔ (\frac{\bar{F} (x X_{n - k : n})}{{\bar{F}}_{n} (X_{n - k : n}; θ)} - x^{- 1 / γ_{1}}) \frac{{\bar{F}}_{n} (x X_{n - k : n}; θ) - \bar{F} (x X_{n - k : n})}{\bar{F} (x X_{n - k : n})}

and

M_{n 5} (x) ≔ \frac{\bar{F} (x X_{n - k : n})}{\bar{F} (X_{n - k : n})} - x^{- 1 / γ_{1}} .

Our goal is to provide a weighted weak approximation to the tail empirical process $D_{n} (x; {\hat{θ}}_{n}; γ_{1})$ ⁠. Let $ξ_{i} ≔ \bar{F} * (X_{i})$ ⁠, i = 1, …, n be a sequence of independent and identically distributed rv’s. Recall that both df’s F and G_θ are assumed to be continuous, this implies that F^* is continuous as well, therefore $P (ξ_{i} \leq u) = u$ ⁠, this means that ${(ξ_{i})}_{i = 1, n}$ are uniformly distributed on $(0,1)$ ⁠. Let us now define the corresponding uniform tail empirical process

α_{n} (s) ≔ \sqrt{k} (U_{n} (s) - s), for 0 \leq s \leq 1,

(5.23)

where

U_{n} (s) ≔ k^{- 1} \sum_{i = 1}^{n} 1 (ξ_{i} < k s / n),

(5.24)

denotes the tail empirical df pertaining to the sample ${(ξ_{i})}_{i = 1, n}$ ⁠. In view of Proposition 3.1 of [31], there exists a Wiener process W such that for every 0 ≤ ϵ < 1/2,

\sup_{0 \leq s < 1} s^{- ϵ} |α_{n} (s) - W (s)| \overset{P}{\to} 0, as n \to \infty .

(5.25)

Let us fix a sufficiently small 0 < ϵ < 1/2. We will successively show that, under the first-order conditions of regular variation $(1.6)$ ⁠, we have, uniformly on x ≥ 1, for all large n:

\sqrt{k} M_{n 2} (x) = \frac{γ}{γ_{1}} x^{1 / γ_{2}} W (t^{- 1 / γ}) + \frac{γ}{γ_{1}} \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ})

(5.26)

and

\sqrt{k} M_{n 3} (x) = - x^{- 1 / γ_{1}} (\frac{γ}{γ_{1}} W (1) + \frac{γ}{γ_{1}} \int_{1}^{\infty} W (t^{- γ_{2} / γ}) d t) + o_{P} (x^{- 1 / γ_{1} + ϵ}),

(5.27)

while

\sqrt{k} M_{n 1} (x) = o_{P} (x^{- 1 / γ_{1} + ϵ}), \sqrt{k} M_{n 4} (x) = o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}),

(5.28)

and

\sqrt{k} M_{n 5} (x) = x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k}) + o_{P} (x^{- 1 / γ_{1}}) .

(5.29)

Throughout the proof, without loss of generality, we assume that aϵ ≡ ϵ, for any constant a > 0. We point out that all the rest terms of the previous approximations are negligible in probability, uniformly on x > 1. Let us begin by the term $M_{n 1} (x)$ which may be made into

\begin{array}{l} \frac{x^{- 1 / γ_{1}}}{\bar{F} (x X_{n - k : n})} P_{n} ({\hat{θ}}_{n}) (\int_{x}^{\infty} \frac{d F_{n}^{*} (X_{n - k : n} w)}{{\bar{G}}_{\hat{θ}} (X_{n - k : n} w)} - \int_{x}^{\infty} \frac{d F_{n}^{*} (X_{n - k : n} w)}{{\bar{G}}_{θ} (X_{n - k : n} w)}) \\ = \frac{x^{- 1 / γ_{1}}}{\bar{F} (x X_{n - k : n})} P_{n} ({\hat{θ}}_{n}) \int_{x}^{\infty} (\frac{1}{{\bar{G}}_{\hat{θ}} (X_{n - k : n} w)} - \frac{1}{{\bar{G}}_{θ} (X_{n - k : n} w)}) d F_{n}^{*} (X_{n - k : n} w) . \end{array}

Applying the mean value theorem (for several variables) to function $θ \to 1 / {\bar{G}}_{θ} (\cdot)$ ⁠, yields

\frac{1}{{\bar{G}}_{\hat{θ}} (z)} - \frac{1}{{\bar{G}}_{θ} (z)} = \sum_{i = 1}^{d} ({\hat{θ}}_{i, n} - θ_{i}) \frac{{\bar{G}}_{\tilde{θ}}^{(i)} (z)}{{\bar{G}}_{\tilde{θ}}^{2} (z)}, for any z > 1,

where ${\tilde{θ}}_{n}$ is such that ${\tilde{θ}}_{i, n}$ is between θ_i and ${\hat{θ}}_{i, n}$ ⁠, for i = 1, …, d, therefore

M_{n 1} (x) = \frac{x^{- 1 / γ_{1}}}{\bar{F} (x X_{n - k : n})} P_{n} ({\hat{θ}}_{n}) \sum_{i = 1}^{d} ({\hat{θ}}_{i} - θ_{i}) \int_{x}^{\infty} \frac{{\bar{G}}_{\tilde{θ}}^{(i)} (X_{n - k : n} w)}{{\bar{G}}_{\tilde{θ}}^{2} (X_{n - k : n} w)} d F_{n}^{*} (X_{n - k : n} w) .

Recall that by assumptions $(1.6)$ and $[A 2]$ both ${\bar{G}}_{θ}$ and ${\bar{G}}_{θ}^{(i)}$ are regularly varying with the same index $(- 1 / γ_{2})$ and, on the other hand, $X_{n - k : n} \overset{P}{\to} \infty$ and w > 1 imply that $X_{n - k : n} w \overset{P}{\to} \infty$ ⁠. Applying Pooter’s inequalities $(1.4)$ ⁠, we get

\frac{{\bar{G}}_{\tilde{θ}} (X_{n - k : n} w)}{{\bar{G}}_{\tilde{θ}} (X_{n - k : n})} = (1 + o_{P} (1)) w^{- 1 / γ_{2} + ϵ} = \frac{{\bar{G}}_{\tilde{θ}}^{(i)} (X_{n - k : n} w)}{{\bar{G}}_{\tilde{θ}}^{(i)} (X_{n - k : n})},

it follows that

\begin{array}{l} M_{n 1} (x) & = (1 + o_{P} (1)) P_{n} ({\hat{θ}}_{n}) \frac{x^{- 1 / γ_{1}}}{{\bar{G}}_{\tilde{θ}} (X_{n - k : n}) \bar{F} (x X_{n - k : n})} \\ \times \sum_{i = 1}^{d} \frac{{\bar{G}}_{\tilde{θ}}^{(i)} (X_{n - k : n})}{{\bar{G}}_{\tilde{θ}} (X_{n - k : n})} |{\hat{θ}}_{i, n} - θ_{i}| \int_{x}^{\infty} w^{1 / γ_{2} - ϵ} d F_{n}^{*} (X_{n - k : n} w) . \end{array}

Under some regularity assumptions, [16] stated that $\sqrt{n} ({\hat{θ}}_{n} - θ)$ is asymptotically a centered multivariate normal rv, which implies that ${\hat{θ}}_{i, n} - θ_{i} = O_{P} (n^{- 1 / 2})$ and thus ${\hat{θ}}_{n} \overset{P}{\to} θ$ ⁠. On the other hand, by the law of large numbers $P_{n} (θ) \overset{P}{\to} P (θ)$ as n → ∞, then we may readily show that $P_{n} ({\hat{θ}}_{n}) \overset{P}{\to} P (θ)$ as n → ∞ as well. Note that since ${\hat{θ}}_{n}$ is a consistent estimator of θ then ${\tilde{θ}}_{n}$ is too. Then by using the fact that $X_{n - k : n} \overset{P}{\to} \infty$ and both conditions $[A 1]$ and $[A 3]$ ⁠, we show readily that

{(X_{n - k : n})}^{- ϵ} \frac{{\bar{G}}_{{\tilde{θ}}_{n}}^{(i)} (X_{n - k : n})}{{\bar{G}}_{{\tilde{θ}}_{n}} (X_{n - k : n})} \overset{P}{\to} 0, as n \to \infty,

and ${\bar{G}}_{θ} (X_{n - k : n}) / {\bar{G}}_{{\tilde{θ}}_{n}} (X_{n - k : n}) \overset{P}{\to} 1$ ⁠. In view of Lemma A1 in Ref. [7], we infer that $X_{n - k : n} = (1 + o_{P} (1)) {(k / n)}^{- γ}$ ⁠, thus

M_{n 1} (x) = {(k / n)}^{- ϵ γ} o_{P} (n^{- 1 / 2}) {\tilde{M}}_{n 1} (x),

where

{\tilde{M}}_{n 1} (x) ≔ \frac{x^{- 1 / γ_{1}} P (θ)}{{\bar{G}}_{θ} (X_{n - k : n}) \bar{F} (x X_{n - k : n})} \int_{x}^{\infty} w^{1 / γ_{2} - ϵ} d F_{n}^{*} (X_{n - k : n} w) .

Making use of representation $(5.22)$ ⁠, we write

\begin{align} {\tilde{M}}_{n 1} (x) & = x^{- 1 / γ_{1}} {(\int_{x}^{\infty} \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} w)} d \frac{F^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})})}^{- 1} \\ \times (\int_{x}^{\infty} w^{1 / γ_{2} - ϵ} d \frac{F_{n}^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})}) . \end{align}

(5.30)

Once again by using the routine manipulations of Potter’s inequalities, we show that the first integral in $(5.30)$ is equal to

(1 + o_{P} (1)) \int_{x}^{\infty} w^{1 / γ_{2} + ϵ / 2} d \frac{F^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})} .

An integration by parts to the previous integral yields

x^{1 / γ_{2} + ϵ / 2} \frac{{\bar{F}}^{*} (X_{n - k : n} x)}{{\bar{F}}^{*} (X_{n - k : n})} + (1 / γ_{2} + ϵ / 2) \int_{x}^{\infty} w^{1 / γ_{2} + ϵ / 2 - 1} \frac{{\bar{F}}^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})} d w .

Recall that from $(1.7)$ ⁠,we have ${\bar{F}}^{*} \in {R V}_{(- 1 / γ)}$ ⁠, then

\frac{{\bar{F}}^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})} = (1 + o_{P} (1)) w^{- 1 / γ + ϵ / 2},

uniformly on w > 1. Therefore, the previous quantity reduces into

(1 + o_{P} (1)) (1 + \frac{1 / γ_{2} + ϵ / 2}{- 1 / γ_{1} + ϵ}) x^{- 1 / γ_{1} + ϵ} .

Thereby the first expression between two brackets in (5.30) equals $O_{P} (x^{1 / γ_{1} - ϵ})$ ⁠. Let us consider the second factor in (5.30). By similar arguments as used for the first factor, we show that

x^{1 / γ_{2} + ϵ / 2} \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} x)}{{\bar{F}}^{*} (X_{n - k : n})} + (1 / γ_{2} + ϵ / 2) \int_{x}^{\infty} w^{1 / γ_{2} + ϵ / 2} \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})} d w,

multiplied by $(1 + o_{P} (1))$ ⁠, uniformly on x > 1. From Lemma 7.1, we have

\frac{{\bar{F}}_{n}^{*} (X_{n - k : n} w)}{{\bar{F}}^{*} (X_{n - k : n})} = O_{P} (w^{- 1 / γ + ϵ / 2}),

which implies that the previous expression equals $O_{P} (x^{- 1 / γ_{1} + ϵ})$ ⁠, thus ${\tilde{M}}_{n 1} (x) = O_{P} (x^{- 1 / γ + ϵ})$ and therefore

\sqrt{k} M_{n 1} (x) = {(k / n)}^{1 / 2 - ϵ γ} O_{P} (x^{- 1 / γ_{1} + ϵ}) .

By assumption k/n → 0, it follows that $\sqrt{k} M_{n 1} (x) = o_{P} (x^{- 1 / γ_{1} + ϵ})$ which meets the result of (5.30). Let now consider the second term $M_{n 2} (x)$ which may be rewritten into

\begin{array}{l} - x^{- 1 / γ_{1}} \frac{k / n}{{\bar{F}}^{*} (X_{n - k : n})} \frac{\bar{F} (X_{n - k : n})}{\bar{F} (x X_{n - k : n})} \frac{{\bar{G}}_{θ} (X_{n - k : n}) / {\bar{F}}^{*} (X_{n - k : n})}{\bar{F} (X_{n - k : n})} \\ \times \int_{x}^{\infty} \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} w)} d \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} w) - {\bar{F}}^{*} (X_{n - k : n} w)}{k / n} . \end{array}

In view of Potter’s inequalities, it is clear that

\frac{\bar{F} (X_{n - k : n})}{{\bar{F}}^{*} (X_{n - k : n}) / {\bar{G}}_{θ} (X_{n - k : n})} \overset{P}{\to} \frac{γ_{1}}{γ} P (θ)

and

\frac{\bar{F} (X_{n - k : n})}{\bar{F} (x X_{n - k : n})} \overset{P}{\to} x^{1 / γ_{1}} .

Smirnov’s lemma (see, e.g. Lemma 2.2.3 in Ref. [5] with the fact that ${\bar{F}}^{*} (X_{n - k : n}) \overset{d}{=} ξ_{k + 1 : n}$ imply that $\frac{n}{k} ξ_{k + 1 : n} \overset{P}{\to} 1$ ⁠, hence $\frac{n}{k} {\bar{F}}^{*} (X_{n - k : n}) = 1 + o_{P} (1)$ ⁠. Therefore,

M_{n 2} (x) = - (1 + o_{P} (1)) \frac{γ}{γ_{1}} \int_{x}^{\infty} \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} w)} d \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} w) - {\bar{F}}^{*} (X_{n - k : n} w)}{k / n} .

On the other hand, using an integration by parts yields

M_{n 2} (x) = (1 + o_{P} (1)) \frac{γ_{1}}{γ} (M_{n 2}^{(1)} (x) + M_{n 2}^{(2)} (x)),

where

M_{n 2}^{(1)} (x) ≔ \int_{x}^{\infty} \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} w) - {\bar{F}}^{*} (X_{n - k : n} w)}{k / n} d \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} w)}

and

M_{n 2}^{(2)} (x) ≔ \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)} \frac{{\bar{F}}_{n}^{*} (X_{n - k : n} x) - {\bar{F}}^{*} (x X_{n - k : n})}{k / n} .

By using the change of variables $t = {\bar{G}}_{θ} (X_{n - k : n}) / {\bar{G}}_{θ} (X_{n - k : n} w)$ ⁠, it is easy to verify that

M_{n 2}^{(1)} (x) = \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} \frac{n}{k} \{{\bar{F}}_{n}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n}) t^{- 1})) - {\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n}) t^{- 1}))\} d t .

Observe that

M_{n 2}^{(1)} (x) = \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} (U_{n} (ϑ_{n} (t; θ)) - ϑ_{n} (t; θ)) d t,

where $ϑ_{n} (t; θ) ≔ \frac{n}{k} {\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n}) t^{- 1}))$ and U_n are the tail empirical df given in (5.24). Thereby,

\sqrt{k} M_{n 2}^{(1)} (x) = \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} α_{n} (ϑ_{n} (t; θ)) d t,

with α_n being the tail empirical process defined in (5.23). Let us decompose the previous integral into

\begin{array}{l} \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} (α_{n} (ϑ_{n} (t; θ)) - W (ϑ_{n} (t; θ))) d t + \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} W (ϑ_{n} (t; θ)) d t \\ = S_{n} + R_{n} . \end{array}

By applying weak approximation (5.25), we get

S_{n} = o_{P} (1) \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} {(ϑ_{n} (t; θ))}^{1 / 2 - ϵ} d t .

Observe that ${\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n}))) = {\bar{F}}^{*} (X_{n - k : n})$ ⁠, thereby

ϑ_{n} (t; θ) = \frac{n}{k} {\bar{F}}^{*} (X_{n - k : n}) \frac{{\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n}) t^{- 1}))}{{\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - {\bar{G}}_{θ} (X_{n - k : n})))} .

It is easy to check that ${\bar{F}}^{*} (G_{θ}^{\leftarrow} (1 - \cdot)) \in R V (γ_{2} / γ)$ ⁠, then once again by means of Pooter’s inequality, we show that $ϑ_{n} (t; θ) = (1 + o_{P} (1)) t^{- γ_{2} / γ + ϵ}$ ⁠, therefore

S_{n} = o_{P} (1) \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{\infty} {(t^{- γ_{2} / γ + ϵ})}^{1 / 2 - ϵ} d t .

By using an elementary integration, we get

S_{n} = o_{P} (1) {(\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)})}^{(- γ_{2} / γ + ϵ) (1 / 2 - ϵ) + 1} = o_{P} (x^{\frac{1}{γ_{2}} - \frac{1}{2 γ} + ϵ}) .

By replacing γ by its by its expression given in (1.8), we end up with

S_{n} = o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) .

The term R_n may be decomposed into

\int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{x^{1 / γ_{2}}} W (ϑ_{n} (t; θ)) d t + \int_{x^{1 / γ_{2}}}^{\infty} W (ϑ_{n} (t; θ)) d t = R_{n 1} + R_{n 2} .

It is clear that

|R_{n 1}| < \{\sup_{t > \frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}} \frac{|W (ϑ_{n} (t; θ))|}{{(ϑ_{n} (t; θ))}^{ϵ}}\} \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{x^{1 / γ_{2}}} {(ϑ_{n} (t; θ))}^{ϵ} d t .

It is ready to check, by using the change of variables $ϑ_{n} (t; θ) = s$ ⁠, that the previous first factor between the curly brackets equals

\sup_{0 < s < \frac{n}{k} {\bar{F}}^{*} (X_{n - k : n} x; θ)} \frac{|W (s)|}{s^{ϵ}} < \sup_{0 < s < \frac{n}{k} {\bar{F}}^{*} (X_{n - k : n}; θ)} \frac{|W (s)|}{s^{ϵ}} .

From Lemma 3.2 in Ref. [31] $\sup_{0 < s \leq 1} s^{- δ} |W (s)| = O_{P} (1)$ ⁠, for any 0 < δ < 1/2, then since $n {\bar{F}}^{*} (X_{n - k : n}; θ) / k \overset{P}{\to} 1$ ⁠, as n → ∞, we infer that

\sup_{0 < s < \frac{n}{k} {\bar{F}}^{*} (X_{n - k : n}; θ)} s^{- ϵ} |W (s)| = O_{P} (1) .

for all large n. On the other hand, we already pointed out above that

ϑ_{n} (t; θ) = (1 + o_{P} (1)) t^{- γ_{2} / γ + ϵ},

which implies that the second factor is equal to

O_{P} (1) \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{x^{1 / γ_{2}}} {(t^{- γ_{2} / γ + ϵ})}^{ϵ} d t = O_{P} (1) \int_{\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)}}^{x^{1 / γ_{2}}} t^{- ϵ γ_{2} / γ + ϵ} d t,

which after integration yields

O_{P} (1) \{{(\frac{{\bar{G}}_{θ} (X_{n - k : n})}{{\bar{G}}_{θ} (X_{n - k : n} x)})}^{- ϵ γ_{2} / γ + ϵ + 1} - {(x^{- 1 / γ})}^{- ϵ γ_{2} / γ + ϵ + 1}\} .

Recall that from formula (1.8), we have γ₂/γ > 1, then by using the mean value theorem and Pooter’s inequalities, we get $R_{n 1} = o_{P} (x^{- ϵ})$ ⁠. The second term R_n2 may be decomposed into

R_{n 2} = \int_{x^{1 / γ_{2}}}^{\infty} (W (ϑ_{n} (t; θ)) - W (t^{- γ_{2} / γ})) d t + \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t .

From Proposition B.1.10 in Ref. [5], we have with high probability,

c_{n} (t; θ) : = |ϑ_{n} (t; θ) - t^{{- γ}_{2} / γ}| \leq ϵ t^{- γ_{2} / γ - ϵ}, as n \to \infty,

(5.31)

this means that $\sup_{x > 1} \sup_{t > x^{1 / γ_{2}}} c_{n} (t; θ) \overset{P}{\to} 0$ ⁠, as n → ∞. This implies by using Levy’s modulus of continuity of the Wiener process (see, e.g. Theorem 1.1.1 in Ref. [32]) that

|W (ϑ_{n} (t; θ)) - W (t^{- γ_{2} / γ})| \leq 2 \sqrt{c_{n} (t; θ) \log (1 / c_{n} (t; θ))},

with high probability. By using the fact that log s < ϵs^−ϵ, for s ↓ 0 together with inequality (5.31), we show that

|W (ϑ_{n} (t; θ)) - W (t^{- γ_{2} / γ})| < 2 ϵ t^{- (γ_{2} / γ - ϵ) / 2},

uniformly on $t > x^{1 / γ_{2}}$ ⁠, it follows that

|\int_{x^{1 / γ_{2}}}^{\infty} (W (ϑ_{n} (t; θ)) - W (t^{- γ_{2} / γ})) d t| = o_{P} (1) |\int_{x^{1 / γ_{2}}}^{\infty} t^{- (γ_{2} / γ - ϵ) / 2} d t| .

Recall that the assumption γ₁ < γ₂ together with equation 1/γ = 1/γ₁ + 1/γ₂, imply that $γ_{2} / (2 γ) > 1$ ⁠, thus $- (γ_{2} / γ - ϵ) / 2 + 1 < 0$ ⁠, therefore $|\int_{x^{1 / γ_{2}}}^{\infty} t^{- (γ_{2} / γ - ϵ) / 2} d t| = o_{P} (x^{- 1 / γ_{1} - ϵ})$ ⁠. Then we showed that

R_{n 1} = o_{P} (x^{- ϵ}) and R_{n 2} = \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (x^{- 1 / γ_{1} - ϵ}),

hence

\sqrt{k} M_{n 2}^{(1)} (x) = R_{n} + S_{n} = \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (x^{- 1 / γ_{1} - ϵ}) + o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) .

It is clear that

(- \frac{1}{γ_{1}} - ϵ) - (\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ) = - \frac{γ_{1} + γ_{2} + 4 ϵ γ_{1} γ_{2}}{2 γ_{1} γ_{2}} < 0 .

then

\sqrt{k} M_{n 2}^{(1)} (x) = \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) .

By using similar arguments, we end up with

\sqrt{k} M_{n 2}^{(2)} (x) = x^{1 / γ_{2}} W (t^{- 1 / γ}) + o_{P} (x^{- \frac{1}{γ_{1}} + ϵ}),

therefore, we omit further details. Finally, we have

\sqrt{k} M_{n 2} (x) = \frac{γ}{γ_{1}} x^{1 / γ_{2}} W (t^{- 1 / γ}) + \frac{γ}{γ_{1}} \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) .

Let us now focus on the term $M_{n 3} (x)$ ⁠. From the latter approximation, we infer that

\begin{align} \sqrt{k} M_{n 2} (1) & = \sqrt{k} \frac{{\bar{F}}_{n} (X_{n - k : n}; θ) - \bar{F} (X_{n - k : n})}{\bar{F} (X_{n - k : n})} \\ = \frac{γ}{γ_{1}} W (1) + \frac{γ}{γ_{1}} \int_{1}^{\infty} W (t^{- γ_{2} / γ}) d t + o_{P} (1), \end{align}

(5.32)

which implies that

\sqrt{k} \frac{{\bar{F}}_{n} (X_{n - k : n}; θ) - \bar{F} (X_{n - k : n})}{\bar{F} (X_{n - k : n})} = O_{P} (1) .

In other words, we have

\frac{{\bar{F}}_{n} (X_{n - k : n}; θ)}{\bar{F} (X_{n - k : n})} = 1 + O_{P} (k^{- 1 / 2}) .

(5.33)

The regular variation of $\bar{F} (\cdot)$ and (5.33) together imply that

\frac{\bar{F} (x X_{n - k : n})}{{\bar{F}}_{n} (X_{n - k : n}; θ)} = x^{- 1 / γ_{1}} + o_{P} (x^{- 1 / γ_{1} + ϵ}) .

(5.34)

By combining the results (5.32) and (5.34), we get

\sqrt{k} M_{n 3} (x) = - x^{- 1 / γ_{2}} (\frac{γ}{γ_{1}} W (1) + \frac{γ}{γ_{1}} \int_{1}^{\infty} W (t^{- γ_{2} / γ}) d t) + o_{P} (x^{- 1 / γ_{1} + ϵ}) .

For the fourth term $M_{n 4} (x)$ ⁠, we write

\sqrt{k} M_{n 4} (x) = (\frac{\bar{F} (x X_{n - k : n})}{{\bar{F}}_{n} (X_{n - k : n}; θ)} - x^{- 1 / γ_{1}}) (\sqrt{k} \frac{{\bar{F}}_{n} (x X_{n - k : n}; θ) - \bar{F} (x X_{n - k : n})}{\bar{F} (x X_{n - k : n})}) .

From (5.34) the first factor of the previous equation equals $o_{P} (x^{- 1 / γ_{1} + ϵ})$ ⁠. On the other hand, the change of variables $s = t^{- γ_{2} / γ}$ yields

\int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t = \frac{γ}{γ_{2}} \int_{0}^{x^{- 1 / γ}} s^{- γ / γ_{2} - 1} W (s) d s .

Since $\sup_{0 < s < 1} s^{- 1 / 2 + ϵ} |W (s)| = O_{P} (1)$ ⁠, then we easily show that

\int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t = O_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}),

it follows that $\sqrt{k} M_{n 2} (x) = O_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ})$ as well. Therefore,

\sqrt{k} \frac{{\bar{F}}_{n} (x X_{n - k : n}; θ) - \bar{F} (x X_{n - k : n})}{\bar{F} (x X_{n - k : n})} = x^{1 / γ_{1}} O_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) = O_{P} (x^{\frac{1}{2 γ} + ϵ}) .

Hence, we have

\sqrt{k} M_{n 4} (x) = o_{P} (x^{- 1 / γ_{1} + ϵ}) O_{P} (x^{\frac{1}{2 γ} + ϵ}) = o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ}) .

By assumption, $\bar{F}$ satisfies the second-order condition of regular variation (1.5), this means that for

\lim_{t \to \infty} \frac{\bar{F} (t x) / \bar{F} (t) - x^{- 1 / γ_{1}}}{A (t)} = x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}},

(5.35)

for any x > 0, where ρ₁ < 0 is the second-order parameter and A is $R V (ρ_{1} / γ_{1})$ ⁠. The uniform inequality corresponding to $(5.35)$ says: there exist t₀ > 0, such that for any t > t₀, we have

|\frac{\bar{F} (t x) / \bar{F} (t) - x^{- 1 / γ_{1}}}{A (t)} - x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}}| < ϵ x^{- 1 / γ_{1} + ρ_{1} / γ_{1} + ϵ},

see for instance assertion (2.3.23) of Theorem 2.3.9 in Ref. [5]. It is easy to check that the latter inequality implies that

\begin{array}{l} \sqrt{k} M_{n 5} (x) & = \sqrt{k} (\frac{\bar{F} (x X_{n - k : n})}{\bar{F} (X_{n - k : n})} - x^{- 1 / γ_{1}}) \\ = x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (X_{n - k : n}) + o_{P} (x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (X_{n - k : n})) . \end{array}

Recall that $a_{k} = F^{*} \leftarrow (1 - k / n)$ and notice that $X_{n - k : n} / a_{k} \overset{P}{\to} 1$ as n → ∞, then in view of the regular variation of A, we infer that $A (X_{n - k : n}) = (1 + o_{P} (1)) A (a_{k})$ ⁠. On the other hand, by assumption $\sqrt{k} A (a_{k})$ is asymptotically bounded, therefore

\sqrt{k} M_{n 5} (x) = x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k}) + o_{P} (x^{- 1 / γ_{1}}) .

To summarize, at this stage, we showed that

\begin{array}{l} D_{n} (x; \hat{θ}) & = \frac{γ}{γ_{1}} x^{1 / γ_{2}} W (t^{- 1 / γ}) + \frac{γ}{γ_{1}} \int_{x^{1 / γ_{2}}}^{\infty} W (t^{- γ_{2} / γ}) d t \\ - x^{- 1 / γ_{2}} (\frac{γ}{γ_{1}} W (1) + \frac{γ}{γ_{1}} \int_{1}^{\infty} W (t^{- γ_{2} / γ}) d t) \\ + x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k}) + ς (x), \end{array}

where $ς (x) ≔ o_{P} (x^{- 1 / γ_{1} + ϵ}) + o_{P} (x^{- 1 / γ_{1}}) + o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ})$ ⁠. By using a change of variables, we show that sum of the first three terms equals the Gaussian process $Γ (x; W)$ stated in Theorem 2.1. Recall that γ₁ < γ₂ and

\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ < 0,

then it is easy to verify that $ς (x) = o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + ϵ})$ ⁠. It follows that

\begin{array}{l} x^{ϵ} \{D_{n} (x; \hat{θ}) - Γ (x; W) - x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k})\} \\ = o_{P} (x^{\frac{1}{2} (\frac{1}{γ_{2}} - \frac{1}{γ_{1}}) + 2 ϵ}) = o_{P} (1), \end{array}

uniformly on x > 1, therefore

\sup_{x > 1} x^{ϵ} |D_{n} (x; \hat{θ}) - Γ (x; W) - x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k})| = o_{P} (1),

for any sample 0 < ϵ < 1/2, which completes the proof of Theorem 2.1.

5.2 Proof of Theorem 2.2

From the representation $(1.16)$ ⁠, we write

{\hat{γ}}_{1} - γ_{1} = T_{n 1} + T_{n 2} + T_{n 3},

where

T_{n 1} ≔ k^{- 1 / 2} \int_{1}^{\infty} x^{- 1} \{D_{n} (x; \hat{θ}; γ_{1}) - Γ (x; W) - x^{- 1 / γ_{1}} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} \sqrt{k} A (a_{k})\} d x

T_{n 2} ≔ k^{- 1 / 2} \int_{1}^{\infty} x^{- 1} Γ (x; W) d x

and

T_{n 3} ≔ - A (a_{k}) \int_{1}^{\infty} x^{- 1 / γ_{1} - 1} \frac{x^{ρ_{1} / γ_{1}} - 1}{ρ_{1} γ_{1}} d x .

Using Theorem 2.1 yields $T_{n 1} = o_{P} (k^{- 1 / 2}) \int_{1}^{\infty} x^{- 1 + ϵ} d x = o_{P} (k^{- 1 / 2}) = o_{P} (1)$ ⁠. Since $E |W (s)| \leq s^{1 / 2}$ ⁠, then it is easy to show that $\int_{1}^{\infty} x^{- 1} Γ (x; W) d x = O_{P} (1)$ ⁠, it follows that $T_{n 2} = O_{P} (k^{- 1 / 2}) = o_{P} (1)$ ⁠. Using an elementary integration, we get $T_{n 3} = A (a_{k}) / (1 - ρ_{1})$ which tends to zero as n → ∞, because a_k → ∞ and $|A|$ is regularly varying with negative index. Therefore, ${\hat{γ}}_{1} \overset{P}{\to} γ_{1}$ ⁠, as n → ∞ which gives the first result of Theorem. To establish the asymptotic normality, we write

\sqrt{k} ({\hat{γ}}_{1} - γ_{1}) = \sqrt{k} T_{n 1} + \sqrt{k} T_{n 2} + \sqrt{k} T_{n 3},

where

\sqrt{k} T_{n 1} = o_{P} (1), \sqrt{k} T_{n 2} = \int_{1}^{\infty} x^{- 1} Γ (x; W) d x

and

\sqrt{k} T_{n 3} = \frac{\sqrt{k} A (a_{k})}{1 - ρ_{1}} .

Note that $Γ (x; W)$ is a centered Gaussian process and by using the assumption $\sqrt{k} A (a_{k}) \to λ < \infty$ ⁠, we end up with

\sqrt{k} ({\hat{γ}}_{1} - γ_{1}) \overset{D}{\to} N (\frac{λ}{1 - ρ_{1}}, E {[\int_{1}^{\infty} x^{- 1} Γ (x; W) d x]}^{2}) .

By elementary calculations (we omit the details), we show that

E {[\int_{1}^{\infty} x^{- 1} Γ (x; W) d x]}^{2} = σ^{2} .

6. Conclusion

On the basis of a semiparametric estimator of the underlying distribution function, we proposed a new estimation method to the tail index of Pareto-type distributions for randomly right-truncated data. Compared with the existing ones, this estimator behaves well both in terms of bias and RMSE. A useful weak approximation of the corresponding tail empirical process allowed us to establish both the consistency and asymptotic normality of the proposed estimator.

The authors are indebted to the reviewers for their pertinent remarks and valuable suggestions that led to a real improvement of the paper.

References

1

Lagakos

SW

,

Barraj

LM

,

De Gruttola

V

.

Nonparametric analysis of truncated survival data, with applications to AIDS

.

Biometrika

.

1988

;

75

:

515

-

23

.

Google Scholar

Crossref

2

Wang

MC

.

A semiparametric model for randomly truncated data

.

J Amer Statist Assoc

.

1989

;

84

:

742

-

8

.

Google Scholar

Crossref

3

Lawless

JF

.

Statistical models and methods for lifetime data

. (2nd ed) .

Wiley Series in Probability and Statistics

;

2002

.

Google Scholar

Crossref

4

Gardes

L

,

Stupfler

G

.

Estimating extreme quantiles under random truncation

.

TEST

.

2015

;

24

:

207

-

27

.

Google Scholar

Crossref

5

de Haan

L

,

Ferreira

A

.

Extreme value theory: an introduction

.

Springer

;

2006

.

Google Scholar

Crossref

6

de Haan

L

,

Stadtmüller

U

.

Generalized regular variation of second order

.

J Aust Math. Soc. (Series A)

.

1996

;

61

:

381

-

95

.

Google Scholar

Crossref

7

Benchaira

S

,

Meraghni

D

,

Necir

A

.

Tail product-limit process for truncated data with application to extreme value index estimation

.

Extremes

.

2016a

;

19

:

219

-

51

.

Google Scholar

Crossref

8

Hill

BM

.

A simple general approach to inference about the tail of a distribution

.

Ann Statist

.

1975

;

3

:

1163

-

74

.

Google Scholar

Crossref

9

Benchaira

S

,

Meraghni

D

,

Necir

A

.

On the asymptotic normality of the extreme value index for right-truncated data

.

Statist Probab Lett

.

2015

;

107

:

378

-

84

.

Google Scholar

Crossref

10

Worms

J

,

Worms

R

.

A Lynden-Bell integral estimator for extremes of randomly truncated data

.

Statist Probab Lett

.

2016

;

109

:

106

-

17

.

Google Scholar

Crossref

11

Lynden-Bell

D

.

A method of allowing for known observational selection in small samples applied to 3CR quasars

.

Monthly Notices Roy Astron Soc

.

1971

;

155

:

95

-

118

.

Google Scholar

Crossref

12

Woodroofe

M

.

Estimating a distribution function with truncated data

.

Ann Statist

.

1985

;

13

:

163

-

77

.

Google Scholar

13

Benchaira

S

,

Meraghni

D

,

Necir

A

.

Kernel estimation of the tail index of a right-truncated Pareto-type distribution

.

Statist Probab Lett

.

2016b

;

119

:

186

-

93

.

Google Scholar

Crossref

14

Haouas

N

,

Necir

A

,

Brahimi

B

.

Estimating the second-order parameter of regular variation and bias reduction in tail index estimation under random truncation

.

J Stat Theor Pract

.

2019

;

13

:

110

-

144

.

Google Scholar

Crossref

15

Haouas

N

,

Necir

A

,

Meraghni

D

,

Brahimi

B

.

A Lynden-Bell integral estimator for the tail index of right-truncated data with a random threshold

.

Afr Stat

.

2018

;

12

:

1159

-

70

.

Google Scholar

16

Andersen

EB

.

Asymptotic properties of conditional maximum-likelihood estimators

.

J Roy Statist Soc Ser B

.

1970

;

32

:

283

-

301

.

Google Scholar

Crossref

17

Moreira

C

,

de Uña-Álvarez

J

.

A semiparametric estimator of survival for doubly truncated data

.

Stat Med

.

2010

;

29

:

3147

-

59

.

Google Scholar

Crossref

PubMed

18

Bilker

WB

,

Wang

MC

.

Asemiparametric extension of the Mann–Whitney test for randomly truncated data

.

Biometrics

.

1996

;

52

:

10

-

20

.

Google Scholar

Crossref

PubMed

19

Li

G

,

Qin

J

,

Tiwari

RC

.

Semiparametric likelihood ratio-based inferences for truncated data

.

J Amer Statist Assoc

.

1997

;

92

:

236

-

45

.

Google Scholar

Crossref

20

Moreira

C

,

de Uña-Álvarez

J

,

Van Keilegom

I

.

Goodness-of-fit tests for a semiparametric model under random double truncation

.

Comput Statist

.

2014

;

29

:

1365

-

137

.

Google Scholar

Crossref

21

Qin

J

,

Wang

MC

.

Semiparametric analysis of truncated data

.

Lifetime Data Anal

.

2001

;

7

(

3

):

225

-

42

.

Google Scholar

Crossref

PubMed

22

Shen

PS

.

Semiparametric analysis of doubly truncated data

.

Comm Statist Theor Methods

.

2010

;

39

:

3178

-

90

.

Google Scholar

Crossref

23

Shen

PS

,

Hsu

H

.

Conditional maximum likelihood estimation for semiparametric transformation models with doubly truncated data

.

Comput Statist Data Anal

.

2020

;

144

:

15

: 106862.

Google Scholar

Crossref

24

Drees

H

,

de Haan

L

,

Li

D

.

Approximations to the tail empirical distribution function with application to testing extreme value conditions

.

J Statist Plann Inference

.

2006

;

136

:

3498

-

538

.

Google Scholar

Crossref

25

Koning

AJ

,

Peng

L

.

Goodness-of-fit tests for a heavy tailed distribution

.

J Statist Plann Inference

.

2008

;

138

:

3960

-

81

.

Google Scholar

Crossref

26

Caeiro

F

,

Gomes

MI

.

Threshold selection in extreme value analysis

.

Chapter in

. In:

Dey

D

,

Yan

J

, (Eds).

Extreme value modeling and risk analysis: methods and applications

.

[PubMed]

.

Chapman-Hall/CRC

;

2015

. p.

69

-

87

.

Google Scholar

27

Reiss

RD

,

Thomas

M

.

Statistical analysis of extreme values with applications to insurance, finance, hydrology and other fields

. (3rd ed) .

Basel, Boston, Berlin

:

Birkhäuser Verlag

;

2007

.

Google Scholar

28

Klein

JP

,

Moeschberger

S

.

Survival analysis: techniques for censored and truncated data

.

Berlin

:

Springer

;

1997

. doi:

https://doi.org/10.1007/978-1-4757-2728-9

.

Google Scholar

Crossref

29

Lui

KJ

,

Lawrence

DN

,

Morgan

WM

,

Peterman

TA

,

Haverkos

HH

,

Breakman

DJ

.

A model-based approach for estimating the mean incubation period of transfusion-associated acquired immunodeficiency syndrome

.

Proc Nat Acad Sc

.

1986

;

83

:

2913

-

7

.

Google Scholar

Crossref

30

Weissman

I

.

Estimation of parameters and large quantiles based on the k largest observations

.

J Am Statist Assoc

.

1978

;

73

:

812

-

15

.

Google Scholar

31

Einmahl

JHJ

,

de Haan

L

,

Li

D

.

Weighted approximations of tail copula processes with application to testing the bivariate extreme value condition

.

Ann Statist

.

2006

;

34

:

1987

-

2014

.

Google Scholar

32

Csörgő

M

,

Révész

P

.

Strong approximations in probability and statistics. Probability andMathematical statistics

.

New York, London

:

Academic Press [Harcourt Brace Jovanovich, Publishers]

;

1981

.

Google Scholar

33

Shorack

GR

,

Wellner

JA

.

Empirical processes with applications to statistics

.

New York

:

Wiley

;

1986

.

Google Scholar

Semiparametric tail-index estimation for randomly right-truncated heavy-tailed data

1. Introduction

2. Main results

3. Simulation study

4. Real data example

5. Proofs

5.1 Proof of Theorem 2.1

5.2 Proof of Theorem 2.2

6. Conclusion

References

Further reading

Appendix

Email Alerts

Cited By

Semiparametric tail-index estimation for randomly right-truncated heavy-tailed data Open Access

1. Introduction

2. Main results

3. Simulation study

4. Real data example

5. Proofs

5.1 Proof of Theorem 2.1

5.2 Proof of Theorem 2.2

6. Conclusion

References

Further reading

Appendix

Email Alerts

Suggested Reading

Related Chapters

Recommended for you

Cited By

Semiparametric tail-index estimation for randomly right-truncated heavy-tailed data