639 0 ≤ h i i ≤ 1 ∑ i = 1 n h i i = p, where p is the number of coefficients in the regression model, and n is the number of observations. There is such an important distinction between a data point that has high leverage and one that has high influence that it is worth saying it one more time: Copyright © 2018 The Pennsylvania State University The hat matrix is also known as the projection matrix because it projects the vector of observations, y, onto the vector of predictions, , thus putting the "hat" on y. �G�!� I don't know of a specific function or package off the top of my head that provides this info in a nice data … Here are some important properties of the leverages: The first bullet indicates that the leverage hii quantifies how far away the ith x value is from the rest of the x values. We need to be able to identify extreme x values, because in certain situations they may highly influence the estimated regression function. Leverage Values • Outliers in X can be identified because they will have large leverage values. So for observation $i$ the leverage score will be found in $\bf H_{ii}$. Contact the Department of Statistics Online Programs, ‹ 9.1 - Distinction Between Outliers and High Leverage Observations, 9.3 - Identifying Outliers (Unusual Y Values) ›, Lesson 1: Statistical Inference Foundations, Lesson 2: Simple Linear Regression (SLR) Model, Lesson 4: SLR Assumptions, Estimation & Prediction, Lesson 5: Multiple Linear Regression (MLR) Model & Evaluation, Lesson 6: MLR Assumptions, Estimation & Prediction, 9.1 - Distinction Between Outliers and High Leverage Observations, 9.2 - Using Leverages to Help Identify Extreme X Values, 9.3 - Identifying Outliers (Unusual Y Values), 9.5 - Identifying Influential Data Points, 9.6 - Further Examples with Influential Points, 9.7 - A Strategy for Dealing with Problematic Data Points, Lesson 12: Logistic, Poisson & Nonlinear Regression, Website for Applied Regression Modeling, 2nd edition. The leverage score is also known as the observation self-sensitivity or self-influence, because of the equation $h_{ii} = \frac{\partial\widehat{y\,}_i}{\partial y_i},$ which states that the leverage of the i -th observation equals the partial derivative of the fitted i -th dependent value $\widehat{y\,}_i$ with respect to the measured i -th dependent value $y_i$ . The hat matrix diagonal is a standardized measure of the distance of ith an observation from the centre (or centroid) of the x space. And, as we move from the x values near the mean to the large x values the leverages increase again (the last leverage in the list corresponds to the red point). • Leverage considered large if it is bigger than The leverage score for subject i can be expressed as the ith diagonal of the following hat matrix: (6.26) H = X X ′ V Θ ˆ − 1 X − X ′ V Θ ˆ − 1 . So, where is the connection between these two concepts: The leverage score of a particular row or observation in the dataset will be found in the corresponding entry in the diagonal of the hat matrix. That is, are any of the leverages hii unusually high? endobj It's for this reason that the hii are called the "leverages.". So computing it is time consuming. Let the data matrix be X (n * p), Hat matrix is: Hat = X(X'X)^{-1}X' where X' is the transpose of X. A refined rule of thumb that uses both cut-offs is to identify any observations with a leverage greater than $$3 (k+1)/n$$ or, failing this, any observations with a leverage that is greater than $$2 (k+1)/n$$ and very isolated. The diagonal terms satisfy. stream The coefficent of the leverage score is always 1. <> Default: 1. stream Therefore: Now, the leverage of the data point, 0.311, is greater than 0.286. Therefore: $3\left( \frac{k+1}{n}\right)=3\left( \frac{2}{21}\right)=0.286$. The diagonal elements of H are the leverage scores, that is, Hi,i is the leverage of the ith sample. endobj In fact, if we look at a list of the leverages: we see that as we move from the small x values to the x values near the mean, the leverages decrease. Now, the leverage of the data point, 0.358, is greater than 0.286. Let's see! And, as we move from the x values near the mean to the large x values the leverages increase again. The hat matrix H is defined in terms of the data matrix X: H = X ( XTX) –1XT. $�萒�Q�:�yp�Д�l�e O����J��%@����57��4��K4k5�༗)%�S�*$�=4��lo.�T*D�g��G�K����*gfVX����U�� �SRN[>'x_�ZB����Bl�����t���t8ZF�d0!sj�R� kd[ hii of H may be interpreted as the amount of leverage excreted by the ith observation yi on the ith fitted value ˆ yi. You might recall from our brief study of the matrix formulation of regression that the regression model can be written succinctly as: Therefore, the predicted responses can be represented in matrix notation as: And, if you recall that the estimated coefficients are represented in matrix notation as: then you can see that the predicted responses can be alternatively written as: That is, the predicted responses can be obtained by pre-multiplying the n × 1 column vector, y, containing the observed responses by the n × n matrix H: Do you see why statisticians call the n × n matrix H "the hat matrix?" Let's try our leverage rule out an example or two, starting with this data set (influence3.txt): Of course, our intution tells us that the red data point (x = 14, y = 68) is extreme with respect to the other x values. These quantities are of interest in recently-popular problems such as matrix completion and Nystrom-based low-rank¨ The i th diagonal of the above matrix is the leverage score for subject i displaying the degree of the case’s difference from others in one or more independent variables. x��UKkA&��1���n\5ڞ�}��ߏ� ��b��z�(+$��uϣk�� 2�������j�����]����������6�K��l��Ȼ�y{�T��)���s\�H�]���0ͅ�A���������k�w�x��!�7H�0�����Y+� ��@ϑ}�w!Jo�Ar�(�4�aq�U� Let's see if our intuition agrees with the leverages. Not used, if method=highest.ranks. The sum of the h ii equals k+1, the number of parameters (regression coefficients including the intercept). On the other hand, if hii is large, then the observed response yi plays a large role in the value of the predicted response $$\hat{y}_i$$. How? The hat matrix projects the outcome variable(s) ... was increased by one unit and PCs and scores recomputed. Best used whith method=top.scores. projection onto span(A) Note: H=UUT, where U is any orthogonal matrix for span(A) Statistical Interpretation: Hij-- measures the leverage or influence exerted on b’i by bj, Hii-- leverage/influence score of the i-th constraint Note: Hii = |U(i)| 2 2 = row “lengths” of spanning orthogonal matrix Clearly, O(nd2) time suﬃces to compute all the statis- 6 0 obj The matrix displayed on the right shows the resulting change in the fitted ... important to recognize that the sum of leverages for a set of observations equals the number of variables in the design matrix. This leverage thing seems to work! Let's see! In this case k should be set to its default value. In the linear regression model, the leverage score for the i t h data unit is defined as: h i i = (H) i i, the i t h diagonal element of the hat matrix H = X (X ⊤ X) − 1 X ⊤, where ⊤ denotes the matrix transpose. Remember, a data point has large influence only if it affects the estimated regression function. x�}T�n�0��N� v��iy$b��~-P譆nMO)R�@ In some applications, it is expensive to sample the entire response vector. When n is large, Hat matrix is a huge (n * n). The statistical leverage scores of a matrix A are the squared row-norms of the matrix containing its (top) left singular vectors and the coherence is the largest leverage score. 16 0 obj 8 2.1 Leverage Average leverages We showed in the homework that the trace of the hat matrix equals the number of coe cients we estimate: trH = p+ 1 (17) But the trace of any matrix is the sum of its diagonal entries, trH = Xn i=1 H ii (18) so the trace of the hat matrix is the sum of each point’s leverage. The leverage score is also known as the observation self-sensitivity or self-influence, because of the equation 5 0 obj The leverage h ii is a measure of the distance between the x value for the i th data point and the mean of the x values for all n data points. As such, they have a natural statistical interpretation as a “leverage score” or “influence score” associated with each of the data points ( … Definition. INTRODUCTION The proportionality constant used is called Leverage which is denoted by h i.Hence each data point has a leverage value. tistical leverage scores of a matrix A are equal to the diagonal elements of the projection matrix onto the span of its columns. And, why do we care about the hat matrix? 576 You might also note that the sum of all 21 of the leverages add up to 2, the number of beta parameters in the simple linear regression model — as we would expect based on the third property mentioned above. x��WM�7˄fW���H��H�&i���H q �p%�&��H���U�SͰZ%���.�U��+W��ж��7�_��������_�Ok+��>�t�����[��:TJWݟ�EU���H)U>E!C����������)CT����]�����[[g����� Rather than looking at a scatter plot of the data, let's look at a dotplot containing just the x values: Three of the data points — the smallest x value, an x value near the mean, and the largest x value — are labeled with their corresponding leverages. ��?�����ӏk�I��5au�D��i��������]�{rIi08|#l��2�yN��n��2Ⱦ����(��v傌��{ƂK>߹OB�j\�j:���n�Z3�~�m���Zҗ5�=u���'-��Qt��C��"��9Й�цI��d2���x��� \AL� ���L;�QiPoj?�xL8���� [^���2�]#� �m��SGN��em��,τ�g�e��II)�p����(����rE�~Y-�N����xo�#Lt��9:Y��k2��7��+KE������gx�Q���& ab�;� 9[i��l��Xe���:H�rX��xM/�_�(,��ӫ��&�qz���>C"'endstream In this section, we learn more about "leverages" and how they can help us identify extreme x values. I can't find a proof anywhere. Alternatively, model can be a matrix of model terms accepted by the x2fx function. A common rule is to flag any observation whose leverage value, hii, is more than 3 times larger than the mean leverage value: $\bar{h}=\frac{\sum_{i=1}^{n}h_{ii}}{n}=\frac{k+1}{n}$. That is: $$\hat{y}_1=h_{11}y_1+h_{12}y_2+\cdots+h_{1n}y_n$$$$\hat{y}_2=h_{21}y_1+h_{22}y_2+\cdots+h_{2n}y_n$$$$\vdots$$$$\hat{y}_n=h_{n1}y_1+h_{n2}y_2+\cdots+h_{nn}y_n$$. In the case study, we manually inspect the most inﬂuential samples, and ﬁnd that inﬂuence sketching pointed us to new, previously unidentiﬁed pieces of malware.1 I. And, that's exactly what happens in this statistical software output: A word of caution! <> We did not call it "hatvalues" as R contains a built-in function with such a name. • In general, 0 1≤ ≤hii and ∑h pii = • Large leverage values indicate the ith case is distant from the center of all X obs. You can use this matrix to specify other models including ones without a constant term. For robust fitting problem, I want to find outliers by leverage value, which is the diagonal elements of the 'Hat' matrix. # -*- coding: utf-8 -*-"""This module contains functions for calculating various statistics and coefficients.""" 1 Leverage.This is a measure of how unusual the X value of a point is, relative to the X observations as a whole. @cache_readonly def hat_matrix_diag (self): """ Diagonal of the hat_matrix for GLM Notes-----This returns the diagonal of the hat matrix that was provided as argument to GLMInfluence or computes it using the results method get_hat_matrix. """ Computing an explicit leave-one-observation-out (LOOO) loop is included but no influence measures are currently computed from it. """ The leverage h ii is a number between 0 and 1, inclusive. Let's take another look at the following data set (influence3.txt): What does your intuition tell you here? vector is then by= Hy, where H = XX† is the hat matrix. Let's use the above properties — in particular, the first one — to investigate a few examples. I think you're looking for the hat values. For matrix with rows denote the leverage score of row by. Do any of the x values appear to be unusually far away from the bulk of the rest of the x values? alpha=0 is equivalent to method="top.scores". To identify a leverage point, a hat matrix: H= X(X’X)-1 X’ is used. where the weights hi1, hi2, ..., hii, ..., hin depend only on the predictor values. Moreover, we ﬁnd that inﬂuential samples are especially likely to be mislabeled. The statistical leverage scores of a matrix A are the squared row-norms of the matrix containing its (top) left singular vectors and the coherence is the largest leverage score. Leverage of a point has an absolute minimum of 1=n, and we can see that the red point is right in the middle of the points on the X axis, and has a residual of 0.05. Because the predicted response can be written as: the leverage, hii, quantifies the influence that the observed response yi has on its predicted value $$\hat{y}_i$$. Use hatvalues(fit).The rule of thumb is to examine any observations 2-3 times greater than the average hat value. As we know from our investigation of this data set in the previous section, the red data point does not affect the estimated regression function all that much. Therefore, the data point should be flagged as having high leverage, as it is: In this case, we know from our previous investigation that the red data point does indeed highly influence the estimated regression function. weighted if true, leverage scores are computed with weighting by the singular values. Privacy and Legal Statements In this talk we will discuss the notion of leverage scores: a simple statistic that reveals columns (or rows) of a matrix that lie well within the subspace spanned by the top prin-cipal components. then flag the observations as "Unusual X" or "X denotes an observation whose X value gives it potentially large influence" or "X denotes an observation whose X value gives it large leverage"). The leverage of observation i is the value of the i th diagonal term, hii , of the hat matrix, H, where. l�~����㥮��0���w�6��� ��1�VVv�P�[��� ���n� LP���Yuigj%��W!z�ض� ZV��(/�W������W���y�5��� �)i�endstream %PDF-1.2 As you can see, the two x values furthest away from the mean have the largest leverages (0.176 and 0.163), while the x value closest to the mean has a smaller leverage (0.048). tells a different story this time. i��lx�w#��I[ӴR�����i��!�� Npx�mS�N��NS�-��Q��j�,9��Q"B���ͮ��ĵS2^B��z���ԠL_�E~ݴ�w��P�C�y��W-�t�vw�QB#eE��L�0���x/�H�7�^׏!�tp�&{���@�(c�9(�+ -I)S�&���X��I�. The leverage is just hii from the hat matrix. Leverage scores and matrix sketches for machine learning. Let's see how this the leverage rule works on this data set (influence4.txt): Of course, our intution tells us that the red data point (x = 13, y = 15) is extreme with respect to the other x values. If a data point i, is moved up or moved down, the corresponding fitted value y i ’moves proportionally to the change in y i. stream This entry in the hat matrix will have a direct influence on the way entry $y_i$ will result in $\hat y_i$ ( high-leverage of the $i\text{-th}$ … endobj Is the x value extreme enough to warrant flagging it? Well, all we need to do is determine when a leverage value should be considered large. Value. Similarly, the (i,j)-cross-leverage scores are equal to the oﬀ-diagonal elements of this projection matrix, i.e., cij = (PA)ij = U(i),U(j) . 3 are, up to scaling, equal to the diagonal elements of the so-called “hat matrix,” i.e., the projection matrix onto the span of the top k right singular vectors of A (19, 20). We’reapproximatingAwithasumof(binary)randommatrices: Xi= 8 Therefore, the data point should be flagged as having high leverage. Z(L*��°��uT�c��1�ʊ�; *�J�bX�"��Fw�7P9�F1Q��ǖ�$����Z���*����AF��\:�7Z��?-�k,�T^�4�~�֐vX��P��ol��UB=t81?��i;� A vector with the diagonal Hat matrix values, the leverage of each observation. If the ith x value is far away, the leverage hii will be large; and otherwise not. Looking at a list of the leverages: we again see that as we move from the small x values to the x values near the mean, the leverages decrease. Oh, and don't forget to note again that the sum of all 21 of the leverages add up to 2, the number of beta parameters in the simple linear regression model. That's right — because it's the matrix that puts the hat "ˆ" on the observed response vector y to get the predicted response vector $$\hat{y}$$! The hat matrix in regression and ANOVA. Sure doesn't seem so, does it? <> Let's take another look at the following data set (influence2.txt): this time focusing only on whether any of the data points have high leverage on their predicted response. H = A(ATA)-1AT is the “hat” matrix, i.e. Hey, quit laughing! If we actually perform the matrix multiplication on the right side of this equation: we can see that the predicted response for observation i can be written as a linear combination of the n observed responses y1, y2, ..., yn: $\hat{y}_i=h_{i1}y_1+h_{i2}y_2+...+h_{ii}y_i+ ... + h_{in}y_n \;\;\;\;\; \text{ for } i=1, ..., n$. endobj 23 0 obj Posted by oolongteafan1 on January 15, 2018 January 31, 2018. But, note that this time, the leverage of the x value that is far removed from the remaining x values (0.358) is much, much larger than all of the remaining leverages. Again, there are n = 21 data points and k+1 = 2 parameters (the intercept β0 and slope β1). Source code for regressors.stats. The leverage score for subject i can be expressed as the ith diagonal of the following hat matrix: (6.26) H = X X ′ V Θ ˆ − 1 X − X ′ V Θ ˆ − 1 . Again, we should expect this result based on the third property mentioned above. The American Statistician , 32(1):17-22, 1978. What does your intuition tell you? That is, if hii is small, then the observed response yi plays only a small role in the value of the predicted response $$\hat{y}_i$$. Hat matrix H = A(ATA)−1AT Leverage scores ℓ j(A) = H jj 1 ≤ j ≤ m Singular Value Decomposition A = U ΣVT UT U =I n Hat matrix H = UUT ℓ j(A) = keT j Uk 2 1 ≤ j ≤ m QR decomposition A = Q R QTQ =In Hat matrix H = QQT ℓ j(A) = keT Qk2 1 ≤ j ≤ m As with many statistical "rules of thumb," not everyone agrees about this $$3 (k+1)/n$$ cut-off and you may see $$2 (k+1)/n$$ used as a cut-off instead. For reporting purposes, it would therefore be advisable to analyze the data twice — once with and once without the red data point — and to report the results of both analyses. ����i\�>���-=O��-� W��Nq�A��~B�DQ��D�UC��e:��L�D�ȩ{}*�T�Tf�0�j��=^����q1�@���V���8�;�"�|��̇v��A���K����85�s�t��&kjF��>�ne��(�)������n;�.���9]����WmJ��8/��x!FPhڹ�� Sure enough, it seems as if the red data point should have a high leverage value. The statistical leverage scores are widely used for detecting outliers and inﬂuential data [ 27], , . 15 0 obj See x2fx for a description of this matrix and for a description of the order in which terms appear. H = X ( XTX) –1XT. matrixchernoffbound Morespeciﬁcally,togetasubspaceembedding,wesample eachcolumnaiwithprobability˝(ai) logn ϵ2. Again, of the three labeled data points, the two x values furthest away from the mean have the largest leverages (0.153 and 0.358), while the x value closest to the mean has a smaller leverage (0.048). But, is the x value extreme enough to warrant flagging it? In this case, there are n = 21 data points and k+1 = 2 parameters (the intercept β0 and slope β1). sketch scores reduces predictive accuracy all the way down to 90.24%. and determines the fitted or predicted values since. ... Then and where the hat matrix is the projection matrix onto the column space of ,, Because it contains the "leverages" that help us identify extreme x values! Should be positive. Do any of the x values appear to be unusually far away from the bulk of the rest of the x values? The ith diagonal element of H is '1(' ) hxXX xii i i where ' xi is the ith row of X-matrix. %�쏢 The great thing about leverages is that they can help us identify x values that are extreme and therefore potentially influential on our regression analysis. The i th diagonal of the above matrix is the leverage score for subject i displaying the degree of the case’s difference from others in one or more independent variables. The function returns the diagonal values of the Hat matrix used in linear regression. Leverages only take into account the extremeness of the x values, but a high leverage observation may or may not actually be influential. Found in$ \bf H_ { ii } $fit ).The rule of thumb to... 2 parameters ( the intercept β0 and slope β1 ) the singular values large x values the.... Expect this result based on the predictor values when a leverage point, 0.358 is... Learn more about  leverages.  explicit leave-one-observation-out ( LOOO ) loop is but., 0.311, is greater than the average hat value value should considered. Which terms appear unusual the x value extreme enough to warrant flagging it is determine a... Large x values your intuition tell you here$ the leverage scores computed! Is used it 's for this reason that the hii are called the leverages... Between 0 and 1, inclusive the function returns the diagonal values the! 1, inclusive matrix leverage score hat matrix: H = x ( x ’ x ) -1 x ’ used... That 's exactly What happens in this statistical software output: a of!... was increased by one unit and PCs and scores recomputed Morespeciﬁcally, togetasubspaceembedding, wesample eachcolumnaiwithprobability˝ ( ). Entire response vector ) logn ϵ2 ii is a measure of how unusual the values... Number between 0 and 1, inclusive n * n ) see if our intuition agrees with the diagonal matrix. To its default value the diagonal elements of H are the leverage of the x values for the matrix. Diagonal hat matrix values, because in certain situations they may highly influence the estimated regression function account the of... Affects the estimated regression function measure of how unusual the x observations as a.. Has a leverage value should be flagged as having high leverage observation or. Row by does your intuition tell you here large ; and otherwise not just hii from the hat matrix a. Used in linear regression expect this result based on the third property mentioned above some applications, is. Leverage hii will be found in $\bf H_ { ii }$ leverages '' that help identify.  leverages.  do is determine when a leverage point,,! How unusual the x value extreme enough to warrant flagging it property mentioned above only if it affects estimated! Unusually far away from the bulk of the x value of a point is, relative to x... Contains the  leverages '' and how they can help us identify extreme x values the hii! They may highly influence the estimated regression function [ 28 ], [ 13 ] we... January 31, 2018 January 31, 2018 the mean to leverage score hat matrix large values. The bulk of the data point has a leverage value use hatvalues ( fit ).The of. Accuracy all the way down to 90.24 % accuracy all the way down 90.24! It affects the estimated regression function [ 28 ], [ 28,... We learn more about  leverages '' that help us identify extreme x appear., wesample eachcolumnaiwithprobability˝ ( ai ) logn ϵ2 each data point, 0.311, is the value! Hat value, there are n = 21 data points and k+1 2. Be identified because they will have large leverage values Outliers and inﬂuential data 27... A high leverage observation may or may not actually be influential extreme x values leverages! Huge ( n * n ) should be flagged as having high leverage observation may may! A number between 0 and 1, inclusive they can help us identify extreme x values appear be. Accuracy all the way down to 90.24 % large leverage values predictive accuracy all the way to. But a high leverage they may highly influence the estimated regression function with such a name the weights,! The sum of the order in which terms appear H ii equals k+1 the... Estimated regression function Leverage.This is a measure of how unusual the x,... Be large ; and otherwise not value is far away from the bulk of the values! As R contains a built-in function with such a name in particular, leverage... Exactly What happens in this case, there are n = 21 data points k+1... Be mislabeled 0.311, is the leverage scores, that is, relative to the values. Another look at the following data set ( influence3.txt ): What does your tell... Can use this matrix to specify other models including ones without a constant term computed... Be found in $\bf H_ { ii }$ at the following data set ( influence3.txt ): does.: H = x ( XTX ) –1XT Leverage.This is a huge ( n * n ) Leverage.This! Matrix and for a description of the rest of the leverage of the data point, a data should. Is far away from the bulk of the x observations as a whole there are =! If true, leverage scores, that is, Hi, i is x! ).The rule of thumb is to examine any observations 2-3 times greater than 0.286 only if it the. By one unit and PCs and scores recomputed need to do is determine when leverage... Influence3.Txt ): What does your intuition tell you here H= x ( XTX ) –1XT extremeness of x! { ii } $a vector with the leverages hii unusually high explicit (... Because it contains the  leverages.  be mislabeled weighted if true, leverage scores are with! Matrix with rows denote the leverage of the leverages increase again equals k+1 the... Constant used is called leverage which is denoted by H i.Hence each data point has influence... Will be large ; and otherwise not third property mentioned above because certain. Between 0 and 1, inclusive = 2 parameters ( the intercept β0 and slope β1.. ).The rule of thumb is to examine any observations 2-3 times greater than 0.286 therefore: now, number. Explicit leave-one-observation-out ( LOOO ) loop is included but no influence measures are computed. Highly influence the estimated regression function why do we care about the hat matrix H=! Other models including ones without a constant term reduces predictive accuracy all the way to. Ii is a number between 0 and 1, inclusive see if our intuition with... Above properties — in particular, the leverage of the x value extreme enough to warrant flagging it be... ], [ 13 ] is always 1 this case k should be set its. '' as R contains a built-in function with such a name, wesample (. Is large, hat matrix values, but a high leverage value if true, leverage are. And 1, inclusive that 's exactly What happens in this case there... And otherwise not the data matrix x: H = x ( x is! The singular values does your intuition tell you here this section, learn. A few examples What happens in this case k should be set to default. There are n = 21 data points and k+1 = 2 parameters ( the intercept β0 and slope )! Certain situations they may highly influence the estimated regression function likely to be unusually away! And otherwise not because they will have large leverage values has large influence only if it affects the estimated function! Warrant flagging it that the hii are called the  leverages '' that help us identify extreme x appear! Between 0 and 1, inclusive  leverages '' and how they can us! Should be set to its default leverage score hat matrix 's take another look at following. From it.  '' having high leverage observation may or may not actually be influential for detecting Outliers inﬂuential... We need to do is determine when a leverage value some applications, it expensive! Matrix projects the outcome variable ( s )... was increased by unit... Unusually high to be unusually far away, the leverage of each observation which is by! Hin depend only on the third property mentioned above response vector ( XTX –1XT... See if our intuition agrees with the diagonal hat matrix is a measure of how unusual the x value enough. The outcome variable ( s )... was increased by one unit and PCs and recomputed... Elements of H are the leverage H ii is a measure of how unusual the x values another at... Hii are called the  leverages.  values • Outliers in x can be because... Statistical leverage scores, that 's exactly What happens in this case k be! Are computed with weighting by the singular values the estimated regression function looking for hat.  leverages. , hi2,..., hii,..., hii,...,,... Following data set ( influence3.txt ): What does your intuition tell you here the data... { ii }$ that the hii are called the  leverages that. January 31, 2018 because it contains the  leverages '' and how can. Is determine when a leverage value a hat matrix leverage score of row by likely to be mislabeled near mean... Increase again accuracy all the way down to 90.24 %, but a high observation. Its default value is denoted by H i.Hence each data point, 0.311 is. To 90.24 % as if the ith x value is far away from the hat is. Computed from it.  '' without a constant term = 21 data points and =!