Cross-sectional dependence (CD) tests for panel data and residuals

Computes Pesaran CD, CDw, CDw+, and CD* tests for cross-sectional dependence in panel variables or residuals. The implementation supports data frames, indexed panel data frames, numeric matrices, and fitted csdm_fit objects.

Usage

cd_test(object, ...)

# Default S3 method
cd_test(
  object,
  type = c("CD", "CDw", "CDw+", "CDstar", "all"),
  n_pc = 4L,
  seed = NULL,
  min_overlap = 2L,
  na.action = c("pairwise", "drop.incomplete.times"),
  reps = 1L,
  ...
)

# S3 method for class 'csdm_fit'
cd_test(
  object,
  type = c("CD", "CDw", "CDw+", "CDstar", "all"),
  n_pc = 4L,
  seed = NULL,
  min_overlap = 2L,
  na.action = c("pairwise", "drop.incomplete.times"),
  reps = 1L,
  ...
)

# S3 method for class 'cd_test'
print(x, digits = 3, ...)

# S3 method for class 'data.frame'
cd_test(
  object,
  ...,
  id = NULL,
  time = NULL,
  type = c("CD", "CDw", "CDw+", "CDstar", "all"),
  n_pc = 4L,
  seed = NULL,
  min_overlap = 2L,
  na.action = c("pairwise", "drop.incomplete.times"),
  reps = 1L
)

# S3 method for class 'pdata.frame'
cd_test(
  object,
  ...,
  id = NULL,
  time = NULL,
  type = c("CD", "CDw", "CDw+", "CDstar", "all"),
  n_pc = 4L,
  seed = NULL,
  min_overlap = 2L,
  na.action = c("pairwise", "drop.incomplete.times"),
  reps = 1L
)

# S3 method for class 'cd_test_list'
print(x, digits = 3, ...)

Arguments

object

A data.frame, pdata.frame, csdm_fit model, or numeric matrix with units in rows and time periods in columns (N x T).

...

For data frames, explicitly selected numeric columns as bare names, quoted names, or character vectors of names. Selections must be unnamed. For matrices and fitted models, additional arguments passed to methods.

type

Which test(s) to compute: one of "CD", "CDw", "CDw+", "CDstar", or "all" (default: "CD").

n_pc

Number of principal components for CD* (default 4).

seed

Integer seed for weight draws. Seeded calls restore the caller's RNG state; NULL uses the current RNG stream.

min_overlap

Minimum number of finite observations for retaining a unit before time filtering (all tests), and minimum overlapping time periods for including a pair in classical CD (default 2).

na.action

How to handle missing data: "drop.incomplete.times" removes time periods with any missing observations to create a balanced panel for CD*; "pairwise" (default) uses pairwise correlations for CD; unbalanced CDw/CDw+ requests error and CD* warns.

reps

Positive integer number of independent weight draws for CDw and CDw+ (default 1). The draws are summed and divided by sqrt(reps). CD and CD* are unaffected. Weighted test results record the number of draws in their reps field. For data inputs, seed is applied separately to each variable's test, making seeded results independent of selection order.

x

An object of class cd_test or cd_test_list.

digits

Number of digits to print (default 3).

id, time

Column names (strings) for the unit and time indexes of a plain data frame. For a pdata.frame, these are inferred from its stored indexes and supplied values are ignored. Time labels need not be numeric.

Value

An object of class cd_test with fields tests, type, N, T, na.action, excluded_units, excluded_times, kept_times, and call. The tests list contains one or more test results, each with statistic and p.value. Data-frame methods return a cd_test_list: a named list containing one cd_test result per selected variable, including when only one variable is selected. Each result also records variable, units, and excluded_unit_ids. The list has call, id, and time attributes and prints a combined table with each variable's sample dimensions. For fitted models, CD* additionally records input = "partial_residuals" and its own sample exclusions in tests$CDstar. With type = "all", top-level sample fields describe the full residual panel; the CD* sample dimensions are recorded in tests$CDstar$N and tests$CDstar$T.

Details

Selecting panel variables

For a plain data frame, supply distinct unit and time column names through id and time. For a pdata.frame, the stored indexes are used, even when the index columns have been removed from the data. Select at least one numeric non-index column explicitly. Each variable is tested separately using its own available sample and the same missing-data policy. No regression is fitted. CDw demeans the observations within each unit, as it does for matrix inputs. Test controls and panel indexes follow ... in the data-frame methods and must be named.

Notation

Let \(E\) contain the selected observations or residuals, with \(N\) cross-sectional units and \(T\) time periods. For each unit pair \((i,j)\), let \(T_{ij}\) be the number of overlapping time periods and \(\widehat\rho_{ij}\) the correlation computed after demeaning both series over that pair's overlapping observations.

Test statistics

CD (Pesaran, 2015)

$$CD = \sqrt{\frac{2}{N(N-1)}} \sum_{i<j} \sqrt{T_{ij}} \, \widehat\rho_{ij}$$ The sum includes pairs with at least min_overlap observations and a finite correlation; \(N\) remains the retained number of units.

CDw (Juodis and Reese, 2022)

Independent Rademacher weights \(w_i \in \{-1,1\}\) are applied by unit, independently of the data. Write \(u_{it}=e_{it}-\bar e_i\) for observations demeaned within unit and define the pooled variance $$\widehat s_W^2=\frac{1}{NT}\sum_{i=1}^N\sum_{t=1}^T w_i^2 u_{it}^2.$$ For a balanced panel, equation (30) of Juodis and Reese is implemented as $$CD_W = (\widehat s_W^2)^{-1} \sqrt{\frac{2}{TN(N-1)}} \sum_{t=1}^T\sum_{i=2}^N\sum_{j=1}^{i-1}w_i u_{it}w_j u_{jt}.$$ Since \(w_i^2=1\), the pooled variance is \(\operatorname{mean}(u_{it}^2)\). This is a weighted covariance statistic, with pooled rather than pair-specific scale normalization. The paper's estimated residuals have zero unit means when unit intercepts are included. With reps = G, draw \(G\) independent sets of unit weights and combine the statistics as in equation (33) of Juodis and Reese: $$\overline{CD}_W = \frac{1}{\sqrt{G}}\sum_{g=1}^G CD_W^{(g)}.$$ The default reps = 1 preserves the single-draw statistic. Use seed for reproducibility. The authors suggest a modest number of draws (for example, 30), since large \(G\) can amplify lower-order terms.

CDw+ (Juodis and Reese, 2022; Fan, Liao, and Yao, 2015)

Equation (32) of Juodis and Reese applies the power-enhancement principle of Fan, Liao, and Yao. Its statistic is $$CD_{W+} = CD_W + \sum_{i<j}|\widehat\rho_{ij}| 1\left\{|\widehat\rho_{ij}| > 2\sqrt{\log(N)/T}\right\}.$$ Here \(\widehat\rho_{ij}\) is the ordinary residual correlation, without multiplication by \(\sqrt{T}\). The nonnegative screening term is asymptotically zero under the conditions of the null hypothesis. With reps > 1, replace \(CD_W\) by \(\overline{CD}_W\) and add the screening term once.

CD* (Pesaran-Xie correction; standardized-PCA variant)

Each unit is demeaned and divided by its sample standard deviation before extracting n_pc principal components. Write \(A\) for this \(T\times N\) standardized matrix. The estimated loading matrix \(\widehat\Gamma\) is normalized so that \(\widehat\Gamma'\widehat\Gamma/N=I\), and the factors are \(\widehat F=A\widehat\Gamma/N\). For the factor-filtered residuals \(r_{it}\) of \(A-\widehat F\widehat\Gamma'\), define $$\widehat\sigma_i=\sqrt{T^{-1}\sum_t r_{it}^2},\quad \widehat\varphi=N^{-1}\sum_i\widehat\gamma_i/\widehat\sigma_i,\quad \widehat a_i=1-\widehat\sigma_i\widehat\varphi'\widehat\gamma_i.$$ The implemented bias correction is $$\widehat\theta=1-N^{-1}\sum_i\widehat a_i^2,\qquad CD^*=\frac{CD(r)+\sqrt{T/2}\,\widehat\theta}{1-\widehat\theta}.$$ This uses the Pesaran-Xie correction algebra, but their PCA procedure operates on unstandardized observations. Standardizing before PCA can change the estimated factor space and is an implementation variant. Setting n_pc = 0 returns classical CD without factor removal. For fitted models, the PCA input is \(\widehat v_{it}=y_{it}-x_{it}'\widehat\beta_i\), where \(x_{it}\) includes all economic and deterministic design columns, including any intercept, trend, and constructed lags. The fitted CSA contribution is retained, following the input construction in Pesaran-Xie's regression procedure. CD, CDw, and CDw+ use the full regression residuals instead.

CD* requires a nondegenerate bias-correction denominator. Near-zero denominators can produce severe size distortions, including proportional loading/error-scale designs after standardization. Numerical rank checks do not establish the validity of the asymptotic approximation.

All p-values use the two-sided standard-normal approximation \(2\Phi(-|\mathrm{statistic}|)\). The implemented tests do not estimate a serial-correlation correction. Stationarity, time-series dependence, factor strength, and relative panel dimensions must satisfy the relevant theory; accepting a numeric variable does not establish these conditions. For CD, omitting pairs or using very short overlaps can also affect calibration. Classical CD can be biased on CCE residuals because common time parameters have been estimated. CDw addresses this problem under Juodis-Reese's assumptions, including \(\sqrt{T}/N\to0\).

Missing data and balance

Time periods containing no finite residual for any retained unit are outside the effective residual sample and are always removed before balance is assessed. Partially observed periods are handled according to na.action.

CD

Uses pairwise-complete observations by default. Each pairwise correlation uses available overlaps.

CDw, CDw+

Require a balanced sample; explicitly select complete times if desired.

CD*

Requires a balanced panel. Explicitly setting na.action = "drop.incomplete.times" removes any time period with missing observations. With na.action = "pairwise", CD* returns NA and a warning when missing values are present.

References

Pesaran MH (2015). “Testing weak cross-sectional dependence in large panels.” Econometric Reviews, 34(6-10), 1089–1117.

Pesaran MH (2021). “General diagnostic tests for cross-sectional dependence in panels.” Empirical Economics, 60(1), 13–50.

Juodis A, Reese S (2022). “The incidental parameters problem in testing for remaining cross-section correlation.” Journal of Business and Economic Statistics, 40(3), 1191–1203. doi:10.1080/07350015.2021.1906687 .

Fan J, Liao Y, Yao J (2015). “Power Enhancement in High-Dimensional Cross-Section Tests.” Econometrica, 83(4), 1497–1541. doi:10.3982/ECTA12749 .

Pesaran MH, Xie Y (2021). “A bias-corrected CD test for error cross-sectional dependence in panel data models with latent factors.” Working Paper 9234, CESifo. Revised August 2022, https://www.ifo.de/DocDL/cesifo1_wp9234.pdf.

Pesaran MH, Xie Y (2026). “How to detect network dependence in latent factor models? A bias-corrected CD test.” Econometric Theory, 1–41. doi:10.1017/S0266466625100212 . First View.

Examples

# Simulate independent and dependent panels
set.seed(1)
E_indep <- matrix(rnorm(100), nrow = 10)
E_dep <- matrix(rnorm(10), nrow = 10, ncol = 10, byrow = TRUE)

# Compute all tests
cd_test(E_indep, type = "all")
#> Cross-sectional dependence tests
#> N = 10, T = 10
#> 
#>        statistic p.value
#> CD        -1.325   0.185
#> CDw       -1.184   0.237
#> CDw+      -1.184   0.237
#> CDstar     0.081   0.935
cd_test(E_dep, type = "CD")
#> Cross-sectional dependence tests
#> N = 10, T = 10
#> 
#>    statistic p.value
#> CD    21.213   0.000

# Specific test with parameters
cd_test(E_indep, type = "CDstar", n_pc = 2)
#> Cross-sectional dependence tests
#> N = 10, T = 10
#> 
#>        statistic p.value
#> CDstar    -0.303   0.762

# Test raw panel variables separately
panel <- expand.grid(id = 1:10, year = 1:10)
panel$x <- rnorm(nrow(panel))
panel$y <- rnorm(nrow(panel))
cd_test(panel, x, "y", id = "id", time = "year")
#> Cross-sectional dependence tests by variable
#> 
#>  variable test statistic p.value  N  T
#>         x   CD     0.440   0.660 10 10
#>         y   CD    -0.559   0.576 10 10

# From a fitted csdm model
data(PWT_60_07, package = "csdm")
df <- PWT_60_07
ids <- unique(df$id)[1:10]
df_small <- df[df$id %in% ids & df$year >= 1970, ]
fit <- csdm(
  log_rgdpo ~ log_hc + log_ck + log_ngd,
  data = df_small,
  id = "id",
  time = "year",
  model = "cce",
  csa = csdm_csa(vars = c("log_rgdpo", "log_hc", "log_ck", "log_ngd"))
)
cd_test(fit, type = "all")
#> Cross-sectional dependence tests
#> N = 10, T = 38
#> 
#>        statistic p.value
#> CD        -3.581   0.000
#> CDw       -1.430   0.153
#> CDw+       0.712   0.477
#> CDstar     4.224   0.000