Quantifying the Evolution of Learned Feature Structure in PINN Latent Space for 2D Burger's Equation via Principal Component Analysis

Abstract: Understanding how Physics-Informed Neural Networks (PINNs) encode complex physical phenomena in their latent spaces is crucial for interpreting their learned representations. This study investigates the statistical structure of the 10-dimensional latent space learned by a PINN for the 2D Burger's equation across 25 viscosity values, a parameter controlling the transition from turbulent-like to diffusive regimes. We applied Principal Component Analysis (PCA) to standardized latent vectors extracted for each viscosity, analyzing the evolution of the eigenvalue spectrum and eigenvector structure. Our analysis quantified how the distribution of variance across latent dimensions changes with viscosity, tracking eigenvalue magnitudes, spectrum concentration (normalized entropy), and effective dimensionality based on variance explained. We also assessed the stability of the dominant principal component directions using cosine similarity. Our results show that as viscosity increases, the variance captured by the leading principal component decreases, and variance becomes more evenly distributed across latent dimensions (increasing spectrum entropy). The PCA-based effective dimensionality exhibits a non-monotonic trend, peaking at intermediate viscosities, which qualitatively aligns with previous intrinsic dimensionality findings. While the primary direction of variation (PC1) shows relative stability across low-to-intermediate viscosities, it undergoes significant rotation at high viscosities, and secondary directions (PC2, PC3) are less stable, particularly when eigenvalues are close. These quantitative findings provide evidence that the PINN adapts its internal latent space structure to the underlying physics. The observed evolution, including changes in variance distribution, non-monotonic complexity, and PC stability, offers insights into how the network implicitly captures physical transitions and potentially reflects principles analogous to coarse-graining as the system simplifies in the diffusion-dominated regime. \

Subjects:	physics.comp-ph; cs.LG
Cite as:	PX:2508.00072