Intrinsic Dimensionality of PINN Latent Spaces for Burger's Equation: Evidence for a Renormalization Group-like Flow

Abstract: Understanding the internal representations learned by neural networks, particularly Physics-Informed Neural Networks (PINNs) used for scientific modeling, is crucial for their interpretation and application. This study investigates the complexity of the 10-dimensional latent space learned by a PINN trained to solve the 2D Burger's equation, focusing on how its intrinsic dimensionality (ID) varies with the physical parameter of viscosity, $\nu$. Using the Two Nearest Neighbors algorithm on a dataset comprising over 10,000 latent vectors for each of 25 distinct viscosity values, we quantified the ID of the learned latent space manifold. Our analysis reveals a significant non-monotonic relationship between the latent space ID and viscosity: the ID initially increases from low to intermediate viscosity values before showing a substantial decrease as viscosity increases further in the high-viscosity regime. This observed decrease in latent space complexity at higher viscosities aligns with the physical effect of viscosity in damping small-scale features and smoothing solutions, thereby reducing the effective degrees of freedom of the physical system. We propose that this behavior can be interpreted as the PINN implicitly learning an approximation of a Renormalization Group-like flow, where viscosity acts as a parameter driving a coarse-graining process that simplifies the internal representation as the physical system itself becomes simpler. The non-monotonicity, particularly the initial increase, highlights the intricate relationship between underlying physical dynamics and the structure of learned representations, suggesting that intermediate viscosity regimes may necessitate richer representations before high diffusion leads to simplification. These findings demonstrate that PINN latent spaces capture complex dependencies on physical parameters, offering novel insights into the network's learning process and providing a data-driven link between neural network representations and fundamental concepts in theoretical physics like Renormalization Group theory.

Subjects:	physics.comp-ph; cs.LG
Cite as:	PX:2508.00071