Deeply Learning Physics

Why Mix Physics with Deep Learning?

Physics-Based Deep Learning (PBDL) represents a interdisciplinary approach that combines the rigor of physics with the flexibility of deep learning (Thuerey et al., 2021). Originally proposed by researchers at Brown and Penn (Raissi et al., 2019), this emerging field leverages the strengths of both domains to address complex scientific problems that traditional methods strive to solve.

PBDL = DL + Physics-Based Loss Functions

At its core, PBDL integrates physical laws and principles directly into the architecture of deep learning models. By embedding these fundamental rules, PBDL ensures that the models not only learn from data but also adhere to the underlying physical realities of the problem. This approach contrasts with purely data-driven models, which may lack interpretability and generalizability, especially in scenarios where data is sparse or noisy.

Industrial Applications

In computer vision, PBDL enhances the accuracy and reliability of image recognition and analysis by incorporating physical principles into the model. For instance, in construction, PBDL can be used to analyze images of buildings and infrastructure to detect structural anomalies, such as cracks or deformations. By integrating physics-based models, the system can better understand how these structures should behave under normal conditions, improving its ability to identify potential issues. This approach reduces false positives and negatives in defect detection, leading to more precise assessments and maintenance recommendations.

In preventive maintenance, PBDL plays a critical role by predicting equipment failures before they occur. By integrating the physics of machinery, such as the wear and tear dynamics, with deep learning algorithms, PBDL can predict when a machine is likely to fail or require maintenance. This is particularly valuable in industries like transportation, where equipment reliability is crucial. For example, in aviation, PBDL can analyze sensor data from aircraft engines to predict potential failures, allowing for timely maintenance that prevents costly and dangerous breakdowns.

Overall, PBDL is a promising avenue that enhances the predictive power and reliability of machine learning models, enabling breakthroughs in various scientific and engineering disciplines by grounding data-driven methods in the solid foundation of physical laws.

The Math of PBDL

It’s best to delve deeper into the formulation of a deep learning network using the Universal Approximation Theorem (UAT) and then incorporate physical laws into it. UAT states that a feedforward neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function, provided it uses a non-linear activation function.

To illustrate, let’s follow three steps:

Formulate a Deep Learning Network using UAT.
Insert Physical Laws into the Network.
Give a Simple Example.

Consider a neural network with an input vector \( \mathbf{x} \in \mathbb{R}^n \), a hidden layer with \( m \) neurons, and an output \( y \in \mathbb{R} \). The network can be described as follows:

\[ \mathbf{h} = \sigma(\mathbf{W}_1 \mathbf{x} + \mathbf{b}_1) \] \[ y = \mathbf{W}_2 \mathbf{h} + b_2 \]

Where:

\( \mathbf{W}_1 \in \mathbb{R}^{m \times n} \) is the weight matrix for the input layer.
\( \mathbf{b}_1 \in \mathbb{R}^m \) is the bias vector for the hidden layer.
\( \sigma \) is a non-linear activation function (e.g., ReLU, sigmoid).
\( \mathbf{W}_2 \in \mathbb{R}^m \) is the weight vector for the output layer.
\( b_2 \in \mathbb{R} \) is the bias term for the output layer.

To insert physical laws into the network, we incorporate known physical relationships directly into the learning process. This can be done by modifying the loss function to penalize violations of physical laws.

Take, for example, a simple physical law, such as Newton’s second law of motion:

\[ F = ma \]

Where \( F \) is the force, \( m \) is the mass, and \( a \) is the acceleration. Suppose we want our neural network to predict the acceleration \( a \) given the force \( F \) and mass \( m \).

We can modify the loss function to include a term that penalizes deviations from this physical law. Let \( \hat{a} \) be the predicted acceleration from the neural network. The loss function \( \mathcal{L} \) can be defined as:

\[\mathcal{L} = \mathcal{L}_{\text{data}} + \lambda \mathcal{L}_{\text{physics}}\]

Where:

\( \mathcal{L}_{\text{data}} \) is the data-driven loss (e.g., mean squared error between predicted and true acceleration).
\( \mathcal{L}_{\text{physics}} = | F - m \hat{a} |^2 \) penalizes deviations from Newton’s second law.
\( \lambda \) is a regularization parameter that controls the importance of the physical constraint.

Example: Predicting the Motion of a Particle

Consider a particle of mass \( m = 2 \) kg subject to a force \( F(t) \) over time \( t \). We want to predict the acceleration \( a(t) \) using a neural network.

Neural Network Model: Input: Force \( F(t) \), Output: Predicted acceleration \( \hat{a}(t) \).
Training Data: Simulated data for \( F(t) \) and true acceleration \( a(t) \).
Physics-Informed Loss Function:

\[\mathcal{L} = \frac{1}{N} \sum_{i=1}^N \left( a_i - \hat{a}_i \right)^2 + \lambda \sum_{i=1}^N \left( F_i - 2 \hat{a}_i \right)^2\]

Where \( N \) is the number of data points.

In this example, the neural network is trained to predict the acceleration while adhering to Newton’s second law of motion. The regularization parameter \( \lambda \) ensures that the physical law is respected during the learning process.

Let’s see the example in action with Jupyter:

Note that how the code computes the highly accurate estimate of acceleration to be 2.500002861022949 \(\textrm{ms}^{-2}\) .

Takeaways

PBDL enhances the accuracy and reliability of predictions by embedding physical laws into the learning process. This approach is particularly useful for systems where physical principles are well understood and can provide additional guidance to the model.

To Dig Deeper

What better way than to listen to the experts on YouTube?

This digital book contains a practical and comprehensive introduction of everything related to deep learning in the context of physical simulations. As much as possible, all topics come with hands-on code examples in the form of Jupyter notebooks to quickly get started. Beyond standard supervised learning from data, we’ll look at physical loss constraints, more tightly coupled learning algorithms with differentiable simulations, as well as reinforcement learning and uncertainty modeling. We live in exciting times: these methods have a huge potential to fundamentally change what computer simulations can achieve.

We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge–Kutta time stepping schemes with unlimited number of stages. The effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction–diffusion systems, and the propagation of nonlinear shallow-water waves.

Why Mix Physics with Deep Learning?

Industrial Applications

The Math of PBDL

Example: Predicting the Motion of a Particle

Takeaways

To Dig Deeper

References

2021

2019