跳转至

AutoDiff(自动微分) 模块

ppsci.autodiff.ad

This module is adapted from https://github.com/lululxvi/deepxde

jacobian = Jacobians() module-attribute

hessian = Hessians() module-attribute

Jacobians

Compute multiple Jacobians.

\[ \rm Jacobian(ys, xs, i, j) = \dfrac{\partial ys_i}{\partial xs_j} \]

A new instance will be created for a new pair of (output, input). For the (output, input) pair that has been computed before, it will reuse the previous instance, rather than creating a new one.

Source code in ppsci/autodiff/ad.py 
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
class Jacobians:
    r"""Compute multiple Jacobians.

    $$
    \rm Jacobian(ys, xs, i, j) = \dfrac{\partial ys_i}{\partial xs_j}
    $$

    A new instance will be created for a new pair of (output, input). For the (output,
    input) pair that has been computed before, it will reuse the previous instance,
    rather than creating a new one.
    """

    def __init__(self):
        self.Js = {}

    def __call__(
        self,
        ys: "paddle.Tensor",
        xs: "paddle.Tensor",
        i: int = 0,
        j: Optional[int] = None,
        retain_graph: Optional[bool] = None,
        create_graph: bool = True,
    ) -> "paddle.Tensor":
        """Compute jacobians for given ys and xs.

        Args:
            ys (paddle.Tensor): Output tensor.
            xs (paddle.Tensor): Input tensor.
            i (int, optional): i-th output variable. Defaults to 0.
            j (Optional[int]): j-th input variable. Defaults to None.
            retain_graph (Optional[bool]): Whether to retain the forward graph which
                is used to calculate the gradient. When it is True, the graph would
                be retained, in which way users can calculate backward twice for the
                same graph. When it is False, the graph would be freed. Default None,
                which means it is equal to `create_graph`.
            create_graph (bool, optional): Whether to create the gradient graphs of
                the computing process. When it is True, higher order derivatives are
                supported to compute; when it is False, the gradient graphs of the
                computing process would be discarded. Default False.

        Returns:
            paddle.Tensor: Jacobian matrix of ys[i] to xs[j].

        Examples:
            >>> import paddle
            >>> import ppsci
            >>> x = paddle.randn([4, 1])
            >>> x.stop_gradient = False
            >>> y = x * x
            >>> dy_dx = ppsci.autodiff.jacobian(y, x)
        """
        key = (ys, xs)
        if key not in self.Js:
            self.Js[key] = _Jacobian(ys, xs)
        return self.Js[key](i, j, retain_graph, create_graph)

    def _clear(self):
        """Clear cached Jacobians."""
        self.Js = {}
__call__(ys, xs, i=0, j=None, retain_graph=None, create_graph=True)

Compute jacobians for given ys and xs.

Parameters:

Name Type Description Default
ys Tensor

Output tensor.

 required
xs Tensor

Input tensor.

 required
i int

i-th output variable. Defaults to 0.

 0
j Optional[int]

j-th input variable. Defaults to None.

 None
retain_graph Optional[bool]

Whether to retain the forward graph which is used to calculate the gradient. When it is True, the graph would be retained, in which way users can calculate backward twice for the same graph. When it is False, the graph would be freed. Default None, which means it is equal to create_graph.

 None
create_graph bool

Whether to create the gradient graphs of the computing process. When it is True, higher order derivatives are supported to compute; when it is False, the gradient graphs of the computing process would be discarded. Default False.

 True

Returns:

Type Description
'paddle.Tensor'

paddle.Tensor: Jacobian matrix of ys[i] to xs[j].

Examples:

>>> import paddle
>>> import ppsci
>>> x = paddle.randn([4, 1])
>>> x.stop_gradient = False
>>> y = x * x
>>> dy_dx = ppsci.autodiff.jacobian(y, x)
Source code in ppsci/autodiff/ad.py 
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
def __call__(
    self,
    ys: "paddle.Tensor",
    xs: "paddle.Tensor",
    i: int = 0,
    j: Optional[int] = None,
    retain_graph: Optional[bool] = None,
    create_graph: bool = True,
) -> "paddle.Tensor":
    """Compute jacobians for given ys and xs.

    Args:
        ys (paddle.Tensor): Output tensor.
        xs (paddle.Tensor): Input tensor.
        i (int, optional): i-th output variable. Defaults to 0.
        j (Optional[int]): j-th input variable. Defaults to None.
        retain_graph (Optional[bool]): Whether to retain the forward graph which
            is used to calculate the gradient. When it is True, the graph would
            be retained, in which way users can calculate backward twice for the
            same graph. When it is False, the graph would be freed. Default None,
            which means it is equal to `create_graph`.
        create_graph (bool, optional): Whether to create the gradient graphs of
            the computing process. When it is True, higher order derivatives are
            supported to compute; when it is False, the gradient graphs of the
            computing process would be discarded. Default False.

    Returns:
        paddle.Tensor: Jacobian matrix of ys[i] to xs[j].

    Examples:
        >>> import paddle
        >>> import ppsci
        >>> x = paddle.randn([4, 1])
        >>> x.stop_gradient = False
        >>> y = x * x
        >>> dy_dx = ppsci.autodiff.jacobian(y, x)
    """
    key = (ys, xs)
    if key not in self.Js:
        self.Js[key] = _Jacobian(ys, xs)
    return self.Js[key](i, j, retain_graph, create_graph)

Hessians

Compute multiple Hessians.

\[ \rm Hessian(ys, xs, component, i, j) = \dfrac{\partial ys_{component}}{\partial xs_i \partial xs_j} \]

A new instance will be created for a new pair of (output, input). For the (output, input) pair that has been computed before, it will reuse the previous instance, rather than creating a new one.

Source code in ppsci/autodiff/ad.py 
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
class Hessians:
    r"""Compute multiple Hessians.

    $$
    \rm Hessian(ys, xs, component, i, j) = \dfrac{\partial ys_{component}}{\partial xs_i \partial xs_j}
    $$

    A new instance will be created for a new pair of (output, input). For the (output,
    input) pair that has been computed before, it will reuse the previous instance,
    rather than creating a new one.
    """

    def __init__(self):
        self.Hs = {}

    def __call__(
        self,
        ys: "paddle.Tensor",
        xs: "paddle.Tensor",
        component: Optional[int] = None,
        i: int = 0,
        j: int = 0,
        grad_y: Optional["paddle.Tensor"] = None,
        retain_graph: Optional[bool] = None,
        create_graph: bool = True,
    ) -> "paddle.Tensor":
        """Compute hessian matrix for given ys and xs.

        Args:
            ys (paddle.Tensor): Output tensor.
            xs (paddle.Tensor): Input tensor.
            component (Optional[int]): If `y` has the shape (batch_size, dim_y > 1), then `y[:, component]`
                is used to compute the Hessian. Do not use if `y` has the shape (batch_size,
                1). Defaults to None.
            i (int, optional): i-th input variable. Defaults to 0.
            j (int, optional): j-th input variable. Defaults to 0.
            grad_y (Optional[paddle.Tensor]): The gradient of `y` w.r.t. `xs`. Provide `grad_y` if known to avoid
                duplicate computation. Defaults to None.
            retain_graph (Optional[bool]): Whether to retain the forward graph which
                is used to calculate the gradient. When it is True, the graph would
                be retained, in which way users can calculate backward twice for the
                same graph. When it is False, the graph would be freed. Default None,
                which means it is equal to `create_graph`.
            create_graph (bool, optional): Whether to create the gradient graphs of
                the computing process. When it is True, higher order derivatives are
                supported to compute; when it is False, the gradient graphs of the
                computing process would be discarded. Default False.

        Returns:
            paddle.Tensor: Hessian matrix.

        Examples:
            >>> import paddle
            >>> import ppsci
            >>> x = paddle.randn([4, 3])
            >>> x.stop_gradient = False
            >>> y = (x * x).sin()
            >>> dy_dxx = ppsci.autodiff.hessian(y, x, component=0)
        """
        key = (ys, xs, component)
        if key not in self.Hs:
            self.Hs[key] = _Hessian(ys, xs, component=component, grad_y=grad_y)
        return self.Hs[key](i, j, retain_graph, create_graph)

    def _clear(self):
        """Clear cached Hessians."""
        self.Hs = {}
__call__(ys, xs, component=None, i=0, j=0, grad_y=None, retain_graph=None, create_graph=True)

Compute hessian matrix for given ys and xs.

Parameters:

Name Type Description Default
ys Tensor

Output tensor.

 required
xs Tensor

Input tensor.

 required
component Optional[int]

If y has the shape (batch_size, dim_y > 1), then y[:, component] is used to compute the Hessian. Do not use if y has the shape (batch_size, 1). Defaults to None.

 None
i int

i-th input variable. Defaults to 0.

 0
j int

j-th input variable. Defaults to 0.

 0
grad_y Optional[Tensor]

The gradient of y w.r.t. xs. Provide grad_y if known to avoid duplicate computation. Defaults to None.

 None
retain_graph Optional[bool]

Whether to retain the forward graph which is used to calculate the gradient. When it is True, the graph would be retained, in which way users can calculate backward twice for the same graph. When it is False, the graph would be freed. Default None, which means it is equal to create_graph.

 None
create_graph bool

Whether to create the gradient graphs of the computing process. When it is True, higher order derivatives are supported to compute; when it is False, the gradient graphs of the computing process would be discarded. Default False.

 True

Returns:

Type Description
'paddle.Tensor'

paddle.Tensor: Hessian matrix.

Examples:

>>> import paddle
>>> import ppsci
>>> x = paddle.randn([4, 3])
>>> x.stop_gradient = False
>>> y = (x * x).sin()
>>> dy_dxx = ppsci.autodiff.hessian(y, x, component=0)
Source code in ppsci/autodiff/ad.py 
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
def __call__(
    self,
    ys: "paddle.Tensor",
    xs: "paddle.Tensor",
    component: Optional[int] = None,
    i: int = 0,
    j: int = 0,
    grad_y: Optional["paddle.Tensor"] = None,
    retain_graph: Optional[bool] = None,
    create_graph: bool = True,
) -> "paddle.Tensor":
    """Compute hessian matrix for given ys and xs.

    Args:
        ys (paddle.Tensor): Output tensor.
        xs (paddle.Tensor): Input tensor.
        component (Optional[int]): If `y` has the shape (batch_size, dim_y > 1), then `y[:, component]`
            is used to compute the Hessian. Do not use if `y` has the shape (batch_size,
            1). Defaults to None.
        i (int, optional): i-th input variable. Defaults to 0.
        j (int, optional): j-th input variable. Defaults to 0.
        grad_y (Optional[paddle.Tensor]): The gradient of `y` w.r.t. `xs`. Provide `grad_y` if known to avoid
            duplicate computation. Defaults to None.
        retain_graph (Optional[bool]): Whether to retain the forward graph which
            is used to calculate the gradient. When it is True, the graph would
            be retained, in which way users can calculate backward twice for the
            same graph. When it is False, the graph would be freed. Default None,
            which means it is equal to `create_graph`.
        create_graph (bool, optional): Whether to create the gradient graphs of
            the computing process. When it is True, higher order derivatives are
            supported to compute; when it is False, the gradient graphs of the
            computing process would be discarded. Default False.

    Returns:
        paddle.Tensor: Hessian matrix.

    Examples:
        >>> import paddle
        >>> import ppsci
        >>> x = paddle.randn([4, 3])
        >>> x.stop_gradient = False
        >>> y = (x * x).sin()
        >>> dy_dxx = ppsci.autodiff.hessian(y, x, component=0)
    """
    key = (ys, xs, component)
    if key not in self.Hs:
        self.Hs[key] = _Hessian(ys, xs, component=component, grad_y=grad_y)
    return self.Hs[key](i, j, retain_graph, create_graph)

clear()

Clear cached Jacobians and Hessians.

Source code in ppsci/autodiff/ad.py 
269
270
271
272
def clear():
    """Clear cached Jacobians and Hessians."""
    jacobian._clear()
    hessian._clear()
最后更新: November 17, 2023
创建日期: November 6, 2023