Radial basis function explained

In mathematics a radial basis function (RBF) is a real-valued function $\backslash varphi$ whose value depends only on the distance between the input and some fixed point, either the origin, so that $\backslash varphi(\backslash mathbf)\; =\; \backslash hat\backslash varphi(\backslash left\backslash |\backslash mathbf\backslash right\backslash |)$, or some other fixed point $\backslash mathbf$, called a center, so that $\backslash varphi(\backslash mathbf)\; =\; \backslash hat\backslash varphi(\backslash left\backslash |\backslash mathbf-\backslash mathbf\backslash right\backslash |)$. Any function $\backslash varphi$ that satisfies the property $\backslash varphi(\backslash mathbf)\; =\; \backslash hat\backslash varphi(\backslash left\backslash |\backslash mathbf\backslash right\backslash |)$ is a radial function. The distance is usually Euclidean distance, although other metrics are sometimes used. They are often used as a collection

which forms a basis for some function space of interest, hence the name.

Sums of radial basis functions are typically used to approximate given functions. This approximation process can also be interpreted as a simple kind of neural network; this was the context in which they were originally applied to machine learning, in work by David Broomhead and David Lowe in 1988,^{[1] [2]} which stemmed from Michael J. D. Powell's seminal research from 1977.^{[3] [4] [5]} RBFs are also used as a kernel in support vector classification.^[6] The technique has proven effective and flexible enough that radial basis functions are now applied in a variety of engineering applications.^{[7] [8]}

Definition

A radial function is a function

Examples

Commonly used types of radial basis functions include (writing $r\; =\; \backslash left\backslash |\backslash mathbf\; -\; \backslash mathbf\_i\backslash right\backslash |$ and using $\backslash varepsilon$ to indicate a shape parameter that can be used to scale the input of the radial kernel^[11]):

Approximation

See main article: Kernel smoothing.

See main article: Radial basis function interpolation.

Radial basis functions are typically used to build up function approximations of the formwhere the approximating function $y(\backslash mathbf)$ is represented as a sum of

radial basis functions, each associated with a different center $\backslash mathbf\_i$, and weighted by an appropriate coefficient $w\_i.$ The weights $w\_i$ can be estimated using the matrix methods of linear least squares, because the approximating function is linear in the weights $w\_i$.

Approximation schemes of this kind have been particularly used in time series prediction and control of nonlinear systems exhibiting sufficiently simple chaotic behaviour and 3D reconstruction in computer graphics (for example, hierarchical RBF and Pose Space Deformation).

RBF Network

See main article: radial basis function network.

The sumcan also be interpreted as a rather simple single-layer type of artificial neural network called a radial basis function network, with the radial basis functions taking on the role of the activation functions of the network. It can be shown that any continuous function on a compact interval can in principle be interpolated with arbitrary accuracy by a sum of this form, if a sufficiently large number $N$ of radial basis functions is used.

The approximant $y(\backslash mathbf)$ is differentiable with respect to the weights $w\_i$. The weights could thus be learned using any of the standard iterative methods for neural networks.

Using radial basis functions in this manner yields a reasonable interpolation approach provided that the fitting set has been chosen such that it covers the entire range systematically (equidistant data points are ideal). However, without a polynomial term that is orthogonal to the radial basis functions, estimates outside the fitting set tend to perform poorly.

RBFs for PDEs

See main article: Kansa method. Radial basis functions are used to approximate functions and so can be used to discretize and numerically solve Partial Differential Equations (PDEs). This was first done in 1990 by E. J. Kansa who developed the first RBF based numerical method. It is called the Kansa method and was used to solve the elliptic Poisson equation and the linear advection-diffusion equation. The function values at points

in the domain are approximated by the linear combination of RBFs:

The derivatives are approximated as such:where

are the number of points in the discretized domain, the dimension of the domain and the scalar coefficients that are unchanged by the differential operator.^[12]

Different numerical methods based on Radial Basis Functions were developed thereafter. Some methods are the RBF-FD method,^{[13] [14]} the RBF-QR method^[15] and the RBF-PUM method.^[16]

See also

• Matérn covariance function
• Radial basis function interpolation
• Kansa method

Further reading

• Hardy. R.L.. 1971. Multiquadric equations of topography and other irregular surfaces. Journal of Geophysical Research. 76. 8. 1905–1915. 10.1029/jb076i008p01905. 1971JGR....76.1905H.
• Hardy . R.L. . 1990 . Theory and applications of the multiquadric-biharmonic method, 20 years of Discovery, 1968 1988 . Comp. Math Applic . 19 . 8/9. 163–208 . 10.1016/0898-1221(90)90272-l. free .
  • Sirayanone, S., 1988, Comparative studies of kriging, multiquadric-biharmonic, and other methods for solving mineral resource problems, PhD. Dissertation, Dept. of Earth Sciences, Iowa State University, Ames, Iowa.
• Sirayanone . S. . Hardy . R.L. . 1995 . The Multiquadric-biharmonic Method as Used for Mineral Resources, Meteorological, and Other Applications . Journal of Applied Sciences and Computations . 1 . 437–475 .

Notes and References

1. http://www.anc.ed.ac.uk/rbf/intro/node8.html Radial Basis Function networks
2. David H. . Broomhead . David . Lowe . Multivariable Functional Interpolation and Adaptive Networks . Complex Systems . 2 . 321–355 . 1988 . https://web.archive.org/web/20140714173428/https://www.complex-systems.com/pdf/02-3-5.pdf . 2014-07-14.
3. Restart procedures for the conjugate gradient method . Michael J. D. Powell . . 12 . 1 . 241–254 . 1977 . 10.1007/bf01593790. 9500591 . Michael J. D. Powell .
4. M.Sc. . Ferat . Sahin . A Radial Basis Function Approach to a Color Image Classification Problem in a Real Time Industrial Application . . 1997 . Radial basis functions were first introduced by Powell to solve the real multivariate interpolation problem. . 26 . 10919/36847 .

5. "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."

6. Web site: Introduction to Support Vector Machines . VanderPlas . Jake . [O'Reilly] . 6 May 2015 . 14 May 2015.
7. Book: Buhmann, Martin Dietrich. Radial basis functions : theory and implementations. 2003. Cambridge University Press. 978-0511040207. 56352083.
8. Book: Biancolini, Marco Evangelos. Fast radial basis functions for engineering applications. 2018. 9783319750118. Springer International Publishing. 1030746230.
9. Book: Fasshauer . Gregory E. . Meshfree Approximation Methods with MATLAB . 2007 . World Scientific Publishing Co. Pte. Ltd. . Singapore . 9789812706331 . 17–25.
10. Book: Wendland . Holger . Scattered Data Approximation . 2005 . Cambridge University Press . Cambridge . 0521843359 . 11, 18-23,64-66.
11. Book: Fasshauer . Gregory E. . Meshfree Approximation Methods with MATLAB . 2007 . World Scientific Publishing Co. Pte. Ltd. . Singapore . 9789812706331 . 37.
12. Kansa . E. J. . 1990-01-01. Multiquadrics—A scattered data approximation scheme with applications to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations. Computers & Mathematics with Applications. en. 19 . 8 . 147–161. 10.1016/0898-1221(90)90271-K. 0898-1221. free.
13. Tolstykh. A. I.. Shirobokov. D. A.. 2003-12-01. On using radial basis functions in a "finite difference mode" with applications to elasticity problems. Computational Mechanics . en . 33. 1. 68–79 . 10.1007/s00466-003-0501-9 . 2003CompM..33...68T . 121511032 . 1432-0924.
14. Shu. C. Ding. H. Yeo. K. S. 2003-02-14 . Local radial basis function-based differential quadrature method and its application to solve two-dimensional incompressible Navier–Stokes equations. Computer Methods in Applied Mechanics and Engineering. en. 192. 7 . 941–954 . 10.1016/S0045-7825(02)00618-7 . 2003CMAME.192..941S. 0045-7825.
15. Fornberg. Bengt. Larsson. Elisabeth . Flyer. Natasha. 2011-01-01. Stable Computations with Gaussian Radial Basis Functions . SIAM Journal on Scientific Computing . 33 . 2 . 869–892 . 10.1137/09076756X. 2011SJSC...33..869F . 1064-8275.
16. Safdari-Vaighani . Ali . Heryudono. Alfa . Larsson. Elisabeth . 2015-08-01. A Radial Basis Function Partition of Unity Collocation Method for Convection–Diffusion Equations Arising in Financial Applications . Journal of Scientific Computing . en . 64 . 2. 341–367 . 10.1007/s10915-014-9935-9 . 254691757 . 1573-7691.