# Quantile Regression with Gaussian Kernels

• Baobin Wang
• Ting Hu
• Hong Yin
Chapter

## Abstract

This paper aims at the error analysis of stochastic gradient descent (SGD) for quantile regression, which is associated with a sequence of varying $$\epsilon$$-insensitive pinball loss functions and flexible Gaussian kernels. Analyzing sparsity and learning rates will be provided when the target function lies in some Sobolev spaces and a noise condition is satisfied for the underlying probability measure. Our results show that selecting the variance of the Gaussian kernel plays a crucial role in the learning performance of quantile regression algorithms.

## Keywords

Quantile regresion Gaussian kernels Reproducing kernel Hilbert spaces Insensitive pinball loss Learning rate

## Notes

### Acknowledgments

The work described in this paper is partially supported by National Natural Science Foundation of China [Nos. 11671307 and 11571078], Natural Science Foundation of Hubei Province in China [No. 2017CFB523] and the Special Fund for Basic Scientific Research of Central Colleges, South-Central University for Nationalities [No. CZY18033].

## References

1. 1.
Aronszajn, N.: Theory of reproducing kernels. Tran. Am. Math. Soc. 68(3), 337–404 (1950)
2. 2.
Hu, T., Yuan, Y.: Learning rates of regression with q-norm loss and threshold. Anal. Appl. 14(06), 809–827 (2016)
3. 3.
Hwang, c., Shim, J.: A simple quantile regression via support vector machine. In: International Conference on Natural Computation, Springer, pp. 512–520 (2005)Google Scholar
4. 4.
Koenker, R., Geling, O.: Reappraising medfly longevity: a quantile regression survival analysis. J. Am. Stat. Assoc. 96(454), 458–468 (2001)
5. 5.
Koenker, R.: Quantile Regression. Cambridge University Press, New York (2005)
6. 6.
Rosset, S.: Bi-level path following for cross validated solution of kernel quantile regression. J. Mach. Learn. Res. 10(11), 2473–2505 (2009)
7. 7.
Shi, L., Huang, X., Tian, Z., Suykens, J.A.: Quantile regression with $$l1$$-regularization and Gaussian kernels. Adv. Comput. Math. 40(2), 517–551 (2014)
8. 8.
Smale, S., Zhou, D.X.: Estimating the approximation error in learning theory. Anal. Appl. 1(01), 17–41 (2003)
9. 9.
Smale, S., Zhou, D.X.: Online Learning with Markov Sampling. Anal. Appl. 7(01), 87–113 (2009)
10. 10.
Stein, E.M.: Singular Integrals and Differentiability Properties of Functions. In Bulletin of the London Mathematical Society (1973)Google Scholar
11. 11.
Steinwart, I., Christman, A.: Sparsity of SVMs that use the $$\epsilon$$-insensitive loss. In: Advances in Neural Information Processing Systems, pp. 1569–1576 2008)Google Scholar
12. 12.
Steinwart, I., Scovel, C., et al.: Fast rates for support vector machines using Gaussian kernels. Ann. Stat. 35(2), 575–607 (2007)
13. 13.
Steinwart, I., Christmann, A., et al.: Estimating conditional quantiles with the help of the pinball loss. Bernoulli. 17(1), 211–225 (2011)
14. 14.
Takeuchi, I., Le, Q.V., Sears, T.D., Smola, A.J.: Nonparametric quantile estimation. J. Mach. Learn. Res. 7, 1231–1264 (2006)
15. 15.
Ting, H., Xiang, D.H., Zhou, D.X.: Online learning for quantile regression and support vector regression. J. Stat. Plan. Inference 142(12), 3107–3122 (2012)
16. 16.
Vapnik, V.: The nature of statistical learning theory. Springer science & business media (2013)Google Scholar
17. 17.
Xiang, D.H., Hu, T., Zhou, D.X.: Approximation analysis of learning algorithms for support vector regression and quantile regression. J. Appl. Math. (2012).
18. 18.
Xiang, D.H., Zhou, D.X.: Classification with Gaussians and Convex Loss. J. Mach. Learn. Res. 10(10), 1447–1468 (2009)
19. 19.
Ying, Y., Zhou, D.X.: Online regularized classification algorithms. IEEE Trans. Inf. Theory. 52(11), 4775–4788 (2006)