Scalable Sketch-Based Sport Video Retrieval in the Cloud

  • Ihab Al KabaryEmail author
  • Heiko Schuldt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12403)


Content-based video retrieval in general and in sport videos in particular has attracted an increasing interest in the past few years, due to the growing interest in sports analytics. Especially sketch-based queries, enabling spatial search in video collections, are increasingly being demanded by coaches and analysts in team sports as an essential tool for game analysis. Although there has been great progress in the last years in the field of sketch-based retrieval in sports, most approaches focus on functional aspects and only consider just a very limited number of games. The problem is to scale these systems to allow for interactive video retrieval on a large game collection, beyond single games. In this paper, we show how SportSense, our sketch-based video retrieval system, can be deployed and scaled-out in the Cloud, allowing managers and analysts to interactively search for scenes of their choice within a large collection of games. In our evaluations, we show how the system can scale to a collection of the size of an entire season with response times that enable real-time analysis.


  1. 1.
    Al Kabary, I., Schuldt, H.: Towards sketch-based motion queries in sports videos. In: 2013 IEEE International Symposium on Multimedia (ISM), December 2013Google Scholar
  2. 2.
    Kabary, I.A., Schuldt, H.: Using hand gestures for specifying motion queries in sketch-based video retrieval. In: de Rijke, M., et al. (eds.) ECIR 2014. LNCS, vol. 8416, pp. 733–736. Springer, Cham (2014). Scholar
  3. 3.
    Probst, L., et al.: SportSense: user interface for sketch-based spatio-temporal team sports video scene retrieval. In: Proceedings of the IUI 2018 Workshop on User Interfaces for Spatial and Temporal Data Analysis, Tokyo, Japan, March 2018Google Scholar
  4. 4.
    Seidenschwarz, P., Jonsson, A., Rauschenbach, F., Rumo, M., Probst, L., Schuldt, H.: Combining qualitative and quantitative analysis in football with sportsense. In: Proceedings of the ACM Workshop on Multimedia Content Analysis in Sports, France, October 2019Google Scholar
  5. 5.
    Al Kabary, I., Schuldt, H.: SportSense: using motion queries to find scenes in sports videos. In: Proceedings of the CIKM 2013, San Francisco, CA, USA. ACM, October 2013Google Scholar
  6. 6.
    Al Kabary, I., Schuldt, H.: Enhancing sketch-based sport video retrieval by suggesting relevant motion paths. In: Proceedings of the 37th International ACM SIGIR Conference, Gold Coast, QLD, Australia. ACM (2014)Google Scholar
  7. 7.
    Ballan, L., Bertini, M., Bimbo, A.D., Nunziati, W.: Soccer players identification based on visual local features. In: Proceedings of the 6th ACM CIVR Conference, July 2007Google Scholar
  8. 8.
    Fleischman, M., Roy, D.: Unsupervised content-based indexing of sports video. In: Proceedings of the 9th ACM International Workshop on Multimedia Information Retrieval, Augsburg, Germany, pp. 87–94. ACM, September 2007Google Scholar
  9. 9.
    Su, C., Liao, H., Tyan, H., Lin, C., Chen, D., Fan, K.: Motion flow-based video retrieval. IEEE Trans. Multimedia 9, 1193–1201 (2007)Google Scholar
  10. 10.
    Chang, S.F., Chen, W., Meng, H., Sundaram, H., Zhong, D.: A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Trans. Circ. Syst. Video Technol. 8, 602–615 (1998)Google Scholar
  11. 11.
    Shitrit, H.B., Berclaz, J., Fleuret, F., Fua, P.: Tracking multiple people under global appearance constraints. In: International Conference on Computer Vision (ICCV), Barcelona, Spain. IEEE, November 2011Google Scholar
  12. 12.
    Wilhelm, P., et al.: An integrated monitoring and analysis system for performance data of indoor sport activities. In: 10th Australasian Conference on Mathematics and Computers in Sport, Darwin, Australia, July 2010Google Scholar
  13. 13.
    Interplay Sports. Accessed Mar 2020
  14. 14.
    OptaSportsPro. Accessed Mar 2020
  15. 15.
    Panasonic Ultra Wide Angle Camera. Accessed Mar 2020
  16. 16.
    Stats Perform. Accessed Mar 2020
  17. 17.
  18. 18.
    Adidas Runtastic. Accessed Mar 2020
  19. 19.
    ZXY. Accessed Mar 2020
  20. 20.
    Probst, L., Brix, F., Schuldt, H., Rumo, M.: Real-time football analysis with streamteam. In: Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems, Barcelona, Spain. ACM, June 2017Google Scholar
  21. 21.
    Sha, L., Lucey, P., Yue, Y., Carr, P., Rohlf, C., Matthews, I.A.: Chalkboarding: a new spatiotemporal query paradigm for sports play retrieval. In: 21st International Conference on Intelligent User Interfaces, Sonoma, CA, USA, March 2016Google Scholar
  22. 22.
    Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the ACM SIGMOD Conference on Management of Data, Atlantic City, NJ, USA, May 1990Google Scholar
  23. 23.
    Fang, Y., Friedman, M., Nair, G., Rys, M., Schmid, A.E.: Spatial indexing in microsoft SQL server 2008. In: Proceedings of the ACM SIGMOD Conference on Management of Data, Vancouver, BC, Canada, pp. 1207–1216. ACM, June 2008Google Scholar
  24. 24.
    Comer, D.: Ubiquitous B-tree. ACM Comput. Surv. 11, 121–137 (1979)zbMATHGoogle Scholar
  25. 25.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)Google Scholar
  26. 26.
    Manchester City Football Club. Accessed Mar 2020
  27. 27.
    Amazon EC2. Accessed Mar 2020

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceUniversity of BaselBaselSwitzerland

Personalised recommendations