Parallel, Distributed, and Grid Computing

  • Wolfgang Schreiner
  • Károly Bósa
  • Andreas Langegger
  • Thomas Leitner
  • Bernhard Moser
  • Szilárd Páll
  • Volkmar Wieser
  • Wolfram Wöß


The core goal of parallel computing is to speedup computations by executing independent computational tasks concurrently (“in parallel”) on multiple units in a processor, on multiple processors in a computer, or on multiple networked computers which may be even spread across large geographical scales (distributed and grid computing); it is the dominant principle behind “supercomputing” respectively “high performance computing”. For several decades, the density of transistors on a computer chip has doubled every 18–24 months (“Moore’s Law”); until recently, this rate could be directly transformed into a corresponding increase of a processor’s clock frequency and thus into an automatic performance gain for sequential programs. However, since also a processor’s power consumption increases with its clock frequency, this strategy of “frequency scaling” became ultimately unsustainable: since 2004 clock frequencies have remained essentially stable and additional transistors have been primarily used to build multiple processors on a single chip (multi-core processors). Today therefore every kind of software (not only “scientific” one) must be written in a parallel style to profit from newer computer hardware.


Grid Computing Resource Description Framework Parallel Program Message Passing Interface Data Grid 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AB06]
    Jeff Andrews and Nick Baker. Xbox 360 System Architecture. IEEE Micro, 26(2):25–37, March–April 2006.CrossRefGoogle Scholar
  2. [ABC+06]
    Krste Asanovic, Ras Bodik, Bryan Christopher Catanzaro, et al. The Landscape of Parallel Computing Research: A View from Berkeley. Technical Report UCB/EECS-2006-183, EECS Department, University of California, Berkeley, December 2006.Google Scholar
  3. [ACP]
    ACPC – Austrian Center for Parallel Computation. information/acpc/acpc.php.Google Scholar
  4. [AMD]
    AMD. ATI Stream Technology. Scholar
  5. [Aus]
    Austrian Grid Project. Scholar
  6. [BCS05]
    Rebhi Baraka, Olga Caprotti, and Wolfgang Schreiner. A Web Registry for Publishing and Discovering Mathematical Services. In IEEE International Conference on e-Technology, e-Commerce, and e-Service (EEE-05), pages 190–193, Hong Kong, April 29–March 1, 2005. IEEE Computer Society Press.Google Scholar
  7. [BJ91]
    B. Buchberger and T. Jebelean. Systolic Algorithms in Computer Algebra. In NATO ASI on Parallel Processing on Distributed Memory Multiprocessors, Ankara, Turkey, 1991.Google Scholar
  8. [BLW06]
    Martin Blöchl, Andreas Langegger, and Wolfram Wöß. Registration of Heterogeneous Data Sources in the Case of the Grid Semantic Data Access Middleware (G-SDAM). In Proceedings of the Austrian Grid Symposium (AGS’06). OCG, 2006.Google Scholar
  9. [Bro]
    BrookGPU. Scholar
  10. [BS05]
    Karoly Bosa and Wolfgang Schreiner. Tolerating Stop Failures in Distributed Maple. Scalable Computing: Practice and Experience, 6(2):59–70, July 2005.Google Scholar
  11. [BS06]
    Rebhi Baraka and Wolfgang Schreiner. Semantic Querying of Mathematical Web Service Descriptions. In Mario Bravetti et al, editor, Third International Workshop on Web Services and Formal Methods (WS-FM 2006), volume 4184 of Lecture Notes in Computer Science, pages 73–87, Vienna, Austria, September 8–9, 2006. Springer.Google Scholar
  12. [BS08a]
    Karoly Bosa and Wolfgang Schreiner. Initial Design of a Distributed Supercomputing API for the Grid. Technical report, Research Institute for Symbolic Computation (RISC), Johannes Kepler University Linz, Austria., 2008.Google Scholar
  13. [BS08b]
    Karoly Bosa and Wolfgang Schreiner. The Porting of a Medical Grid Application from Globus 4 to the gLite Middleware. In Peter Kacsuk at al., editor, Proceedings of DAPSYS 2008, pages 51–61. Springer, September 2008.Google Scholar
  14. [BSBK05]
    Karoly Bosa, Wolfgang Schreiner, Michael Buchberger, and Thomas Kaltofen. SEE-GRID, A Grid-Based Medical Decision Support System for Eye Muscle Surgery. In Jens Volkert et al., editor, Proceedings of 1st Austrian Grid Symposium 2005, pages 61–74, Hagenberg, Austria, December 01-02 2005. Austrian Computer Society (OCG).Google Scholar
  15. [BSBK07]
    Karoly Bosa, Wolfgang Schreiner, Michael Buchberger, and Thomas Kaltofen. A Grid Software for Virtual Eye Surgery Based on Globus 4 and gLite. In Proceedings of ISPDC 2007, pages 151–158. IEEE Computer Society, 2007.Google Scholar
  16. [Buc78]
    Bruno Buchberger. Computer-Trees and Their Programming. In 4th Colloquium “Les arbres en algebre et en programmation”, pages 1–18, University of Lille, France, February 16–18, 1978.Google Scholar
  17. [Buc85]
    Bruno Buchberger. The L-Machine: An Attempt at Parallel Hardware for Symbolic Computation. In Symposium on Applied algebra, Algebraic algorithms, and Error Correcting Codes (AAECC), volume 229 of Lecture Notes in Computer Science, pages 333–347. Springer, 1985.Google Scholar
  18. [Buc04]
    Michael Buchberger. Biomechanical Modelling of the Human Eye. PhD thesis, Johannes Kepler University, Linz, Austria, March 2004.Google Scholar
  19. [But97]
    David R. Butenhof. Programming with POSIX Threads. Addison-Wesley, 1997.Google Scholar
  20. [Buy99]
    Rajkumar Buyya. High Performance Cluster Computing. Prentice Hall, 1999.Google Scholar
  21. [CAS]
    CASA — Computer Algebra System for Algebraic Geometry. http://www.risc. Scholar
  22. [CBM+08]
    Shuai Che, Michael Boyer, Jiayuan Meng, et al. A Performance Study of General-Purpose Applications on Graphics Processors Using CUDA. Journal of Parallel and Distributed Computing, 68:1370–1380, 2008.CrossRefGoogle Scholar
  23. [CFK+01]
    A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications, 23(3):187–200, 2001.CrossRefGoogle Scholar
  24. [CGP99]
    Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. Model Checking. MIT Press, 1999.Google Scholar
  25. [CS02]
    Olga Caprotti and Wolfgang Schreiner. Towards a Mathematical Service Description Language. In International Congress on Mathematical Software (ICMS), Bejing, China, August 20–28, 2002. World Scientific Publishers.Google Scholar
  26. [CUD]
    CUDA Zone. home.html.Google Scholar
  27. [Dis]
    Distributed Maple. Scholar
  28. [Ede]
    Eden: Parallel Functional Programming.∼eden.Google Scholar
  29. [EGE]
    Enabling Grids for E-sciencE (EGEE). Scholar
  30. [Erl]
    Erlang. Scholar
  31. [Eur]
    Euro-Par Conference Series — European Conference on Parallel and Distributed Computing. Scholar
  32. [FK98]
    Ian Foster and Carl Kesselmann. The Grid 2: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, 2nd edition, 1998.Google Scholar
  33. [FKT01]
    I. Foster, C. Kesselman, and S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Lecture Notes in Computer Science, 2150, 2001.Google Scholar
  34. [Fos95]
    Ian Foster. Designing and Building Parallel Programs: Concepts and Tools for Parallel Software Engineering. Addison-Wesley, 1995.Google Scholar
  35. [FR96]
    M. J. Flynn and K. W. Rudd. Parallel Architectures. ACM Computing Surveys, 28(1):67–70, 1996.CrossRefGoogle Scholar
  36. [gLi]
    gLite — Lightweight Middleware for Grid Computing. Scholar
  37. [Glo]
    The Globus Toolkit. Scholar
  38. [GPG]
    General-Purpose Computation Using Graphics Hardware. http://www.gpgpu. org.Google Scholar
  39. [GPH]
    Glasgow Parallel Haskell (GPH).˜dsg/gph.Google Scholar
  40. [HNS95]
    Hoon Hong, Andreas Neubacher, and Wolfgang Schreiner. The Design of the SACLIB/PACLIB Kernels. Journal of Symbolic Computation, 19:111–132, 1995.zbMATHCrossRefMathSciNetGoogle Scholar
  41. [HP06]
    John L. Hennessy and David A. Patterson. Computer Architecture: A Quantitative Approach. Academic Press, 4th edition, 2006.Google Scholar
  42. [HPF]
    High Performance Fortran (HPF). Scholar
  43. [Int]
    Int.EU.Grid Project. Scholar
  44. [KRD+03]
    Ujval J. Kapasi, Scott Rixner, William J. Dally, et al. Programmable Stream Processors. IEEE Computer, 36(8):54–62, August 2003.Google Scholar
  45. [KTF03]
    N. Karonis, B. Toonen, and I. Foster. MPICH-G2: A Grid-Enabled Implementation of the Message Passing Interface. Journal of Parallel and Distributed Computing (JPDC), 63(5):551–563, May 2003.zbMATHCrossRefGoogle Scholar
  46. [Lam02]
    Leslie Lamport. Specifying Systems: The TLA+ Language and Tools for Hardware and Software Engineers. Addison Wesley, 2002.Google Scholar
  47. [LNOM08]
    Erik Lindholm, John Nickolls, Stuart Oberman, and John Montrym. NVIDIA Tesla: A Unified Graphics and Computing Architecture. IEEE Micro, 28(2):39–55, March–April 2008.CrossRefGoogle Scholar
  48. [Loi92]
    Hans-Wolfgang Loidl. A Parallelizing Compiler for the Functional Programming Language EVE. In Austrian-Hungarian Workshop on Transputer Applications, pages 1–10, Sopron, Hungary, October 8–10, 1992. Hungarian Academy of SciencesTechnical Report KFKI-1992-34/M,N.Google Scholar
  49. [LWB08]
    A. Langegger, W. Wöß, and M. Blöchl. A Semantic Web Middleware for Virtual Data Integration on the Web. In Proceedings of the European Semantic Web Conference 2008, Tenerife, pages 493–507. Springer, 2008.Google Scholar
  50. [Mit05]
    Daniel Mitterdorfer. Grid-Capable Persistance Based on a Metamodel for Medical Decision Support. Master’s thesis, Upper Austria University of Applied Sciences, Hagenberg, Austria, July 2005.Google Scholar
  51. [MPIa]
    MPI (Message Passing Interface) Forum. Scholar
  52. [MPIb]
    MPICH Project. Scholar
  53. [MPIc]
    MPICH-G2 Project. Scholar
  54. [MSW00]
    Christian Mittermaier, Wolfgang Schreiner, and Franz Winkler. A Parallel Symbolic-Numerical Approach to Algebraic Curve Plotting. In Vladimir Gerdt and Ernst W. Mayr, editors, CASC-2000, Third International Workshop on Computer Algebra in Scientific Computing, pages 301–314, Samarkand, Uzbekistan, October 5–9, 2000. Springer, Berlin.Google Scholar
  55. [MW96]
    Michael Mnuk and Franz Winkler. CASA - A System for Computer Aided Constructive Algebraic Geometry. In J. Calmet and C. Limongelli, editors, International Symposium on the Design and Implementation of Symbolic Computation Systems (DISCO’96), volume 1128 of Lecture Notes in Computer Science, pages 297–307, Karsruhe, Germany, 1996. Springer.Google Scholar
  56. [NBGS08]
    John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable Parallel Programming with CUDA. Queue, 6(2):40–53, March–April 2008.CrossRefGoogle Scholar
  57. [NVI08a]
    NVIDIA. CUDA CUBLAS Library Documentation, March 2008. http:// 0/docs/CUBLAS Library 2.0. pdf.Google Scholar
  58. [NVI08b]
    NVIDIA. CUDA CUFFT Library Documentation, March 2008. http:// 0/docs/CUFFT Library 2.0.pdf.Google Scholar
  59. [NVI08c]
    NVIDIA. NVIDIA Tesla Personal Supercomputer, 2008. http://www.nvidia. com/object/personal supercomputing.html.Google Scholar
  60. [OGS]
    Open Grid Services Architecture WG (OGSA-WG). http://forge.gridforum. org/sf/projects/ogsa-wg.Google Scholar
  61. [OHL+08]
    John D. Owens, Mike Houston, David Luebke, Simon Green, John E. Stone, and James C. Phillips. GPU Computing. Proceedings of the IEEE, 96(5):879–899, May 2008.CrossRefGoogle Scholar
  62. [OLG+07]
    John D. Owens, David Luebke, Naga Govindaraju, Mark Harris, Jens Krüger, Aaron E. Lefohn, and Timothy J. Purcell. A Survey of General–Purpose Computation on Graphics Hardware. Computer Graphics Forum, 26(1):80–113, March 2007.CrossRefGoogle Scholar
  63. [Ope]
    The OpenMP API Specification for Parallel Programming. Scholar
  64. [Owe05]
    John Owens. Streaming Architectures and Technology Trends. In Matt Pharr, editor, GPU Gems 2, chapter 29, pages 457–470. Addison Wesley, 2005.Google Scholar
  65. [P´08]
    Szilárd Páll. GPU Computing Approach for Parallelizing Support Vector Machine Classification. Master’s thesis, Johannes Kepler University Linz, July 2008.Google Scholar
  66. [PAS]
    Parallel Symbolic Computation (PASCO) ’07. conferences/pasco2007/site/index.html.Google Scholar
  67. [Pat]
    Parallel Programming Patterns. Scholar
  68. [Pat04]
    David A. Patterson. Latency Lags Bandwith. Communications of the ACM, 47(10):71–75, October 2004.CrossRefGoogle Scholar
  69. [PVS]
    PVS Specification and Verification System. Scholar
  70. [RJ07]
    L. Ruff and T. Jebelean. Functional Based Synthesis of a Systolic Array for GCD Computation. In V. Zsok Z. Horvath, editor, Implementation and Application of Functional Languages, volume 4449 of LNCS, pages 37–54. Springer, 2007.Google Scholar
  71. [Sch91]
    Wolfgang Schreiner. ADAM – An Abstract Dataflow Machine and its Transputer Implementation. In Arndt Bode, editor, Second European Conference on Distributed Memory Computing (EDMCC2), volume 487 of Lecture Notes in Computer Science, pages 392–401, Munich, Germany, April 22–24, 1991. Springer.Google Scholar
  72. [Sch94]
    Wolfgang Schreiner. Virtual Tasks for the PACLIB Kernel. In Parallel Processing: CONPAR 94 - VAPP VI Third Joint International Conference on Vector and Parallel Processing, volume 854 of Lecture Notes in Computer Science, pages 533–544, Linz, Austria, September 6–8, 1994. Springer.Google Scholar
  73. [Sch95]
    Wolfgang Schreiner. Application of a Para-Functional Language to Problems in Computer Algebra. In A. P. Wim Böhm and John T. Feo, editors, High Performance Functional Computing, pages 10–24, Denver, Colorado, April 9–11, 1995. Lawrence Livermore National Laboratory Report CONF-9504126.Google Scholar
  74. [Sch96]
    Wolfgang Schreiner. A Para-Functional Programming Interface for a Parallel Computer Algebra Package. Journal of Symbolic Computation, 21:593–614, 1996.zbMATHCrossRefMathSciNetGoogle Scholar
  75. [SCI09]
    The SCIEnce Project (Symbolic Computation Infrastructure for Europe), 2009. Scholar
  76. [SEE]
    SEE-KID. Scholar
  77. [SFR03]
    W. Richard Stevens, Bill Fenner, and Andrew M. Rudoff. Unix Network Programming: The Sockets Networking API, 2003.Google Scholar
  78. [Sh]
    Sh High Level Metaprogramming Language. Scholar
  79. [Sie93]
    Kurt Siegl. Parallelizing Algorithms for Symbolic Computation Using MAPLE . In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 179–186, San Diego, CA, May 19–22, 1993. ACM Press.Google Scholar
  80. [Sie94]
    Kurt Siegl. A Parallel Factorization Tree Gröbner Basis Algorithm. In Hoon Hong, editor, International Symposium on Parallel Symbolic Computation (PASCO), volume 5 of Lecture Notes Series in Computing, pages 356–362, Hagenberg, Austria, September 26–28, 1994. World Scientific.Google Scholar
  81. [SMB03]
    Wolfgang Schreiner, Christian Mittermaier, and Karoly Bosa. Distributed Maple: Parallel Computer Algebra in Networked Environments. Journal of Symbolic Computation, 35:305–347, 2003.zbMATHCrossRefMathSciNetGoogle Scholar
  82. [SMW00a]
    Wolfgang Schreiner, Christian Mittermaier, and Franz Winkler. Analyzing Algebraic Curves by Cluster Computing. In Peter Kacsuk and Gabriele Kotsis, editors, Distributed and Parallel Systems – From Instruction Parallelism to Cluster Computing, DAPSYS’2000, 3rd Austrian-Hungarian Workshop on Distributed and Parallel Systems, pages 175–184, Balatonfüred, Hungary, September 10–13, 2000. Kluwer, Boston.Google Scholar
  83. [SMW00b]
    Wolfgang Schreiner, Christian Mittermaier, and Franz Winkler. On Solving a Problem in Algebraic Geometry by Cluster Computing. In Arndt Bode, Thomas Ludwig, Wolfgang Karl, and Roland Wismüller, editors, 6th International Conference on Parallel Computing (Euro-Par 2000), volume 1900 of Lecture Notes in Computer Science, pages 1196–1200, Munich, Germany, August 29 – September 1, 2000. Springer.Google Scholar
  84. [SMW00c]
    Wolfgang Schreiner, Christian Mittermaier, and Franz Winkler. Plotting Algebraic Space Curves by Cluster Computing. In X.-S. Gao and D. Wang, editors, 4th Asian Symposium on Computer Mathematics, pages 49–58, Chiang Mai, Thailand, December 17-21, 2000. World Scientific Publishers, Singapore.Google Scholar
  85. [Spi]
    On-the-Fly, LTL Model Checking with Spin. Scholar
  86. [SS01]
    Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning). The MIT Press, 2001.Google Scholar
  87. [Sup]
    SCXY Conference Series — The International Conference for High Performance Computing, Networking, Storage, and Analysis. Scholar
  88. [Top]
    Top 500 Supercomputing Sites. Scholar
  89. [UPC]
    Berkeley UPC — Unified Parallel C. Scholar
  90. [VBR06]
    Srikumar Venugopal, Rajkumar Buyya, and Kotagiri Ramamohanarao. A Taxonomy of Data Grids for Distributed Data Sharing, Management, and Processing. ACM Comput. Surv., 38(1):3, 2006.CrossRefGoogle Scholar
  91. [Ven03]
    Suresh Venkatasubramanian. The Graphics Card as a Streaming Computer. CoRR, cs.GR/0310002, 2003.Google Scholar
  92. [Wat08]
    Johannes Watzl. Investigations on Improving the SEE-GRID Optimization Algorithm. Diploma thesis, Research Institute for Symbolic Computation (RISC), Johannes Kepler University, Linz, Austria, June 2008.Google Scholar
  93. [WSH99]
    Rich Wolski, Neil T. Spring, and Jim Hayes. The Network Weather Service: a Distributed Resource Performance Forecasting Service for Metacomputing. Future Generation Computer Systems, 15(5–6):757–768, 1999.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Wolfgang Schreiner
    • 1
  • Károly Bósa
    • 1
  • Andreas Langegger
    • 2
  • Thomas Leitner
    • 2
  • Bernhard Moser
    • 3
  • Szilárd Páll
    • 3
  • Volkmar Wieser
    • 3
  • Wolfram Wöß
    • 2
  1. 1.Research Institute for Symbolic Computation (RISC)Johannes Kepler University Linz (JKU)LinzAustria
  2. 2.Institute for Application Oriented Knowledge Processing (FAW)Johannes Kepler University Linz (JKU)LinzAustria
  3. 3.Software Competence Center Hagenberg (SCCH)HagenbergAustria

Personalised recommendations