SpringerOpen Newsletter

Receive periodic news and updates relating to SpringerOpen.

This article is part of the series Embedded Vision System.

Open Access Open Badges Research Article

Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

Mainak Sen1*, Ivan Corretjer1, Fiorella Haim1, Sankalita Saha1, Jason Schlessman2, Tiehan Lv2, Shuvra S Bhattacharyya1 and Wayne Wolf2

Author Affiliations

1 Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA

2 Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA

For all author emails, please log on.

EURASIP Journal on Embedded Systems 2007, 2007:049236  doi:10.1155/2007/49236

The electronic version of this article is the complete one and can be found online at: http://jes.eurasipjournals.com/content/2007/1/049236

Received:1 May 2006
Revisions received:8 October 2006
Accepted:9 October 2006
Published:28 January 2007

© 2007 Sen et al.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.


  1. S Sriram, SS Bhattacharyya, Embedded Multiprocessors: Scheduling and Synchronization (Marcel Dekker, New York, NY, USA, 2000)

  2. M Sen, SS Bhattacharyya, T Lv, W Wolf, Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5, 133–136

  3. SS Bhattacharyya, R Leupers, P Marwedel, Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 47(9), 849–875 (2000). Publisher Full Text OpenURL

  4. E Lee, D Messerschmitt, Synchronous data flow. Proceedings of the IEEE 75(9), 55–64 (1987). Publisher Full Text OpenURL

  5. G Bilsen, M Engels, R Lauwereins, J Peperstraete, Cyclo-static dataflow. IEEE Transactions on Signal Processing 44(2), 397–408 (1996). Publisher Full Text OpenURL

  6. JT Buck, A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France, 165–172

  7. B Bhattacharya, SS Bhattacharyya, Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 49(10), 2408–2421 (2001). Publisher Full Text OpenURL

  8. M Gokhale, J Stone, J Arnold, M Kalinowski, Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA, 49–56

  9. CAR Hoare, Communicating Sequential Processes (Prentice-Hall, New York, NY, USA, 1985)

  10. S Chappell, C Sullivan, Handel-C for co-processing & co-design of Field Programmable System on Chip. White Paper (Celoxica, Oxford, UK, 2002)

  11. P Banerjee, D Bagchi, M Haldar, A Nayak, V Kim, R Uribe, Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA, 484–487

  12. B Kienhuis, E Rijpkema, E Deprettere, Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA, 13–17

  13. G Kahn, The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden, 471–475

  14. EA Lee, Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA, 295–306

  15. W Wolf, B Ozer, T Lv, Smart cameras as embedded systems. Computer 35(9), 48–53 (2002). Publisher Full Text OpenURL

  16. J Eker, JW Janneck, EA Lee, et al. Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 91(1), 127–144 (2003). Publisher Full Text OpenURL

  17. F Haim, M Sen, D-I Ko, SS Bhattacharyya, W Wolf, Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3, 1052–1055

  18. J Horstmannshoff, T Grötker, H Meyr, Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland, 283–292

  19. M Sen, SS Bhattacharyya, Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5, 229–232

  20. MC Williamson, EA Lee, Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2, 1340–1343

  21. Data-sheet for ZBT memory, http://www.samsung.com/Products/

  22. OpenCores Organization, WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores (revision B), . 3, September 2002, http://www.opencores.org webcite

  23. AK Jain, Fundamentals of Digital Image Processing (Prentice-Hall, New York, NY, USA, 1989)

  24. IEEE Working Group, (http://www), . eda.org/vhdl-200x/vhdl-200x-ft/packages/files.html webcite