====== Programming libraries for audio, graphics and media development ======


.

**Here are listed some libraries and APIs, that may be helpful when developing audio, video and media applications.**


//

Note: This Page should include only libraries (APIs) that are [[http://www.linfo.org/cross-platform.html| cross-platform]] and have an [[http://www.opensource.org/licenses/category| OSI]] or [[http://www.fsf.org/| FSF]] ([[http://fsfe.org/|FSFE]]) accepted open-source license.

//

{{topic>media development}}

.

.


===== libraries for audio and midi output =====

 **Jack audio API** - [[http://jackaudio.org/]] - cross platform audio/midi IO

 **Portmedia** - [[http://portmedia.sourceforge.net/]] with its sub-projects - [[http://www.portaudio.com/| port audio]] and [[http://sourceforge.net/apps/trac/portmedia/wiki/portmidi| port midi]]

 **Pulseaudio** - [[http://pulseaudio.org/]] .

 **OpenML** - [[http://www.khronos.org/openml/]]

 **RtMidi** - [[http://www.music.mcgill.ca/~gary/rtmidi/]] - Cpp library for midi IO

 **WebAudio** - [[http://www.w3.org/TR/webaudio/]] - W3C WebAudio API 2012
\\

\\

===== libraries for audio processing and signal processing (DSP) =====

  * [[http://code.google.com/p/ugen/| UGen++ audio processing API ]] synthesis framework in C++. It provides oscillators, filters, envelopes and event management etc.

  * [[http://code.google.com/p/pyo/| Python digital signal processing module ]] written in C to help digital signal processing script creation. 

 * [[http://jamoma.org/ | Jamoma DSP ]] - interface for C++, with an emphasis on real-time signal processing

 * [[http://www.flatmax.org/gtkiostream/html/JackClientTest_8C-example.html | gtkIOStream jackd client ]] - trivialises using jackd for audio input and output DSP processing.

\\

\\

===== Articles =====
  * [[http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing| real-time audio programming 101]]

\\
\\

===== Libraries for Atomic Operations =====
\\
'Atomic Operations' are not allowed to be interrupted (e.g. by other threads).

  * [[http://developer.gnome.org/glib/unstable/glib-Atomic-Operations.html| Atomics in glib ]]
  * [[https://trac.mcs.anl.gov/projects/openpa/| OpenPA - Portable Atomics ]]
  * [[http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html| gcc - Atomic Builtins ]]
  * [[http://www.hpl.hp.com/research/linux/atomic_ops/| HP atomic_ops project ]]
  * [[http://www.chaoticmind.net/~hcb/projects/boost.atomic/| Boost Atomic (C++ or C++11 or C++0x) ]]
  * [[http://www.boost.org/doc/libs/1_39_0/boost/interprocess/detail/atomic.hpp| Boost Atomic.hpp (c++) ]]
  * [[http://threadingbuildingblocks.org/| Intel® Threading Building Blocks (Intel TBB) ]]
  * [[http://lttng.org/urcu| Userspace RCU library]] - User-level Read-Copy Update, Lock-free data structures (RCU lock-free hash table with auto-resize, queues, stacks), atomic operations, efficient user-level wait/wake-up
  * [[http://www.ioremap.net/node/224| libsync - Atomic operations and userspace RCU ]]
  * [[http://www.pjsip.org/docs/latest/pjlib/docs/html/index.htm| PJLIB ]] - A small footprint framework library written in C for making scalable applications. Includes Atomic Ops.
  * [[http://www.flatmax.org/gtkiostream/html/ThreadTest_8C-example.html| gtkIOStream thread]] and [[http://www.flatmax.org/gtkiostream/html/classMutex.html| gtkIOStream mutex ]] - Set of mutex and thread C++ headers to trivialise threading. Can use either pthreads or GLib threads.

\\

\\
===== Lockfree (non-blocking) Data Structures - Libraries =====
\\
  * [[http://lttng.org/urcu| Userspace RCU (read-copy-update) ]]
  * [[http://lwn.net/Articles/262464/| What is RCU - Article ]]
  * [[http://sourceforge.net/projects/mc-fastflow/| FastFlow - Lockfree FIFO Queues ]]
  * [[http://atomic-ptr-plus.sourceforge.net/| Atomic Ptr Plus ]]
  * [[http://code.google.com/p/nbds/| nbds - non-blocking data structures ]]
  * [[http://www.liblfds.org/| libLFDS ]]
  * [[http://www.cl.cam.ac.uk/research/srg/netos/lock-free/| lock-free library of the University Cambridge ]]
  * [[https://dev.longitekk.com/Rig/| Rig library ]] - lock-free data structures, safe strings and misc others C library
  * [[http://apr.apache.org/| ARP - Apache Portable Runtime ]]
  * [[http://webpages.charter.net/appcore/vzoom/refcount/| RefCount library ]] - lock-free word-based atomic reference counting
  * [[http://www.sunrisetel.net/software/devtools/sunrise-data-dictionary.shtml| Sunrise Data Dictionary for C ]] - Library for lock-free hash table storage of arbitrary data
  * [[http://tim.klingt.org/code/projects/boost-lockfree| Boost.Lockfree ]] - A boost-like, but NOT boost, library for lockfree structs.
  * [[http://www.noble-library.org/| Noble library ]] - a library of non-blocking synchronization protocols 
  * [[http://lttng.org/urcu| Userspace RCU library liburcu-cds]] - Userspace RCU Lock-free data structures (RCU lock-free hash table with auto-resize, queues, stacks)
  * [[https://github.com/OpenMusicKontrollers/varchunk| varchunk ]] - rt-safe, lock-free, zero-copy, contiguous-chunk ringbuffer optimized for event handling. 

\\

===== Parallel Programming =====

\\
  * [[http://www.par4all.org/features/| Par4all compiler ]] - Generates OpenCL, CUDA or OpenMP parallel code from (sequenciel) C code.
\\
  * [[http://openmp.org/wp/about-openmp/| OpenMP (Open Multi-Processing) ]]  - OpenMP is an API (Application Programming Interface) that supports multi-platform shared memory multiprocessing programming. 
  * [[http://gcc.gnu.org/projects/gomp/| Gomp - GNU implementation of the OpenMP spec. ]]
  * [[http://en.wikipedia.org/wiki/OpenMP| (WP) OpenMP ]]
\\
  * [[http://www.open-mpi.org/| Open-MPI (message passing interface) ]]  -  MPI addresses primarily a message-passing parallel programming model.
  * [[http://www.mpi-forum.org/docs/docs.html| MPI 2.2 specification ]]
  * [[http://www.mcs.anl.gov/research/projects/mpich2/| MPICH2 - open MPI implementation ]]
  * [[http://code.google.com/p/mpi4py/| mpi 4 py  ]] - MPI for Python - [[http://mpi4py.scipy.org/| mpi 4 py Documentation]]
  * [[http://en.wikipedia.org/wiki/Message_Passing_Interface| (WP) MPI - Message Passing Interface]]
  * [[http://www.faqs.org/faqs/mpi-faq/| MPI FAQ]]
\\
  * [[http://www.mpirt.org/| MPI-RT 1.1 spec. ]] - real-time message-passing-interface v1.1 (an implementation doesn't exist !?)
\\
  * [[http://www.khronos.org/opencl/| OpenCL ]]  - Standard for cross-platform, parallel programming of modern processors.
  * [[http://www.alphaworks.ibm.com/tech/opencl| OpenCL develop-kit Linux ]]  - OpenCL defines an extended C language for writing compute kernels and a set of APIs for creating and managing these kernels.
  * [[http://www.khronos.org/registry/cl/| OpenCL API ]]
  * [[http://www.macresearch.org/opencl_episode1| OpenCL Tutorial ]]
  * [[http://mathema.tician.de/software/pyopencl| PyOpenCL ]] - use OpenCL parallel computation API from Python
  * [[http://packages.python.org/pyfft/| PyFFT ]] - fast fourier transform with OpenCL (or Cuda) using Python
\\
\\
  * [[http://upc.gwu.edu/| Unified Parallel C]]  - UPC is an extension of the C programming language designed for high-performance computing.
  * [[http://www.gccupc.org/| GCC-UPC ]]  - GNU implementation of the UPC language spec.
\\
  * [[http://libcsp.sourceforge.net/| CSP For Posix Threads]] - "<<Communicating Sequential Processes>> is an algebra for analysing the behaviour of concurrent systems ... a library ...  that makes programming with threads much easier."
\\
\\
  * [[https://computing.llnl.gov/tutorials/parallel_comp/| Parallel Computing Manual ]]
  * [[http://users.evtek.fi/~tk/rtp/parallel-programming-theory.html| Parallel Programming - Basic Theory ]]
\\
\\
**see also: **
  * [[http://www.concurrencykit.org/doc/| The concurrency kit - An online Manual ]]
  * [[http://en.wikipedia.org/wiki/Inter-process_communication#Main_IPC_methods| (WP) IPC Methods (Inter-Process Communication)]]
  * [[http://www.cs.kent.ac.uk/projects/ofa/c%2B%2Bcsp/]] - CppCSP (Communicating Sequential Processes) - Easy Concurrency for Cpp
  * [[http://en.wikipedia.org/wiki/List_of_concurrent_and_parallel_programming_languages| (WP) concurrent programming languages 1]]
  * [[http://en.wikipedia.org/wiki/Concurrent_computing#Concurrent_programming_languages| (WP) Concurrent programming languages 2]]
  * [[http://en.wikipedia.org/wiki/Communicating_Sequential_Processes| (WP) CSP - Communicating sequential processes]]
  * [[http://tldp.org/HOWTO/RTLinux-HOWTO-6.html|| RTLinux IPC]]
  * [[https://computing.llnl.gov/tutorials/pthreads/| Posix Threads ]]
  * [[http://tldp.org/LDP/tlk/ipc/ipc.html| IPC Mechanisms ]] (Inter-Process-Communication)
\\

===== Session handling =====

\\
Restore application-state and audio/midi connections.
\\

There are at least three different APIs for session handling in linux.
They allow to save and restore the state of different applications, including midi and audio connections !
\\
  * JackSession - is kind of basic, requires applications to be restarted
  * Ladish - is advanced, has a concept of storing session-"rooms"
  * NSM - has most of the really useful functions (based on OSC) - does not require apps to be restarted
\\
For more information, visit the projects websites:

  * [[http://non.tuxfamily.org/| NSM - The NON Session-Manager ]] - comes as part of NON-DAW - [[http://non.tuxfamily.org/nsm/API.html| API Link]] and [[/apps/all/non-session-manager| screenshot]]
  * [[http://trac.jackaudio.org/wiki/WalkThrough/Dev/JackSession| JackAudio Session Handler]]
  * [[http://ladish.org/| ladish - Linux Audio Dev. Session Handler]]
\\

\\

===== Thread management and CPU-affinity in linux =====

  * there is a tool called schedtool, which allows to "set all CPU scheduling parameters Linux is capable of or display information for given processes." 
  * also to mention: nice and renice tools (part of the "bsdutils" package)
  * and linux API functions: sched_setscheduler(), sched_setaffinity()
\\
Related MAN-Pages:

  * [[http://manpages.ubuntu.com/manpages/precise/man8/schedtool.8.html| schedtool]] (formerly: schedutils)
  * [[http://manpages.ubuntu.com/manpages/precise/man1/perf-sched.1.html| perf-sched]] - Trace or measure scheduler properties (latencies) (perf in package "linux-tools-common")
  * [[http://manpages.ubuntu.com/manpages/precise/man2/sched_setaffinity.2.html| sched_setaffinity]]
  * [[http://manpages.ubuntu.com/manpages/precise/man2/sched_setscheduler.2.html| sched_setscheduler]]
  * [[http://manpages.ubuntu.com/manpages/precise/de/man1/nice.1.html| nice ]] - note: don't confuse nice priorities with real-time-priorities (in package "bsdutils")
  * [[http://linux.die.net/man/1/chrt| chrt (part of "util-linux" package]] - a tool to maniplate real-time attributes (priorities)

\\
Related Articles:
  * [[http://www.thinkingparallel.com/2006/08/18/more-information-on-pthread_setaffinity_np-and-sched_setaffinity/| example ]]
  * [[http://www.ibm.com/developerworks/linux/library/l-affinity/index.html| Take charge of processor affinity]]
  * [[http://www.gnugeneration.com/books/linux/2.6.20/kernel-api/| The Linux-Kernel API (2.6.20) ]]
  * [[https://rt.wiki.kernel.org/| real-time kernel wiki ]] - more tools and information
  * [[http://linuxgazette.net/133/saha.html| About Linux Processes ]]
  * [[http://www.cs.rochester.edu/~sanders/linux-scheduler-proj/lottery_scheduler/node3.html| task_struct]]
  * [[http://www.linuxjournal.com/article/6799| CPU Affinity - LinuxJournal ]]
  * [[http://students.mimuw.edu.pl/SO/LabLinux/PROCESY/ZRODLA/sched.h.html| sched.h - header file]]
  * [[http://www.gnu.org/software/libc/manual/html_node/Priority.html#Priority| GNU C - CPU Priority And Scheduling]]
  * [[http://www.oreilly.de/german/freebooks/linuxdrive2ger/book1.html| Linux Driver HowTO (german!) ]]
  * [[http://www.cyberciti.biz/tips/setting-processor-affinity-certain-task-or-process.html| Setting processor affinity]]
  * [[http://stackoverflow.com/questions/939778/linux-api-to-list-running-processes| list running processes ]]
  * [[http://kernelnewbies.org/KernelHacking| kernel hacking]]
\\
  * [[http://www.open-mpi.org/projects/hwloc/| hwloc - portable multi threading ]]
\\
Part of an Answer at [[http://stackoverflow.com/questions/766395/how-does-sched-setaffinity-work| stackoverflow.com]] :
<code>

sched_setaffinity() simply tells the scheduler which 
CPUs is that process/thread allowed to run on, then calls 
for a re-schedule.

The scheduler actually runs on each one of the CPUs, so it 
gets a chance to decide what task to execute next on 
that particular CPU.

If you're interested in how you can actually call some code 
on other CPUs, I suggest you take a look at 
smp_call_function_single(). In case we want to call something 
on another CPU, this calls generic_exec_single(). 
The latter simply adds the function to the target CPU's 
call queue and forces a reschedule through some 
IPI stuff (if the queue was empty).

Bottom line is: there no actual SMP variant of 
the _jmp_ instruction. Instead, code running on other CPUs 
cooperates in order to accomplish the task.

</code>

\\
\\
\\

===== GUI libraries - create graphical user interfaces =====

Comment: additional info would be nice, for example: does it support skins/themes, download size, is it for graphics only or a whole framework...


==== 2-dimensional ====

 **Juce** (framework) - [[http://www.rawmaterialsoftware.com/juce.php| Juce]]  and [[http://www.anticore.org/jucetice/| Juced ]]

 **Sfml** - [[http://www.sfml-dev.org/features.php]] .

 **Lgi** - [[http://www.memecode.com/lgi.php]]

 **OpenFrameworks** (framework) - [[http://www.openframeworks.cc/about]] .

 **Fox toolkit** - [[http://www.fox-toolkit.org/]] .

 **Qt (framework)** - [[http://qt.nokia.com/products]] .

 **Gtk** - [[http://www.gtk.org/]] .

 **Fltk** - [[http://www.fltk.org/]] .

 **WxWidgets** - [[http://www.wxwidgets.org/]] .

 **Ultimate++** - [[http://ultimatepp.org/www$uppweb$overview$en-us.html]] - Cpp cross-platform Framework .

 **gtkIOStream** - [[http://sourceforge.net/projects/gtkiostream/]] - Inline headers for C++ iostream style GTK GUI building .

==== with 3-dimensional capabilities ====

 **LibNui** - [[http://www.libnui.net/pages/about.php]] .

 **Coin3d** - [[http://www.coin3d.org/lib/about_coin3d/coin3d-overview]] .

 **Clutter** - [[http://www.clutter-project.org/about]] .

 **COGL** - [[http://www.clutter-project.org/documentation]] (is a part of Clutter).

 **Visualization Library** - [[http://www.visualizationlibrary.com/jetcms/]]

 **VTK** - [[http://www.vtk.org/]]

 **OpenInventor** - [[http://oss.sgi.com/projects/inventor/]] .

 **Agar** - [[http://wiki.libagar.org/wiki/Main_Page]] - [[http://libagar.org/index.html.en]] .

 **SDL** - [[http://www.libsdl.org/]] .

 **glfw** - [[http://www.glfw.org/]]

 **guichan** - [[http://guichan.sourceforge.net/wiki/index.php/About]]

 **glam** - [[http://glam.sourceforge.net/]] (rel.2004-02)

 **gigi** - [[http://gigi.sourceforge.net/]] (rel.2008-03)

 **GLV gui** - [[http://mat.ucsb.edu/glv/]]

 **MyGUI** - [[http://www.ogre3d.org/addonforums/viewtopic.php?f=17&t=7488]] (rel.2010-02) uses OGRE game-lib

 **MiniGUI** - [[http://www.minigui.org/res.shtml]]  for embedded devices

 **ClanLib** - [[http://clanlib.org/wiki/Main_Page]]  game-lib with gui-toolkit

 **CEGUI** - [[http://www.cegui.org.uk/wiki/index.php/Main_Page]]

 **Allegro** - [[http://alleg.sourceforge.net/readme.html]]


.


==== Targeted at embedded devices ====

 **Disko** - [[http://www.diskohq.com/]]  (embedded library with OpenGL support)

 **MeeGo** - [[http://meego.com/about]]  (for embedded)

 **Moblin** - [[http://moblin.org/about-moblin]] Moblin joined MeeGo

 **MoSync** - [[http://www.mosync.com/]] - Cpp, for Smartphones etc.

 **Kivy** - [[http://kivy.org/]] - Python library for rapid development, innovative ui, multi-touch apps

 **Tizen** - [[https://www.tizen.org/]] - "Environment for application developers, based on HTML5. "


.


==== SVG (vector graphics) ====

 **AGG Anti Grain Geometry** - [[http://www.antigrain.com/]]

 **GGI** - [[http://www.ggi-project.org/]] - graphic-system abstraction layer

 **Cairo Graphics** - [[http://cairographics.org/]]

 **Batik** - [[http://xmlgraphics.apache.org/batik/]]

 **LibSugar** - [[http://libsugar.sourceforge.net/]]

 **librsvg** - [[http://librsvg.sourceforge.net/]]

.

===== misc libraries =====

 **OpenGL Mathematics library (GLM)** - [[http://glm.g-truc.net/about.html]]

 **CML (Configurable Math Library) ** - [[http://cmldev.net/?page_id=7]]

 **TLSF** - [[http://rtportal.upv.es/rtmalloc/]] - for real-time memory allocation

.

.