====== IPAACA – Incremental processing architecture for artificial conversational agents ======


This page describes the basic concepts underlying IPAACA. For a theoretical background have a look at the research papers that were foundational to IPAACA's design:

  * Schlangen, D. & Skantze, G. (2009). A general, abstract model of incremental dialogue process- ing. In //Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics,// pp. 710–718, Athens, Greece. [[http://www.aclweb.org/anthology/E/E09/E09-1081.pdf|PDF]]
  * Schlangen, D. & Skantze, G. (2011). A general, abstract model of incremental dialogue processing. //Dialogue and Discourse,// 2, 83–111. doi:[[https://doi.org/10.5087/dad.2011.105|10.5087/dad.2011.105]], [[https://search.iczhiku.com/paper/SWkajux5ZBLernr2.pdf|PDF]]

For a short documentation of the command line tools that can be used with IPAACA, see [[public:ipaaca:tools|IPAACA tools]].

==== Overview ====

=== IUs and Messages ===

The basic unit of information transmitted (shared) in the ipaaca system is the **Incremental Unit** (IU), based on the "General, abstract model of incremental dialogue processing" by Schlangen et al. An IU is an object characterized by the following basic attributes:

  * //uid// - a globally unique identifier
  * //**category**// - a string representing the broad category of data, e.g. "asrresults" for transmitting the results of ASR
  * //owner// - the buffer name (see below) that initially produced this IU
  * //committed// - a flag that specifies whether the owner is committed to this IU and its contents (that it will remain valid and is final)
  * //**payload**// - the IU payload: a map of string->JSON object representation, free to use by the application (see below)
  * //links// - a map of string->string, representing the links of the IU (see below) 

IUs are persistent objects. That means that they can be modified at any time (unless specified as read-only by the user), and any changes of published IUs are automatically transmitted as updates to all relevant parties. They can also be modified from the remote side, unless specified otherwise.

The **Message** is a special case of an IU: it is a non-persistent read-only version of the IU. It can be used whenever you just want to send current information (akin to lightweight message-passing systems, hence the name), without the possibility of later modification. The benefit is that Messages are only present for the time of reception and do not occupy additional cumulative recources.

=== Buffers ===

IUs are objects that exist in **Buffers**. A program can have any number of OutputBuffers and InputBuffers. When a new IU has been created, it has to be placed in an OutputBuffer. It is thereby published.

InputBuffers that components have initialized have a list of **category interests**, set by the user. Whenever an IU (or Message) of said categories is published or modified anywhere on the system, the corresponding InputBuffers will receive a notification of this, along with the updated IU contents.

Changes to IUs that are not marked as read-only can be effected at both ends of the pipeline: by simply writing to an IU present in a Buffer. When an IU has been written to from the remote side, the owner of that IU will also be able to receive a notification of this on the OutputBuffer where the IU was originally placed. IUs that have not yet been published can be written to at leisure without generating any events anywhere.

=== Payload: structured user data in IUs ===

The actual contents of an IU are stored in its payload field. The payload is a key-value map, the contents of which are not mandated by the specification aside from the data format. Prior to protocol version 2 (Feb 2015), the payload was a string->string dictionary. Basic transport-layer compatibility with components using this older API is guaranteed. Starting at protocol version 2, the entries in the payload map are now JSON, enabling the use of lightweight structured objects.

=== Links: networks of IUs ===

The IU model offers the possibility of representing (light-weight) dependency networks for IUs. To this effect, ipaaca offers the links member of an IU, a map str->str. The keys inside the hash represent the link type (name), while the values are comma-separated lists of IU UIDs which represent the target IUs of outgoing dependency links of the present IU. For example, and as a reference convention, the "grounded-in" links ("grin") of an IU represent IUs upon which the data of the current IU is based. ( e.g. iu.add_links("grin", "uid-of-the-evidence-iu" ) )

==== Example IU ====

<code>
IU
category: myCategory
uid: c4f74af9-ac0f-44d6-9b04-b9294930f9bb
owner: /ipaaca/component/MySenderIDd17eda72/OB
links: {"grounded-in": ['e2a40dc0-6337-4521-ba15-0e320116910f'] }
payload: {
            "state": "DONE",
            "resultList": [12, 24, 36, 48, 60],
         }
</code>
In this IU, the payload map contains two entries (state and resultList). state is a JSON object of simple string type (directly compatible with older API versions). resultList is a list of integer values. Accessing these data is language dependent, descriptions can be found below.

===== Development =====


=== Windows ===

There are Windows builds of the C++ version of IPAACA. Currently, two flavors exist:
  * old API version, built for Visual Studio 2010, and using rsb0.10 from pre-built 'rsx' (no longer available from CI server in this version)
    * Several Windows projects rely on this at the moment (2016-12), like DragonSpeech and EyeX
  * new (JSON) API version, built for Visual Studio 2015 and using our own pre-built libs
    * Contained dep versions (as of 2016-12): Boost 1_57, Protobuf 2.5, Spread 4.4, RSC/RSB 0.14 (static Boost linkage)
    * Usage:
      - Install Visual Studio 2015 (probably won't include / link using older VS versions)
      - Create a repository folder somewhere (mine is Documents\Develop\repo in my home), and set the SOA_REPO_DIR environment variable for the current user to refer to it (via control panel).
      - Unpack one of the following archives into that folder:
        - 64-bit Release build (recommended): https://www.techfak.uni-bielefeld.de/~ryaghoub/ipaaca-windows-complete-64bit.zip (about 88 MiB; Caveat 64 bit version: currently (2017-02), socket transport seems to be broken (but spread is OK))
        - 32-bit Debug build: https://www.techfak.uni-bielefeld.de/~ryaghoub/ipaaca-windows-complete-32bit.zip (about 150 MiB)
      - The archive contains ipaaca, its dependencies, and a sample C++ Visual Studio 2015 project called "IpaacaWinConsoleTest" as well as a sample C# project called "IpaacaWinCSharpWindow". The C++ project can be used to glean the necessary variable settings, which are provided relative to the SOA_REPO_DIR environment variable. Refer to the rightclick->Properties of the .sln in VS2015 and check the Debugging, C++, and Linker tabs. Set the project to the correct build type (x64 Release for the first archive, x86 Debug for the second one).