Trigger (polling)

The Trigger module is a special module that saves the information about what was the last item processed and continues the execution from that item, if there is some.

You can configure the trigger module to:

  • Process all available items and wait for new ones, without repeated processing of the old item.

  • Process items starting from a specific date and time.

  • Process items starting with a specific item.

Use this module when you need to process items sequentially in the order they were created/updated.

Components

pageCommunication
  • Communication response is extended with the trigger object.

response.trigger

The trigger collection specifies directives that will control how the trigger will work and how your data will be processed

Key

Type

Description

date or id

Specifies how the trigger will behave and sort items

asc or desc

Specifies in what order the remote API returns items

IML String

Must return current item’s Id

IML String

When used, must return current item’s date

response.trigger.type

Required: yes Values: id or date

The trigger.type directive specifies how the trigger will sort and iterate through items.

If the processed item has a create/update date, then date should be used as a value, and a correct getter should be specified in trigger.date directive. The trigger will then sort all items by their date and id fields and return only unprocessed items.

If the processed item does not have a create/update date, but only an id, then id should be used as a value, and a correct getter should be specified in trigger.id directive.

response.trigger.order

Required: yes Values: asc, desc or unordered

The trigger.order directive specifies in what order the remote API is returning items - descending, ascending, or unordered. This information is needed to correctly determine if there are more pages to be fetched or not. It is also needed to correctly sort the incoming items and display them to the user in ascending order.

So if the API is returning items in ascending order (low to high), then asc should be used. If the API is returning items in descending order (high to low), then desc should be used. If the API is returning items in no apparent order, then unordered should be used.

When specifying the Trigger's communication, you should sort the results in descendant order, if possible, as the module won't have to page all over the old records to get the new ones.

response.trigger.id

Required: yes

This directive specifies the item’s id. It must always be present. For example, if your item looks like this:

{
    "id": 24,
    "name": "Fred",
    "friend_count": 5
}

then you should specify your trigger.id directive like this: {{item.id}}:

{
    "response": {
        "trigger": {
            "id": "{{item.id}}"
        }
    }
}

response.trigger.date

Required: yes, if the trigger type is date

This directive specifies the item’s date. It must be specified when the trigger.type is set to date. Note that trigger.id must always be specified.

For example, if your item looks like this:

{
    "id": 24,
    "name": "Fred",
    "friend_count": 5,
    "created_date": "2017-07-05T13:05"
}

Then you should specify your trigger.date directive like this: {{item.created_date}}, and your trigger collection might look something like this:

{
    "response": {
        "trigger": {
            "id": "{{item.id}}",
            "date": "{{item.created_date}}"
        }
    }
}

pageEpoch

The Epoch panel is a specific component of the trigger allowing a user to choose the starting item.

pageStatic parameters

The Trigger module can only have static parameters. There's no reason to have anything mappable in the Trigger as this module is always the first module in the scenario.

pageInterface

The Trigger module can return multiple bundles at once.

Samples

pageSamples

To help the users with setting up your module, you can provide samples to it.

Scope

pageScope

When using an OAuth type of connection, use the Scope to define scopes required by this trigger.

Available IML Variables

These IML variables are available for you to use everywhere in this module:

  • now - Current date and time.

  • environment - TBD

  • temp - Contains custom variables created via temp directive.

  • parameters - Contains module’s input parameters.

  • connection - Contains connection’s data collection.

  • common - Contains app’s common data collection.

  • data - Contains module’s data collection.

    • data.lastDate - Returns the date from the last retrieved item in a previous execution.

    • data.lastId - Returns the ID of the last retrieved item in a previous execution.

  • scenario - TBD

  • metadata.expect - Contains module’s raw parameters array the way you have specified it in the configuration.

  • metadata.interface - Contains module’s raw interface array the way you have specified it in the configuration.

Additional variables available to Response Object:

  • output - When using the wrapper directive, the output variable represents the result of the outputdirective.

Additional variables available after using the iterate directive, i.e. in wrapper or pagination directives:

  • iterate.container.first - Represents the first item of the array you iterated.

  • iterate.container.last - Represents the last item of the array you iterated.

In the Trigger module, the iterate.container.last can be used for handling the pagination of the new items correctly.

"response": {
    "limit": "{{parameters.limit}}",
    "output": "{{parseItem(item.data)}}",
    "iterate": "{{body.data.children}}",
    "trigger": {
        "id": "{{item.data.name}}",
        "date": "{{parseDate(item.data.created_utc, 'X')}}",
        "type": "date",
        "order": "desc"
    }
},
"pagination": {
    "qs": {
        "after": "{{iterate.container.last.data.name}}"
    }
}

Additional variables available to pagination and response objects:

  • body - Contains the body that was retrieved from the last request.

  • headers - Contains the response headers that were retrieved from the last request.

  • item - When iterating this variable represents the current item that is being iterated.

Example

Last updated