The `url` parameter of handle_data() could hold a string or nil when
invoked from handle_event_data(), in which case resolving `url` in a
created event would fail with a type error. Moreover, the logic did not
have any guard for URI errors. This commit should fix them.
Fixes#1765.
If given, it is used as a Liquid template for each event created by the
Agent, instead of directly emitting the results of extraction as events.
An existing spec needs to be fixed because WebsiteAgent now has the
`template` option, which may not be a hash of hashes.
Previously, WebsiteAgent always assumed that a content with no charset
specified in the Content-Type header would be encoded in UTF-8. This
enhancement is to make use of the encoding detector implemented in
Nokogiri for HTML/XML documents, instead of blindly falling back to
UTF-8.
When the document `type` is `html` or `xml`, WebsiteAgent tries to
detect the encoding of a fetched document from the presence of a BOM,
XML declaration, or HTML `meta` tag.
This fixes#1742.
The `as_object` returns the received data/object as is without casting it to a string like liquid normally does. It
can be used as a JSONPath replacement or to emit result of a Liquid filter chain as an array.
`catch` and `throw` needs to be used to break out of Liquid render chain. Liquid aggregates the output of every
expression an array and [joins](https://github.com/Shopify/liquid/blob/v3.0.6/lib/liquid/block.rb#L147) it together that
join makes it impossible to get anything else than a string out of a Liquid template.
The implementation of latest_events() assumed that while it wasn't
actually the case; the association of received_events has a default
scope of `order(id: :desc)`.
FeedNormalizer is no longer maintained, and its Atom support has flaws
in that it throws away what RSS::Parser returns and falls back to using
SimpleRSS which is not capable of handling XML entities, resulting in
getting ususable URLs such as ones including `&`.
Feedjira is highly customizable as it implements parsers for various
feed formats on its own using sax-machine.
The `clean` option is reimplemented using the loofah gem.
`Feedjira::FeedUtilities#sanitize_entries!` is not used because it
tries to sanitize non-HTML properties too.
A new option `include_feed_info` is added, with which turned on feed
information is added to each event payload.
A new key `links` is added, which lists all `link` elements.
A new key `enclosure` is added.
JSONEditor double escapes escape characters to properly display them in the UI, however when dry running Agents we
actually need to send the Event's characters as they were.
Before an event containing new lines like this `Line A\nLineB` would be send as `Line A\\nLineB` to the backend. By
replacing the double escaped slashes we ensure the event is handled as it would be when not dry running.
#1677
* Clarify path for a simple body_text event.
It's not really clear with regard to a normal agent that this is simply the name of the field.
* updated text
Especially when using Agent gems it happens that an Agent was deleted from the gem or the user removed a gem from the
configuration. Instead of failing with an Internal Server Error the user now is offered to delete all undefined agents.
`type.constantize` is needed to invoke the Rails auto_loader in development since not all Agent classes are loaded when
the application boots.
The old serializeObject function serialized checked checkboxes as an array which was not correct and with
strong_parameters enabled it triggered a `UnpermittedParameters` error.
In the AgentEditPage javascript we only need to initialize `select2` when then `#agent_type` field is a select
box (trying to call `select2` on normal text fields caused a javascript error in the feature specs)
* Ensure admin attribute can not be set at sign up and the agents user_id is not changeable
* Remove ACCESSIBLE_ATTRIBUTES from the User model
* Remove side effect on agent_params from AgentsController#build_agent
When `cache_response` is set to `false` for completable fields the returned options will not be
cached on the client side. The default is `true`.
Clearing the cache is needed when the completable response depends on a different option of the Agent.
Usage Example:
```
form_configurable :models, roles: :completable, cache_response: false
```
* Disable the completion callback when `select2` is opened programmatically to avoid an infinite loop
* Moved the completion ajax call from `select2-open` to `select2-opening` to clear the cached results before
`completableDefaultOptions` is called (shows the "Loading ..." message instead of the previous results)
* Extract option prviders for non-standard Omniauth payloads
- should enable huginn_agent gem developers to add their own providers
- devs would reopen OmniauthCallbacksController and define their specific action, supplying their own option provider
* Creates a registry for option providers on the Service class
- enables huginn_agent gem developers to register their own omniauth option interpreters
* Cleans up changes to the global Service class after the spec
* Add language parameter to the weather agent
* Add text forecast for Wunderground data provider
Wunderground API returns forecast for the next few days. Returned JSON response has two types of data under the "forecast" key:
- "txt_forecast": pre-formatted text representation
- "simpleforecast": raw data representation
The "period" key contains which day the node belongs to.
Currently only "simpleforecast" is available to users of the Weather agent.
This commit appends the "txt_forecast" representation to the current data set.
Add new `emit_error_events` option which is set to `false` for all new `TwitterActionAgent`s. This will ensure the Agent
is retried when the action failed by raising an exception.
For all existing `TwitterUserAgent`s the option is set to `true` to keep the previous behavior.
* EmailDigestAgent relies on received events, rather than storing them in memory
- Addressing a problem where 40,000+ events were stored in one agent's memory, and the agent never loaded from the database
* Cleans up references to memory["queue"]
- incorporates suggested style and safeguards
* Adds clarification on event inclusion in email
* Use `reorder` rather than stripping away all scoping
