Commit graph

1638 commits

Author SHA1 Message Date
Dominik Sander
77842eae64 Merge pull request #1889 from dsander/rename-scope
Rename Delayed::Job failed scope to prevent warning
2017-01-31 11:19:38 +01:00
Alex Jordan
5eef730904 Add titles to all pages (#1884)
Fixes #1882
2017-01-30 21:07:37 -05:00
Dominik Sander
c023c08cb1 Rename Delayed::Job failed scope to prevent warning
Fixes #1806
2017-01-30 13:11:13 +01:00
Dominik Sander
9048f5e7b0 Merge pull request #1885 from strugee/abort-to-cancel
Change "Abort" button to say "Cancel"
2017-01-30 11:32:01 +01:00
Thiago Talma
51b9bbca17 Allow Redirect Requests (#1881)
* Allow Redirect Requests

* specs

* type cast
2017-01-28 16:58:41 -05:00
Tsao
80debd63bb let default table inherit the boostrap #table css style (#1883)
* let default inherit boostrap #table css style

* move the default table style inside of .well.description
2017-01-28 16:43:04 -05:00
Alex Jordan
70e4abfb87
Change "Abort" button to say "Cancel" 2017-01-28 16:19:15 -05:00
Dominik Sander
e2917e8854 Fix scenario import when merges are required
Fixes #1875
2017-01-17 22:00:38 +01:00
Andrew Cantino
af5c5b5f62 Odds and ends (#1866)
* Fix brittle spec

* remove unused module

* Fix another flaky test
2017-01-08 13:55:05 -05:00
Andrew Cantino
5680547ed2 Credential create should use ace too (#1865) 2017-01-08 12:40:12 -05:00
Andrew Cantino
3ea44cb8bb Fix #1863 validation issue with schedule names (#1864)
* Fix #1863 validation issue with schedule names

* Expose as a class method
2017-01-08 11:13:25 -05:00
Enfop
b8765fbd31 default selected value for scenario.icon select (#1861) 2017-01-06 20:05:47 -05:00
Dominik Sander
89a9f2594f Merge pull request #1844 from dsander/fixed-schedule-times
Run every_X schedules on fixed times
2017-01-05 17:42:00 +01:00
Akinori MUSHA
a5874da0ae Expose Agent#id to Liquid (and to JavaScriptAgent) 2017-01-05 23:56:42 +09:00
Jin Liu
ed85ad5ed3 Allow weibo publish agent tweet with a picture (#1336)
* can publish with pictures

* refactor and add specs

* Take care of the case where most_recent_event is nil.

* Sleep when necessary
2017-01-03 07:54:42 -05:00
Dominik Sander
51f2d095df Merge pull request #1847 from dsander/tweet-mode-extended
Ensure Twitter REST API calls always get extended tweets
2017-01-03 09:51:39 +01:00
Akinori MUSHA
f8cf6bf9c6 Add a new option include_sort_info
If enabled, all events created by the Agent will have a `sort_info` key
whose value is a hash containing the keys `position` and `count`.

This overrides #1768.
2017-01-01 03:27:24 +09:00
Akinori MUSHA
fe3b43ba56 Merge branch 'master' into fix_http_status_agent 2017-01-01 02:54:59 +09:00
Akinori MUSHA
21c166f455 Do not let values in a received event override option values
There's no need to, because Agent options can interpolate values in an
event payload.
2017-01-01 02:48:33 +09:00
Dominik Sander
d700afa9b7 Ensure Twitter REST API calls always get extended tweets
Tweets that include embedds are truncated by default https://dev.twitter.com/overview/api/upcoming-changes-to-tweets
by passing tweet_mode=extened to the REST API calls we ensure to get the full response including embedded images and
videos
2016-12-30 13:22:37 +01:00
Dominik Sander
1f4d54a112 Run every_X schedules on fixed times
Makes the schedules more predictable by running them on fixes hours/minutes instead of X seconds after startup.

Changed the "every 7d" schedule to "every monday" night to make it clearer when it runs because every 7 days is ambiguous
without a explicit start date.

Fixes #1103

Possibly related to #1840

Example runtimes:

```
every_1m (*/1 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:29:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:31:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:33:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:37:00 +0100

every_2m (*/2 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:38:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:42:00 +0100
2016-12-29 11:44:00 +0100
2016-12-29 11:46:00 +0100

every_5m (*/5 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:45:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 11:55:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:05:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:15:00 +0100

every_10m (*/10 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:20:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 12:40:00 +0100
2016-12-29 12:50:00 +0100
2016-12-29 13:00:00 +0100

every_30m (*/30 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 13:30:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 14:30:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 15:30:00 +0100
2016-12-29 16:00:00 +0100

every_1h (0 */1 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 17:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 19:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 21:00:00 +0100

every_2h (0 */2 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 22:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 02:00:00 +0100
2016-12-30 04:00:00 +0100
2016-12-30 06:00:00 +0100

every_5h (0 */5 * * *)
2016-12-29 15:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 05:00:00 +0100
2016-12-30 10:00:00 +0100
2016-12-30 15:00:00 +0100
2016-12-30 20:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 05:00:00 +0100
2016-12-31 10:00:00 +0100

every_12h (0 */12 * * *)
2016-12-29 12:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 12:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 12:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-01 12:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-02 12:00:00 +0100
2017-01-03 00:00:00 +0100

every_1d (0 0 */1 * *)
2016-12-30 00:00:00 +0100
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-04 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-06 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-08 00:00:00 +0100

every_2d (0 0 */2 * *)
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-11 00:00:00 +0100
2017-01-13 00:00:00 +0100
2017-01-15 00:00:00 +0100
2017-01-17 00:00:00 +0100

every_7d (0 0 * * 1)
2017-01-02 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-16 00:00:00 +0100
2017-01-23 00:00:00 +0100
2017-01-30 00:00:00 +0100
2017-02-06 00:00:00 +0100
2017-02-13 00:00:00 +0100
2017-02-20 00:00:00 +0100
2017-02-27 00:00:00 +0100
2017-03-06 00:00:00 +0100
```
2016-12-29 21:52:25 +01:00
Dominik Sander
070946e879 Work around possible regression in rails
`pluck` used to always trigger a database query. With Rails 5.0.1 it uses data from
already loaded associations, this also includes new records for which we can not
generate agent_paths
2016-12-27 23:40:09 +01:00
Andrew Cantino
65cea03062 Fix spec failures 2016-12-20 13:32:48 -05:00
Irfan Charania
4e2d1775a6 PhantomJs Cloud Agent (#1503)
* Initial draft of PhantomJsCloudAgent

Generates event with url for fetching html/plainText content

* Add options

* Pass in event instead of url
Fix hash syntax
Remove whitespace
Add mode merge

* Add some tests

* Style changes

- Add link to wiki entry for manually creating agent with full set of
options
2016-12-20 12:44:54 -05:00
Akinori MUSHA
2a524abff5 Fix the regex pattern to reduce backtracking
This should fix #1832.
2016-12-20 10:27:51 +09:00
Akinori MUSHA
3612ba7333 Add podcast tags to events emitted by RssAgent
The keys are only added when the feed is a podcast.
2016-11-29 23:59:32 +09:00
Andrew Cantino
8a14a57e00 Beeper.io is no more (#1808)
* Beeper.io is no more

* Avoid event propagation or scheduling of missing agents

* Update undefined_agents.html.erb with a link to the wiki
2016-11-27 15:30:50 -05:00
Akinori MUSHA
5f5f3cd38f Do not err if headers is a valid headers hash 2016-11-27 13:38:23 +09:00
Akinori MUSHA
9074f3115e Make sure status is an integer when set 2016-11-27 13:38:23 +09:00
Akinori MUSHA
a94cd7fd6d Allow an empty or null base URI 2016-11-27 13:38:23 +09:00
Akinori MUSHA
3a0c9e6274 Disable automatic URL normalization and absolutization on url
This was discussed in #1766.

For backward compatibility, existing WebsiteAgents with a key named
`url` will be given a `template` to resolve `url`.
2016-11-27 13:38:23 +09:00
Akinori MUSHA
3d91469733 Make WebsiteAgent merge template with the results of extract (#1816)
A new extraction option `hidden` is added so that keys with it gets
excluded from the final payloads while they can be used in `template`.
2016-11-27 13:33:24 +09:00
bobbysteel
f530305edc Add class of service chooser for Google Flights Agent (#1778)
* Add class of service chooser

* Add cabin chooser test

* Fix preferredCabin

* Per @cantino feedback taking out check
2016-11-26 15:13:00 -05:00
Akinori MUSHA
6d4fdd3196 Use url and headers_to_save in a received event if included
That seems to be what was actually intended, judging from the specs.

See [my comment](https://github.com/cantino/huginn/pull/1521#discussion_r86488243) in #1521.
2016-11-23 10:06:29 +09:00
Akinori MUSHA
cc28f780c5 Make headers_to_save case-insensitive
See [my comment](https://github.com/cantino/huginn/pull/1521#discussion_r86489253) in #1521.
2016-11-23 10:06:29 +09:00
Akinori MUSHA
833985d723 Fix a bug where redirected is always true
This key was introduced in #1590 but it is likely that it has never
worked as expected.
2016-11-23 10:06:29 +09:00
Akinori MUSHA
b8d88aa9a3 Merge pull request #1813 from cantino/fix_decoding_in_rss_agent
Fix a double-decoding problem in RssAgent
2016-11-23 09:05:46 +09:00
Dominik Sander
5d69bd2d93 Merge pull request #1804 from dsander/cache-agent-type-dropdown
Cache Agent type select options in Agent#new
2016-11-22 21:04:29 +01:00
Dominik Sander
6daa6cc75e Cache Agent type select options in Agent#new
Agent#new was by far the slowest action of all controllers. When using
many Agent gems the response time goes up to a point where the user
starts to wonder if something is going wrong. By caching the Agent
description the response time goes down from about 1 second to 100ms
in development.
2016-11-22 20:02:27 +01:00
Akinori MUSHA
aa6d8be697 Assume that an XML declaration is at the beginning of a document 2016-11-23 01:17:46 +09:00
Akinori MUSHA
0b3700999b Fix a double-decoding problem in RssAgent
The SAX parser Feedjira uses (Nokogiri::XML::SAX) tries to detect the
encoding of a document from the content even if it is already known
and given.  This results in a content being decoded twice by
WebRequestConcern and the SAX parser if its encoding is declared in
both the Content-Type header and the XML declaration.

This commit makes RssAgent remove the `encoding` attribute from the
XML declaration of a document if the encoding is already known by the
Content-Type header.

Fixes #1797.
2016-11-22 12:14:28 +09:00
Akinori MUSHA
c575af959b Merge pull request #1769 from cantino/website_agent_repeat_option
Add a `repeat` option for extractors to WebsiteAgent
2016-11-20 19:16:36 +09:00
The Doctor
5e1191534c Fixed the online documentation for the Weather Agent class. (#1803)
Signed-off-by: The Doctor <drwho@virtadpt.net>
2016-11-19 10:39:15 -05:00
Akinori MUSHA
bd9455d5d0 Add a repeat option to extractors
This allows user to include a value that only appears once in a content
in all events created from the content.
2016-11-18 18:47:32 +09:00
Akinori MUSHA
3a66c152ef Make extract_each prepare a storage for results 2016-11-18 16:34:45 +09:00
Andrew Cantino
770706463b Prevent submit from disabling on invalid json (#1790) 2016-11-13 15:21:43 -05:00
Akinori MUSHA
74077b0ad4 Auto-focus on Agent Type when creating an agent 2016-11-09 23:25:55 +09:00
Akinori MUSHA
486246e63c Add "image" to Event Description 2016-11-07 12:47:13 +09:00
Akinori MUSHA
1e70b31e7f Merge pull request #1770 from cantino/revert-1071
Revert the special treatment for CDATA introduced in #1071
2016-11-03 12:14:42 +09:00
Akinori MUSHA
50123dca53 Fix event_description broken in full JSON mode or without a template 2016-11-02 21:47:01 +09:00