Commit graph

2675 commits

Author SHA1 Message Date
Enfop
b8765fbd31 default selected value for scenario.icon select (#1861) 2017-01-06 20:05:47 -05:00
Dominik Sander
89a9f2594f Merge pull request #1844 from dsander/fixed-schedule-times
Run every_X schedules on fixed times
2017-01-05 17:42:00 +01:00
Akinori MUSHA
3e8a40c2b1 Merge pull request #1860 from cantino/expose_agent_id
Expose Agent#id to Liquid (and to JavaScriptAgent)
2017-01-06 01:16:39 +09:00
Dominik Sander
1ecb78dfd3 Update rufus-scheduler to released version 2017-01-05 16:58:13 +01:00
Akinori MUSHA
a5874da0ae Expose Agent#id to Liquid (and to JavaScriptAgent) 2017-01-05 23:56:42 +09:00
Lucas Verney
2dab52ae9b Make migrations compatible with SQLite (#1842) 2017-01-03 07:55:36 -05:00
Jin Liu
ed85ad5ed3 Allow weibo publish agent tweet with a picture (#1336)
* can publish with pictures

* refactor and add specs

* Take care of the case where most_recent_event is nil.

* Sleep when necessary
2017-01-03 07:54:42 -05:00
Dominik Sander
51f2d095df Merge pull request #1847 from dsander/tweet-mode-extended
Ensure Twitter REST API calls always get extended tweets
2017-01-03 09:51:39 +01:00
Akinori MUSHA
a824bc9c5f Merge pull request #1772 from cantino/include_sort_info
Add a new option `include_sort_info`
2017-01-01 03:55:31 +09:00
Akinori MUSHA
f8cf6bf9c6 Add a new option include_sort_info
If enabled, all events created by the Agent will have a `sort_info` key
whose value is a hash containing the keys `position` and `count`.

This overrides #1768.
2017-01-01 03:27:24 +09:00
Akinori MUSHA
c0509ba3fe Merge pull request #1776 from cantino/fix_http_status_agent
Fix HttpStatusAgent
2017-01-01 03:26:51 +09:00
Akinori MUSHA
fe3b43ba56 Merge branch 'master' into fix_http_status_agent 2017-01-01 02:54:59 +09:00
Akinori MUSHA
21c166f455 Do not let values in a received event override option values
There's no need to, because Agent options can interpolate values in an
event payload.
2017-01-01 02:48:33 +09:00
Dominik Sander
d700afa9b7 Ensure Twitter REST API calls always get extended tweets
Tweets that include embedds are truncated by default https://dev.twitter.com/overview/api/upcoming-changes-to-tweets
by passing tweet_mode=extened to the REST API calls we ensure to get the full response including embedded images and
videos
2016-12-30 13:22:37 +01:00
Dominik Sander
d3cbd14dd5 Simplify cronlines 2016-12-29 21:52:25 +01:00
Dominik Sander
1f4d54a112 Run every_X schedules on fixed times
Makes the schedules more predictable by running them on fixes hours/minutes instead of X seconds after startup.

Changed the "every 7d" schedule to "every monday" night to make it clearer when it runs because every 7 days is ambiguous
without a explicit start date.

Fixes #1103

Possibly related to #1840

Example runtimes:

```
every_1m (*/1 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:29:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:31:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:33:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:37:00 +0100

every_2m (*/2 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:38:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:42:00 +0100
2016-12-29 11:44:00 +0100
2016-12-29 11:46:00 +0100

every_5m (*/5 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:45:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 11:55:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:05:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:15:00 +0100

every_10m (*/10 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:20:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 12:40:00 +0100
2016-12-29 12:50:00 +0100
2016-12-29 13:00:00 +0100

every_30m (*/30 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 13:30:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 14:30:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 15:30:00 +0100
2016-12-29 16:00:00 +0100

every_1h (0 */1 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 17:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 19:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 21:00:00 +0100

every_2h (0 */2 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 22:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 02:00:00 +0100
2016-12-30 04:00:00 +0100
2016-12-30 06:00:00 +0100

every_5h (0 */5 * * *)
2016-12-29 15:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 05:00:00 +0100
2016-12-30 10:00:00 +0100
2016-12-30 15:00:00 +0100
2016-12-30 20:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 05:00:00 +0100
2016-12-31 10:00:00 +0100

every_12h (0 */12 * * *)
2016-12-29 12:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 12:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 12:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-01 12:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-02 12:00:00 +0100
2017-01-03 00:00:00 +0100

every_1d (0 0 */1 * *)
2016-12-30 00:00:00 +0100
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-04 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-06 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-08 00:00:00 +0100

every_2d (0 0 */2 * *)
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-11 00:00:00 +0100
2017-01-13 00:00:00 +0100
2017-01-15 00:00:00 +0100
2017-01-17 00:00:00 +0100

every_7d (0 0 * * 1)
2017-01-02 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-16 00:00:00 +0100
2017-01-23 00:00:00 +0100
2017-01-30 00:00:00 +0100
2017-02-06 00:00:00 +0100
2017-02-13 00:00:00 +0100
2017-02-20 00:00:00 +0100
2017-02-27 00:00:00 +0100
2017-03-06 00:00:00 +0100
```
2016-12-29 21:52:25 +01:00
Dominik Sander
71b1bf0a10 Merge pull request #1841 from dsander/upgrade-rails
Upgrade rails to 5.0.1
2016-12-29 20:52:57 +01:00
Dominik Sander
070946e879 Work around possible regression in rails
`pluck` used to always trigger a database query. With Rails 5.0.1 it uses data from
already loaded associations, this also includes new records for which we can not
generate agent_paths
2016-12-27 23:40:09 +01:00
Dominik Sander
0285a1a993 Update rails to 5.0.1 2016-12-27 23:40:09 +01:00
Dominik Sander
802714bbd8 Merge pull request #1763 from strugee/fix-bundler-git-protocol
Don't use the insecure git:// protocol when fetching git gems
2016-12-27 22:10:15 +01:00
Alex Jordan
f813a73dd0 Update Gemfile.lock with HTTPS URLs for git gems 2016-12-27 11:20:06 -08:00
Dominik Sander
db4de696b4 Override git_source method in the Gemfile
When the bundler version is below 2 override `git_source` to ensure
`github` remotes always use HTTPS
2016-12-23 08:27:50 -08:00
Andrew Cantino
65cea03062 Fix spec failures 2016-12-20 13:32:48 -05:00
Irfan Charania
4e2d1775a6 PhantomJs Cloud Agent (#1503)
* Initial draft of PhantomJsCloudAgent

Generates event with url for fetching html/plainText content

* Add options

* Pass in event instead of url
Fix hash syntax
Remove whitespace
Add mode merge

* Add some tests

* Style changes

- Add link to wiki entry for manually creating agent with full set of
options
2016-12-20 12:44:54 -05:00
Akinori MUSHA
2a524abff5 Fix the regex pattern to reduce backtracking
This should fix #1832.
2016-12-20 10:27:51 +09:00
Akinori MUSHA
cb2b906067 Merge branch 'travis_mysql' 2016-12-19 15:17:02 +09:00
Akinori MUSHA
3159b547c7 Use the builtin mysql on Travis CI 2016-12-19 14:51:32 +09:00
Akinori MUSHA
fd18e14538 Fast-forward faraday_middleware
The version constraint on faraday is relaxed while at it, although
some gems still have the strict constraint.
2016-12-13 19:31:32 +09:00
Akinori MUSHA
15347c8a3c Feedjira 2.1.0 is out, yay! 2016-12-13 18:08:16 +09:00
Akinori MUSHA
e26c07e75b Fix a merge conflict 2016-11-30 01:24:36 +09:00
Akinori MUSHA
1151a3ab6f Merge branch 'rss_agent_podcast_tags' 2016-11-30 00:07:57 +09:00
Akinori MUSHA
3612ba7333 Add podcast tags to events emitted by RssAgent
The keys are only added when the feed is a podcast.
2016-11-29 23:59:32 +09:00
Andrew Cantino
8a14a57e00 Beeper.io is no more (#1808)
* Beeper.io is no more

* Avoid event propagation or scheduling of missing agents

* Update undefined_agents.html.erb with a link to the wiki
2016-11-27 15:30:50 -05:00
Akinori MUSHA
8e36fdd60c Revert "Use faraday_middleware 0.10.1, which was recently released"
This reverts commit f4308afdf7.

I realized 0.10.1 did not include my fix for raw deflate.
2016-11-27 23:22:42 +09:00
Akinori MUSHA
08b43cc8b9 Update the RUBY VERSION with 2.3.3p222 2016-11-27 22:14:02 +09:00
Akinori MUSHA
f4308afdf7 Use faraday_middleware 0.10.1, which was recently released 2016-11-27 22:12:40 +09:00
Akinori MUSHA
14ccea6a4f Merge pull request #1771 from cantino/website_agent_resolves_url_optionally
Disable automatic URL normalization and absolutization on `url`
2016-11-27 14:38:18 +09:00
Akinori MUSHA
5f5f3cd38f Do not err if headers is a valid headers hash 2016-11-27 13:38:23 +09:00
Akinori MUSHA
9074f3115e Make sure status is an integer when set 2016-11-27 13:38:23 +09:00
Akinori MUSHA
a94cd7fd6d Allow an empty or null base URI 2016-11-27 13:38:23 +09:00
Akinori MUSHA
3a0c9e6274 Disable automatic URL normalization and absolutization on url
This was discussed in #1766.

For backward compatibility, existing WebsiteAgents with a key named
`url` will be given a `template` to resolve `url`.
2016-11-27 13:38:23 +09:00
Akinori MUSHA
3d91469733 Make WebsiteAgent merge template with the results of extract (#1816)
A new extraction option `hidden` is added so that keys with it gets
excluded from the final payloads while they can be used in `template`.
2016-11-27 13:33:24 +09:00
bobbysteel
f530305edc Add class of service chooser for Google Flights Agent (#1778)
* Add class of service chooser

* Add cabin chooser test

* Fix preferredCabin

* Per @cantino feedback taking out check
2016-11-26 15:13:00 -05:00
Akinori MUSHA
7ac691652b Spec that force_encoding works with encoding declaration in RssAgent 2016-11-26 13:26:09 +09:00
Akinori MUSHA
8d2ebe8fad Add a spec to test case-insensitivity with headers_to_save 2016-11-23 19:14:59 +09:00
Akinori MUSHA
29fd8aca28 Move the spec file for HttpStatusAgent where it belongs 2016-11-23 10:06:37 +09:00
Akinori MUSHA
c9e567edb3 Fix the specs for HttpStatusAgent
- Raplace hand-made mocking for web requests with Webmock

- Stop overriding internal methods of Agent like `interpolated`, because
  that made the specs not reflect actual behavior
2016-11-23 10:06:37 +09:00
Akinori MUSHA
6d4fdd3196 Use url and headers_to_save in a received event if included
That seems to be what was actually intended, judging from the specs.

See [my comment](https://github.com/cantino/huginn/pull/1521#discussion_r86488243) in #1521.
2016-11-23 10:06:29 +09:00
Akinori MUSHA
cc28f780c5 Make headers_to_save case-insensitive
See [my comment](https://github.com/cantino/huginn/pull/1521#discussion_r86489253) in #1521.
2016-11-23 10:06:29 +09:00
Akinori MUSHA
833985d723 Fix a bug where redirected is always true
This key was introduced in #1590 but it is likely that it has never
worked as expected.
2016-11-23 10:06:29 +09:00