GitHub-Mirror/huginn
2
0
Fork 0
mirror of https://github.com/Fishwaldo/huginn.git synced 2025-03-17 04:11:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
2813 commits 15 branches 0 tags 8.2 MiB
Commit graph

143 commits

Author SHA1 Message Date
Gergely Brautigam
073ae10332
Removing obsolete test and environment property. 2017-05-29 20:48:50 +02:00
Dominik Sander
e1f702adf1 Upgrade liquid (#1982) 
* Update Liquid to 4.0

* Handle Liquid renaming before_method to liquid_method_missing

* Adopt `regex_replace` tags to new Liquid BlockBody class
 2017-04-26 22:09:21 +02:00
Dominik Sander
d3cbd14dd5 Simplify cronlines 2016-12-29 21:52:25 +01:00
Dominik Sander
1f4d54a112 Run every_X schedules on fixed times 
Makes the schedules more predictable by running them on fixes hours/minutes instead of X seconds after startup.

Changed the "every 7d" schedule to "every monday" night to make it clearer when it runs because every 7 days is ambiguous
without a explicit start date.

Fixes #1103

Possibly related to #1840

Example runtimes:

```
every_1m (*/1 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:29:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:31:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:33:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:37:00 +0100

every_2m (*/2 * * * *)
2016-12-29 11:28:00 +0100
2016-12-29 11:30:00 +0100
2016-12-29 11:32:00 +0100
2016-12-29 11:34:00 +0100
2016-12-29 11:36:00 +0100
2016-12-29 11:38:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:42:00 +0100
2016-12-29 11:44:00 +0100
2016-12-29 11:46:00 +0100

every_5m (*/5 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:35:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:45:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 11:55:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:05:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:15:00 +0100

every_10m (*/10 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 11:40:00 +0100
2016-12-29 11:50:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:10:00 +0100
2016-12-29 12:20:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 12:40:00 +0100
2016-12-29 12:50:00 +0100
2016-12-29 13:00:00 +0100

every_30m (*/30 * * * *)
2016-12-29 11:30:00 +0100
2016-12-29 12:00:00 +0100
2016-12-29 12:30:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 13:30:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 14:30:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 15:30:00 +0100
2016-12-29 16:00:00 +0100

every_1h (0 */1 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 13:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 15:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 17:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 19:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 21:00:00 +0100

every_2h (0 */2 * * *)
2016-12-29 12:00:00 +0100
2016-12-29 14:00:00 +0100
2016-12-29 16:00:00 +0100
2016-12-29 18:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-29 22:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 02:00:00 +0100
2016-12-30 04:00:00 +0100
2016-12-30 06:00:00 +0100

every_5h (0 */5 * * *)
2016-12-29 15:00:00 +0100
2016-12-29 20:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 05:00:00 +0100
2016-12-30 10:00:00 +0100
2016-12-30 15:00:00 +0100
2016-12-30 20:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 05:00:00 +0100
2016-12-31 10:00:00 +0100

every_12h (0 */12 * * *)
2016-12-29 12:00:00 +0100
2016-12-30 00:00:00 +0100
2016-12-30 12:00:00 +0100
2016-12-31 00:00:00 +0100
2016-12-31 12:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-01 12:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-02 12:00:00 +0100
2017-01-03 00:00:00 +0100

every_1d (0 0 */1 * *)
2016-12-30 00:00:00 +0100
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-02 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-04 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-06 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-08 00:00:00 +0100

every_2d (0 0 */2 * *)
2016-12-31 00:00:00 +0100
2017-01-01 00:00:00 +0100
2017-01-03 00:00:00 +0100
2017-01-05 00:00:00 +0100
2017-01-07 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-11 00:00:00 +0100
2017-01-13 00:00:00 +0100
2017-01-15 00:00:00 +0100
2017-01-17 00:00:00 +0100

every_7d (0 0 * * 1)
2017-01-02 00:00:00 +0100
2017-01-09 00:00:00 +0100
2017-01-16 00:00:00 +0100
2017-01-23 00:00:00 +0100
2017-01-30 00:00:00 +0100
2017-02-06 00:00:00 +0100
2017-02-13 00:00:00 +0100
2017-02-20 00:00:00 +0100
2017-02-27 00:00:00 +0100
2017-03-06 00:00:00 +0100
```
 2016-12-29 21:52:25 +01:00
Akinori MUSHA
3612ba7333 Add podcast tags to events emitted by RssAgent 
The keys are only added when the feed is a podcast.
 2016-11-29 23:59:32 +09:00
Akinori MUSHA
58fabb885c Add a new Liquid filter rebase_hrefs 2016-10-29 20:40:52 +09:00
Akinori MUSHA
4d10132709 Fix a bug where an empty <link> is wrongly parsed 
Due to a problem in sax-machine's internals, an empty `<link/>` in RSS
would be parsed to JSON as `{ "href": "no_buffer" }`.  Now empty
`<link/>` elements in RSS and ATOM are simply ignored just like other
collection elements like `<category>`.
 2016-10-27 09:17:46 +09:00
Akinori MUSHA
5f5e246552 Use Struct#each_pair 2016-10-27 08:13:43 +09:00
Akinori MUSHA
e5c938aa85 Exclude empty entries from authors 2016-10-27 08:12:48 +09:00
Akinori MUSHA
852f39d480 Rescue error from Mail::Address#name and #address 
`Mail::Address.new('')` does not raise any error but calling `name` on
the created instance does.
 2016-10-27 08:11:10 +09:00
Akinori MUSHA
ba0d2ee0de Retire ar_mysql_column_charset 
Rails 5 has native support for charset/collate.
 2016-10-11 22:25:54 +09:00
Akinori MUSHA
df6fb7e5fc Fix super which is not defined yet here 2016-10-07 19:32:02 +09:00
Akinori MUSHA
d63f714ae9 RssAgent: Migrate from FeedNormalizer to Feedjira 
FeedNormalizer is no longer maintained, and its Atom support has flaws
in that it throws away what RSS::Parser returns and falls back to using
SimpleRSS which is not capable of handling XML entities, resulting in
getting ususable URLs such as ones including `&amp;`.

Feedjira is highly customizable as it implements parsers for various
feed formats on its own using sax-machine.

The `clean` option is reimplemented using the loofah gem.
`Feedjira::FeedUtilities#sanitize_entries!` is not used because it
tries to sanitize non-HTML properties too.

A new option `include_feed_info` is added, with which turned on feed
information is added to each event payload.

A new key `links` is added, which lists all `link` elements.

A new key `enclosure` is added.
 2016-10-06 11:49:34 +09:00
Akinori MUSHA
7e79d576b5 Improve Utils.normalize_uri (#1719) 
* Improve Utils.normalize_uri

Globally replacing generally unsafe characters in a URL would not fix
invalid authorities and paths, so use Addressable::URI to normalize them
when necessary.

This should fix #1701.

* Remove an unused function

* Fix the test case to make sure an IPv6 address is supported
 2016-10-05 14:17:04 +09:00
Karol Bucek
8ca39a5802 minor tweaks (#1669) 
* avoid unnecesary  method pollution

* production.rake typo fix

* remove un-used local variable assignment
 2016-09-06 13:36:10 -04:00
Dominik Sander
811f9c27d1 Allow specifing Agent gem in the Gemfile 2016-06-20 10:37:33 +02:00
Judy Ngai
1abcd947f5 Uploadicons (#1427) 
* checking in work for glyph upload

* checking in icon selections

* added more icon options

* work on upload icon

* fix issue

* switched to options for select

* removed wrapper and removed commment

* add migration to support icon

* remove right floating

* rm params session add it to model

* rebuilt the menu

* rm extra div

* converted js to coffee

* quick change in indentation

* using the view helper

* submitting icons and loading into form

* added icon to both scenario export and import

* added icon to spec

* low level caching the icons

* tentative work in making a new coffee class

* renamed js page

* switched to cache in memory and worked on coffee class

* made pathsmatching compatible for all scenarios

* switched to rails root

* fixed all rubocop stylings

* fixed line is too long styling

* rm private method definition

* patch to avoid having no icon

* blank or nil?

* reordering the lines
 2016-06-15 23:52:50 -07:00
Pablo Carranza
74a3c88608 Add time tracking to http status agent (#1517) 
* Add time tracking to http status agent

* Replace time tracking to monotonic

* Add tests for time tracker
 2016-05-29 19:13:40 -04:00
Dominik Sander
a6a06c1cd1 Separate feature and non feature spec runs on travis 
Add database_cleaner gem
Set up coveralls parralel builds in travisci
Require capybara_helper in rails_helper to make clear it affects all tests
 2016-05-12 10:03:23 +02:00
Dominik Sander
73bcd927e3 Do not run/enqueue event propagation when a AgentPropagateJob is already enqueued 
Fixes the creation of duplicated events caused by Agent.receive! running multiple times concurrently.
 2016-04-17 17:15:50 +02:00
Brian Petro
b941f740e5 replace delayed_job queue methods with ActiveJob counterparts 2016-04-12 09:49:49 -04:00
Dominik Sander
38f1044078 Add LocalFileAgent and introduce concept to handle files 
The idea is to have source agents (`LocalFile` and `S3Agent` for now), which just emit a "file_pointer" and consuming
agents (`ReadFile` and `CsvAgent`) which get access to an `IO` object to actually access the file contents. This way we
do not have to pass megabytes of data through the database and the consuming agents can work with relatively unlimited
file sizes.
 2016-03-18 20:43:28 +01:00
Dominik Sander
ee1ebea830 Allow to configure various devise settings in .env 2016-03-17 20:47:45 +01:00
Andrew Cantino
67fa70553d Allow a key with credential instead of a key_file 2016-02-01 20:29:40 -08:00
Akinori MUSHA
96b756ca11 Default ENV['RAILS_ENV'] to 'development' 2016-01-05 16:23:45 +09:00
Dominik Sander
daf45f7f0e Do not try to load .env file on heroku 2015-12-10 20:54:11 +01:00
eric
d233946feb Export Agents in GUID order so that the result is amenable to version control tracking 2015-12-07 10:30:14 -07:00
Dominik Sander
9c16b0fb51 Allow usage of PostgreSQL without exporting ON_HEROKU 
Vendorizing the dotenv gem allows access to the `.env` configuration
in the `Gemfile`

Manual installation: Configure before installing gems
 2015-12-04 21:31:45 +01:00
Andrew Cantino
9bdbdfa10d Merge remote-tracking branch 'origin/master' into prevent_extra_threads_in_specs 2015-11-20 08:56:30 -08:00
Andrew Cantino
60d880e8be remove p statements 2015-11-20 08:19:22 -08:00
Andrew Cantino
ff05700458 Prevent buildup of threads in specs 2015-11-19 21:39:03 -08:00
Akinori MUSHA
bde6a7269d Merge pull request #1125 from cantino/website_agent_normalize_uri 
Introduce Utils.normalize_uri and use it in WebsiteAgent
 2015-11-15 01:53:18 +09:00
Akinori MUSHA
cdfdc7f3c5 Introduce Utils.normalize_uri and use it in WebsiteAgent 
This fixes #938, and the specs are from #958. (Thanks @irfancharania!)
 2015-11-15 01:41:19 +09:00
Dominik Sander
28983a7378 Merge pull request #1142 from dsander/fix-twitter-agent-restart-leak 
Do check restarting LongRunnable::Worker for liveliness
 2015-11-12 17:26:08 +01:00
Dominik Sander
e303239281 Do check restarting LongRunnable::Worker for liveliness 
Fixes a memory leak caused by starting a new Worker when the old one is still restarting.
 2015-11-12 00:07:30 +01:00
Akinori MUSHA
43a6a025a4 Remove SchedulerAgent#check! which name is confusing 2015-11-06 17:11:09 +09:00
oroce
4487a9270d force-stopping services 2015-10-21 07:30:25 +02:00
Andrew Cantino
797e7e3b3c When payloads get truncated in the DB due to encoding issues, at least do not blow up on reading them 2015-09-30 20:28:30 -07:00
Dominik Sander
06b022f161 Merge pull request #819 from dsander/agent-runner 
Add AgentRunner and LongRunnable to support long running agents
 2015-09-09 16:36:56 +02:00
Dominik Sander
e0f14698ba Ensure database connection are checked back into the pool by wrapping calls in with_connection blocks 2015-09-07 15:52:54 +02:00
Dominik Sander
44ac1ed909 Restart TwitterStreamAgent::Worker every hour to prevent stalling 2015-09-07 15:52:54 +02:00
Dominik Sander
08028aa9ea Revert TwitterStreamAgent back to the twitter-stream gem 2015-09-07 15:52:48 +02:00
Dominik Sander
991777ca29 Background worker load Dotenv if APP_SECRET_TOKEN is not set 
Unified background worker initialization into pre_runner_boot

Set Rails.configuration.cache_classes = true for background workers
 2015-08-23 16:58:44 +02:00
Dominik Sander
0f64e4e6b9 Add AgentRunner and LongRunnable to support long running agents 
This changes HuginnScheduler and TwitterStream to use the new
AgentRunner

TwitterStream now uses the `twitter` gem to access the streaming API
which removes the `eventmachine` dependency (in production).

Make JabberAgent FormConfigurable and LongRunnable

Expect LongRunnable::Worker#run to be overridden
 2015-08-23 16:58:44 +02:00
Ian Blenke
7b952fa8ac Merged master to re-converge after merge of PR #983 2015-08-21 21:57:12 -04:00
Dominik Sander
1361c5693b Moved deployment configuration files and changed the repository 
Update ruby to 2.2.3
 2015-08-19 22:12:50 +02:00
Dominik Sander
cec02c1bd6 Use runit instead of upstart as init system 2015-08-19 00:13:45 +02:00
Dominik Sander
0e4c9be723 First version of the manual deployment guide 2015-08-19 00:13:44 +02:00
Ian Blenke
16b4c6cad1 Fixed conflict in docker/scripts/init with bind to 0.0.0.0 2015-08-16 20:58:56 -04:00
Akinori MUSHA
51e8414257 Retire lib/prepend.rb 
We do not support Ruby 1.9 any more.
 2015-08-01 16:55:04 +09:00