GitHub-Mirror/huginn
2
0
Fork 0
mirror of https://github.com/Fishwaldo/huginn.git synced 2025-03-17 12:21:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Improve documentation of Website Agent (#2066)

Browse source 
Add sample JSON input and output events to clarify how JSONPath works.
This commit is contained in:
Géza Búza 2017-07-25 19:10:37 +02:00 • committed by Andrew Cantino
parent e9f025d063
commit 6a71295205
1 changed files with 46 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

47
app/models/agents/website_agent.rb
View file

@ -58,13 +58,58 @@ module Agents
# Scraping JSON
When parsing JSON, these sub-hashes specify [JSONPaths](http://goessner.net/articles/JsonPath/) to the values that you care about. For example:
When parsing JSON, these sub-hashes specify [JSONPaths](http://goessner.net/articles/JsonPath/) to the values that you care about.
Sample incoming event:
{ "results": {
"data": [
{
"title": "Lorem ipsum 1",
"description": "Aliquam pharetra leo ipsum."
"price": 8.95
},
{
"title": "Lorem ipsum 2",
"description": "Suspendisse a pulvinar lacus."
"price": 12.99
},
{
"title": "Lorem ipsum 3",
"description": "Praesent ac arcu tellus."
"price": 8.99
}
]
}
}
Sample rule:
"extract": {
"title": { "path": "results.data[*].title" },
"description": { "path": "results.data[*].description" }
}
In this example the `*` wildcard character makes the parser to iterate through all items of the `data` array. Three events will be created as a result.
Sample outgoing events:
[
{
"title": "Lorem ipsum 1",
"description": "Aliquam pharetra leo ipsum."
},
{
"title": "Lorem ipsum 2",
"description": "Suspendisse a pulvinar lacus."
},
{
"title": "Lorem ipsum 3",
"description": "Praesent ac arcu tellus."
}
]
The `extract` option can be skipped for the JSON type, causing the full JSON response to be returned.
# Scraping Text