Skip to content
Snippets Groups Projects
Commit 32ac1508 authored by Alex Ellis's avatar Alex Ellis
Browse files

Add sample scraping/automation example via Phantomjs


Signed-off-by: default avatarAlex Ellis <alexellis2@gmail.com>
parent 12c12b1a
No related branches found
No related tags found
No related merge requests found
FROM alexellis2/phantomjs-docker:latest
ADD https://github.com/alexellis/faas/releases/download/0.5.6-alpha/fwatchdog /usr/bin
RUN chmod +x /usr/bin/fwatchdog
ENV fprocess="phantomjs /dev/stdin"
HEALTHCHECK --interval=5s CMD [ -e /tmp/.lock ] || exit 1
CMD ["fwatchdog"]
### Phantomjs function
[Phantomjs](http://phantomjs.org) is a headless web-browser used for scraping and automation testing. This function will scrape a web-page with the JavaScript sent in through the function.
Example usage:
```
$ time curl --data-binary @cnn.js http://localhost:8080/function/phantomjs
Status: success
CNN - Breaking News, Latest News and Videos
real 0m8.729s
```
This script visits the front page of CNN and once it is fully loaded - scrapes the title.
See [cnn.js](https://github.com/alexellis/faas/tree/master/sample-functions/Phantomjs/cnn.js) as an example of a Phantomjs script.
Another example script [feedly_subscribers.js](https://github.com/alexellis/faas/tree/master/sample-functions/Phantomjs/feedly_subscribers.js) gives the count of subscribers for an RSS Feed registered on Feedly.
var webPage = require('webpage');
var page = webPage.create();
page.open('https://www.cnn.com/', function(status) {
console.log('Status: ' + status);
console.log(page.title);
// Do other things here...
phantom.exit();
});
var webPage = require('webpage');
var page = webPage.create();
var feed = "http://blog.alexellis.io/rss/";
var url = "https://feedly.com/i/subscription/feed/" + feed;
page.open(url, function(status) {
var title = page.evaluate(function(s) {
return document.querySelector(s).innerText;
}, ".count-followers");
console.log(title.split(" ")[0]);
phantom.exit();
});
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment