Jsdom Scraping On Cloudfoundry Doesn't Work

I'm using the jsdom node.js module for an app that does some web scraping to get some data it needs. This works perfectly fine when I run locally. When I push the application to c

Solution 1:

I was able to reproduce this problem pretty easily. It seems jsdom relies on a native compiled library called Contextify (https://github.com/brianmcd/contextify) and also has some bindings to Python from what I can gather. These are not supported by Cloud Foundry at present. There is a similar question on Github about this problem;

https://github.com/tmpvar/jsdom/issues/436

The last post at the time of writing this suggests the use of two 100% JS libraries called Domino and Zepto Node, they may well be worth checking out.

Solution 2:

With only what you posted to go by its pretty much impossible to give you a direct answer.

But I have a couple of suggestions.

1. You have environmental variables that don't work the same in production as they do in development.

2. jsdom relies on jquery that is normally loaded-in externally. Perhaps jquery is not loading properly.

Have you heard of cheerio. Its an alternative to jsdom that implements a subset of jquery and does not need to load jquery in externally. This makes much faster than jsdom.

You can watch this video created by the author for introduction to cheerio and learn more of its benefits.

Use the sample code on the cheerio github page and upload it your cloudfoundry account to see if everything works. Then you can make decision on weather to switch to cheerio.