Cloud Computing & Recommendation Engines

From time-to-time Avolusis addresses internal projects that expand our experience beyond our traditional practice areas. We tackled the Netflix Prize, not to win, but to prove that using simple algorithms and cloud computing, a company could affordably match the results Netflix had achieved prior to the competition. Our goal: prove we could match Netflix’s staff of professionals, on our own, without a bench of Silicon Valley whiz-kids and top-tier PhD’s.
In our experience, particularly in finance and risk management, many companies don’t want to and don’t need to have cutting edge analysis. They just need to be slightly ahead of their competitors or at least in the middle-of-the-pack. More importantly, they need to do it cost-effectively and leveraging existing company personnel.

Avolusis built a 796-CPU computer cluster using Amazon EC2 and S3. We developed the tools to manage the cluster using a completely open source platform of Linux, R Statistical Package (an open-source equivalent to SAS, SPSS, or Matlab), MySQL, and MPI. And we did all this at an approximate cost savings of 90% from what could have been possible just a year or two earlier.
We ran one of the simplest algorithms in collaborative filtering – something someone with even the most casual understanding of statistics would comprehend.

And we essentially matched the results Netflix had started with. No PhD’s. No fancy algorithm’s to go awry. And something business people could manage, comprehend, and put their trust in.

Photo licensed under creative commons from kishjar? on flickr

 

Recent Posts
Contact Us

We're not around right now. But you can send us an email and we'll get back to you, asap.