Home > Blogs > VMware vFabric Blog


Case Study: CircuitLab Cuts Rendering time from 44 Hours to 57 Minutes Using RabbitMQ

Rendering-16_000-circuits-in-the-cloudCloud computing and open source models have helped the team over at CircuitLab.com perform a re-rendering job on 16,544 production circuit graphics in 57 minutes at a cost of a few CPU dollars and “just a bit of modification” to the existing codebase.  Does this sound unrealistic? Impossible?  Well, they did it. In an article they wrote last week, CircuitLab did a great job of articulating the size and scope of the job they set out to achieve and how RabbitMQ helped.

Solution Requirements

CircuitLab.com allows people to design and share the designs of electronic circuits.  When a user saves a circuit design, the system stores information and generates a render request message to generate various thumbnails, images, and more.  Each render request takes about 10+ seconds of 1 core CPU time as it travels through various components (see article for more).  To process all 16K+ circuit images, the total rendering time would add up to 160,000 seconds or 44 hours.  This was too long.

Additionally, the two founders (bright MIT grads) knew they could solve the problem faster using a parallel processing solution “in the cloud.”  They began working with M5 Cloud Hosting and provisioned 8 virtual machines on separate hypervisors with 8 cores each to get to 64 total high-speed CPU cores.  But, how would they distribute and manage the jobs across cores while production ran?

How the RabbitMQ Architecture Helped

Since their system was designed for scale with RabbitMQ as a part of the architecture, there was an inherent message passing architecture built in.  In the use case “user saves circuit,” the event places a message into a queue, and RabbitMQ distributes the message to various consumers.  Where appropriate, consumer processes start work.  This architecture made it easy to add a new, separate, cloud-based re-rendering function.  Of course, RabbitMQ provided the additional benefits of asynchronous processing, monitoring, retries, reporting, buffering, parallel computing, timeouts, error handling, and more.

Cloud Solution Architecture with RabbitMQ

Since the job needed to run on production data while the live site was still operating, a new (additional) set of queues were set up with RabbitMQ for “production reprocess” where the temporary cloud-based cores and cloud services would operate from.  These queues allowed the site’s rendering services to run “as is” while the batch re-rendering services occurred in the cloud.

Results

As reported, “with just a bit of modification to our existing code-base,” 16,544 messages were injected into RabbitMQ.  57 minutes later, a total of 61 “core hours” processed the entire job.  The cost ended up being a few dollars of CPU time to run!

For more information, read the entire article here.

We’d like to extend a special thanks to the team at CircuitLab for taking the time to publish this story!

Screen shot 2012-07-03 at 3.42.00 PM About the Author: Stacey Schneider has over 15 years of working with technology, with a focus on working with sales and marketing automation as well as internationalization. Schneider has held roles in services, engineering, products and was the former head of marketing and community for Hyperic before it was acquired by SpringSource and VMware. She is now working as a product marketing manager across the vFabric products at VMware, including supporting Hyperic. Prior to Hyperic, Schneider held various positions at CRM software pioneer Siebel Systems, including Group Director of Technology Product Marketing, a role for which her contributions awarded her a patent. Schneider received her BS in Economics with a focus in International Business from the Pennsylvania State University.
This entry was posted in Case Study, RabbitMQ and tagged , , , , , , on by .
Stacey Schneider

About Stacey Schneider

Stacey Schneider has over 15 years of working with technology, with a focus on working with sales and marketing automation as well as internationalization. Schneider has held roles in services, engineering, products and was the former head of marketing and community for Hyperic before it was acquired by SpringSource and VMware. She is now working as a product marketing manager across the vFabric products at VMware, including supporting Hyperic. Prior to Hyperic, Schneider held various positions at CRM software pioneer Siebel Systems, including Group Director of Technology Product Marketing, a role for which her contributions awarded her a patent. Schneider received her BS in Economics with a focus in International Business from the Pennsylvania State University.

14 thoughts on “Case Study: CircuitLab Cuts Rendering time from 44 Hours to 57 Minutes Using RabbitMQ

  1. google 2

    very nice blueprint.

    Reply
  2. سایپا دیزل

    great

    Reply
  3. آمیتیس

    very good

    Reply
  4. ابزار دقیق

    thanks your web saite

    Reply
  5. لیزر برش

    That’s Good article

    Reply
  6. تحصیل در بلاروس

    Thanks Stacey

    Reply
  7. نور مصنوعی

    Very Good .
    Thanks For this Article

    Reply
  8. لامپ رشد گیاه

    that’s great.
    thanks

    Reply
  9. کمپرسی

    very nice blueprint.

    Reply
  10. حفاظ پنجره

    حفاظ پنجره

    Reply
  11. درب آکاردئونی

    درب آکاردئونی

    Reply
  12. درب آکاردئونی آهنی

    درب آکاردئونی آهنی

    Reply
  13. حفاظ شاخ گوزنی

    حفاظ شاخ گوزنی

    Reply
  14. newsong

    دانلود آهنگ جدید-دانلود آهنگ جدید خارجی-دانلود آهنگ جدید شهرزاد-دانلود آهنگ جدید محسن ابراهیم زاده-دانلود آهنگ جدید ilyas yalcintas-دانلود آهنگ جدید تی ام بکس-دانلود آهنگ جدید شایع-دانلود آهنگ جدید آمریکایی 2017-دانلود آهنگ جدید سحر چه حالیه-دانلود آهنگ جدید نیکیتا-دانلود آهنگ جدید از محسن ابراهیم زاده-دانلود آهنگ جدید ایرانی-دانلود آهنگ جدید ابراهیم زاده-دانلود آهنگ جدید امین بانی-دانلود آهنگ جدید ایهام-دانلود آهنگ جدید اشوان-دانلود آهنگ جدید الیاس-دانلود آهنگ جدید امید جهان-دانلود آهنگ جدید اوزیر مهدی زاده 2017-دانلود آهنگ جدید انریکه 2018-دانلود آهنگ جدید آذری-دانلود آهنگ جدید آرمین تو ای اف ام-دانلود آهنگ جدید آوا بهرام-دانلود آهنگ جدید آرمین رقص اول-دانلود آهنگ جدید آوات بوکانی-دانلود آهنگ جدید آشوان-دانلود آهنگ جدید آرمین-دانلود آهنگ جدید آرتا-دانلود آهنگ جدید آرین یاری-دانلود آهنگ جدید باکو-دانلود آهنگ جدید بانی-دانلود آهنگ جدید بهرام-دانلود آهنگ جدید بهزاد پکس-دانلود آهنگ جدید باران-دانلود آهنگ جدید بلک کتس-دانلود آهنگ جدید بیرصدا-دانلود آهنگ جدید باران خیلی بی انصافی-دانلو
    http://new-song.ir/

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*