Home > Blogs > VMware vFabric Blog > Tag Archives: GemFire

Tag Archives: GemFire

Advanced GemFire + Lucene + Spring Data for Text Searching

In this article, we’ll talk about how to integrate the Lucene text searching solution using Spring Data and GemFire to provide a flexible, parallel fast search engine. By combining the two independent products we can leverage each product to its fullest capability. The end result provides an elastic search capability with the in memory data speeds of a distributed cache platform and high availability.

Motivation—Why Combine These?

The motivation of the project was to provide an alternative search capability for GemFire while providing users a natural method to define searchable domain object attributes. Performance was also a key driver to ensure constant search performance irrespective of scale. The solution outlined below provides a baseline approach for developers to build upon. Continue reading

Q&A with Shay Banon: 10 “Bonsai Cool” Things About elasticsearch

We are very fortunate to post an interview with Shay Banon, the founder of elasticsearch. Elasticsearch is technology that is very popular among some of the coolest companies on the web today, including  SoundCloud, StumbleUpon, Mozilla and Klout. These companies use elasticsearch to help them deploy powerful search capabilities in their applications that are easy to set up, scalable and built for the cloud.  In this interview, we get to learn all kinds of cool things:

  1. How Shay got into search
  2. How he came up with the idea for elasticsearch
  3. Why elastic search is different than other OSS search projects
  4. Example elasticsearch users like Foursquare, Brewster, GitHub, Sony, and Klout
  5. About the elasticsearch architecture for big data
  6. The strategy behind JSON over HTTP for search
  7. Connecting elasticsearch with RabbitMQ
  8. Connecting elasticsearch with Spring
  9. Connecting elasticsearch with GemFire
  10. Running elasticsearch on virtualized infrastructure

Without further ado, here is the interview.

Q1. So, how did you end up getting into search?
About 10 years ago, I moved from Israel to London because my wife was going to study to be a chef at the Cordon Bleu. I had no job. I was in a new country. I was unemployed. So, I started to get into the latest, cool, new technologies. Continue reading

Part 2: The Value, Architecture, & Code for Building Geography-Based Apps

In our last post, we 1) covered how geographic data can release value in mobile and machine-based applications, 2) explained how technology is used to overcome barriers to these types of big data scenarios, and 3) detailed the architecture for a data fabric or grid (like vFabric GemFire) that works with geographic data and specialized or alternative indexes. There were also code examples to explain the object model, the spatial index, and data changes.

Now, we will continue the examples, show you how to make the index highly available, and use a function to access the data via the index.

The Scenario for a Highly Available Index

In some cases, a piece of data may be added to a node, or become primary on a node without a clean method call. This happens in the cases of both failover and rebalancing. In the case of failover, a bucket that is on a node (that was also a redundant copy) may suddenly become the primary copy if the node that held the primary failed.

In the case of rebalancing, an entire bucket can be moved to a new node that was added to the system without the benefit of capturing the “put” call on each piece of data. Continue reading

Part 1: The Value, Architecture, & Code for Building Geography-Based Apps

Will machine-generated data be larger than mobile and tablet-generated data?

No matter where you might place your chips on that bet, they both rely on geographic data for quite a number of business applications. These geographic data applications stand to release a tremendous amount of business value, and, in this two-part series, we will explain:

  1. How geographic data can release business value through applications.
  2. Where technology overcomes big data barriers to release the business value
  3. The concepts behind vFabric GemFire’s data fabric as well as an object model and data architecture for software services connecting to geographic data fabrics
  4. How to use an open source quadtree index and the related Java code for interacting with geographic data in vFabric GemFire

How Geographic Data Releases Business Value

Many early versions of geography, location, or proximity-based applications can be found in the market. Recently, we published a few examples of these types of applications in articles about ocean sensor data and mobile applications, but there are more: Continue reading

6 Simple Steps: Installing vFabric Administration Server (VAS) and Provisioning tc Server across 3 Nodes

As a suite of products, vFabric offers a single SKU model that includes the entire suite. Besides offering an economically beneficial licensing model that is flexible, based on average use, and hardware independent, the vFabric team has developed the vFabric Administration Server (VAS) to help make installation, configuration, and operations easier.

If you missed a previous article explaining how VAS work and what it can do for you, VAS is based on a single system image design (SSI) and allows you to define a group of nodes where each operation on the group is applied to each node who is a member of the group. This approach provides time savings and reduces errors by keeping systems more consistent, allowing easier ways to scale, and simplifying administration. VAS also provides a robust set of REST APIs for virtually all of its functionality. In addition to vFabric tc Server, VAS can manage vFabric RabbitMQ, vFabric GemFire, vFabric SQLFire, and vFabric Web Server. VAS 1.1.1 was recently released with new features and many bug fixes.

In this article (and demonstration further below), we will show you six steps that give you an idea of how easy it is to provision using VAS. We will show you how to install VAS and use it to provision vFabric tc Server across three nodes along with a WAR file. The explanation below refers to examples from RubyGems.org and GitHub/vFabric/VAS-Ruby-API along with the latest VAS documentation.

Continue reading

Announcing the Availability of vFabric Data Director 2.5, GemFire 7, EM4J 1.2, and More

Application developers and data management teams continue to look for ways to modernize legacy apps, manage costs more effectively, build new apps on robust application platforms, and solve big data problems. These are some of the key reasons why vFabric is on the CIO (or CTO) agenda. With several new product releases in the vFabric Suite, VMware continues to provide a best-in-class application platform and help customers solve their top application development and data management problems.

vFabric Data Director 2.5

Database as a Service (DBaaS) helps companies virtualize data engines and automate management while getting a handle on the costs and compliance issues related to data sprawl. In the newest version of Data Director, several new data engines are supported (in addition to Oracle and Postgres) along with other new capabilities:

  • Support for Microsoft SQL Server 2008 R2 and SQL Server 2012
  • Support for Hadoop deployment, management, and monitoring across all major distributions through Project Serengeti
  • Enhanced automation of Oracle and SQL Server template creation
  • Broad support for Red Hat Enterprise Linux (RHEL) and Oracle Linux
  • Enhanced Oracle database ingestion, including ingestion to a point-in-time and more
  • Support for static IP database virtual machines (DBVMs)
  • Express set-up for development or experimentation

Continue reading

3 Steps on Using Spring Insight Developer to Analyze Code

If you don’t know about Spring Insight Developer, this post may save you tons of time and potentially headache.

Imagine that you need to update some code behind a button, but you didn’t write the code. What if you could press the to-be-coded button and then see what code was invoked (including methods and arguments), the SQL invoked, and the time it took to execute?

This is what Spring Insight Developer allows you to do, and more.

It’s also free, and it uses AspectJ and AOP to load-time weave your application, you do not have to make any changes to your application code to use it.

Let’s take a look at a simple example of tracing your app, viewing the details, and seeing the code in action.

Continue reading

How We Improved APM’s Monitoring Capacity by 500% with Gemfire, RabbitMQ, vPostgres, and a Few Scalability Design Patterns

Earlier this week, we announced the general availability of a major upgrade to vFabric Application Performance Manager (APM). This release started one year ago, after we released the first version of the product to market. When we started work on this release, we knew we would need to invest heavily in scalability. APM is designed to help simplify monitoring and management for highly dynamic, large web applications living in the cloud. To succeed, we needed to make sure our product could scale gracefully with our customers. So, we set out with a challenging goal to increase the capacity of APM by a factor of 5.

Transforming a complex product such as APM into a more scalable architecture is not an easy task, let alone doing so in a single release. For this reason we’ve started by modifying the architecture in steps, starting with local improvements inside our virtual appliance, (available in the APM 5.0 release) and moving towards a horizontal scale solution in future releases. Continue reading

Upcoming Webinar: Going In-Memory with VMware vFabric GemFire 7.0 and Spring

Social media, mobile devices, and an ever-expanding commercial world wide web have transformed our culture to an always-on, always-connected, and increasingly impatient creatures. As a result, how applications deal with data management is probably the area of computing that has undergone the most change in recent years. It’s undergoing a renaissance, if you will.

Traditional relational database management systems (RDBMs) are being replaced by new, in-memory data systems that deliver high performance and can scale-out horizontally quickly to meet the needs of the next Facebook, Netflix, or Pinterest application that shines in the market.

In an upcoming webinar, VMware will show you how you can use vFabric GemFire  to transform your data management strategy to achieve the speed, scale and reliability that internet pioneers have been trusting us to do for years.  We’ll also cover some of the recent advancements, including how GemFire 7 makes managing and monitoring your GemFire data much easier, and how Spring Data GemFire project makes writing new modern applications easier than ever before.

Join VMware on Thursday, December 6, 2012 at 10:00AM PDT/1:00PM EDT for the webinar titled, The Data Renaissance; Going in-memory with VMware vFabric GemFire 7.0 and Spring.

>> Register for Webinar here

Map-Reduce Style: Data Aware Distributed Querying in vFabric GemFire

vFabric GemFire ArchitectureBig, fast data is powering some of the most interesting computing opportunities in today’s market. But in order to get there, we need to change our approach to the data tier. Enterprises are trying to move from costly mainframe architectures to virtualized datacenters and utilize commodity hardware more efficiently. With the data tier, this means an architecture that scales horizontally by adding more commodity-based computing and storage at runtime.

To scale the data tier horizontally, companies use systems like vFabric GemFire, a distributed data system that is designed to specifically accommodate large data sets across commodity hardware nodes. In GemFire, data is spread across members of a cluster with members referred to as “nodes,” and the distribution of data across those nodes is called “partitioning.” vFabric GemFire then allows developers to query the data that resides across many nodes while retaining core values of very high performance at scale. How? In short, the answer is “Data Aware Querying” – a query API that allows a query to execute on selective nodes instead of all nodes (i.e. execute in a map-reduce style).

Continue reading