the BPM freak !!

Home » BigData » The new era of Data!!

The new era of Data!!

Follow me on Twitter


Everyday we have something new happening in the data world to start with we had the Database concepts – RDBMS, then followed by  Data Ware housing, Business Intelligence, Business Analytics, BigData Hadoop and many more.

With all this happening all around it becomes quite confusing to define an implementation approach for a use case or a requirement, i.e where can we draw a thin line for using the different products and concepts.

For meeting my enterprise needs, shall I use RDBMS or BI or the new concepts which others are following like Hadoop or Big Data or can’t i use Business Analytics!! .The question list is endless and just enough to baffle a user!!

(Image: jscreationzs /

As Herbert Simon(a Nobel Laureate) has very rightly stated “a wealth of information creates a poverty of attention“.

Its so very true and relates seamlessly with the Data related concepts and Issues we are facing these days!!

Couple of decades back things were so much under control and the IT and the business used to get excited with simple data storage, data retrieval and reporting features.

But, in the current scenario things have changed exponentially not only from the data front even the concepts and tools that drives these.

Terabyte and Peta bytes of data have become very common for the companies. It becomes really interesting if we can make the best use of this huge amount of data as a value add rather than just getting rid of it or purging it!!

So, what is the main consequence of this data flooding or information overload : “PERFORMANCE!!”, “RESPONSE-TIME” to name a few.

The answer to this humongous amount of data and performance degradation and most importantly getting rich info from these data  is to use the Map-Reduce algorithm and the concepts of hadoop and big data!!

Some questions that can be tossed are :

  • “Is Hadoop or Big Data the only answer!!” or
  • “Shall I use it just because many other companies are using it ?” or
  • “When can I use this and expect to reap the benefits out of it”  or
  • “Google, Yahoo and other product companies have used it!!…and its been a great story till date!!…What makes the Service Companies pitch in!!”

In a NUTSHELL, all the above questions point to one simple query “What are the use cases where we can fit in hadoop/bigdata ?

Sometimes, even I faced the same situation, puzzled by such questions each time I started talking about hadoop or bigdata!!… I always had the doubt about mapping a use case or its a general approach….though I always had a soft corner and liked the new concepts in the data arena!!

What can Hadoop Offer ?

Apache Hadoop is an ideal platform for consolidating large-scale data from a variety of new and legacy sources. It complements existing data management solutions with new analyses and processing tools. It delivers immediate value to companies in a variety of vertical markets.” – Quote from Cloudera

A brief highlight if the Data related use cases!! (as mentioned below – the list can be huge!!) :

In Retail/CPG

  • Merchandizing and market basket analysis.
  • Campaign management and customer loyalty programs.
  • Supply-chain management and analytics.
  • Event- and behavior-based targeting.
  • Market and consumer segmentation.

In Telecommunications

  • Revenue assurance and price optimization.
  • Customer churn prevention.
  • Campaign management and customer loyalty.
  • Call Detail Record (CDR) analysis.
  • Network performance and optimization.

In Health & Life Sciences

  • Campaign and sales program optimization.
  • Brand management.
  • Patient care quality and program analysis.
  • Supply-chain management.
  • Drug discovery and development analysis.

In Government Bodies

  • Fraud detection and cyber security.
  • Compliance and regulatory analysis.
  • Energy consumption and carbon footprint management.

In Banking & Financial Services

  • Compliance and regulatory reporting.
  • Risk analysis and management.
  • Fraud detection and security analytics.
  • CRM and customer loyalty programs.
  • Credit scoring and analysis.
  • Trade surveillance.

In E-tailing

  • Recommendation engines — increase average order size by recommending complementary products based on predictive analysis for cross-selling.
  • Cross-channel analytics — sales attribution, average order value, lifetime value (e.g., how many in-store purchases resulted from a particular recommendation, advertisement or promotion).
  • Event analytics — what series of steps (golden path) led to a desired outcome (e.g., purchase, registration).

(Source : Cloudera Site)

What makes Services companies pitching in!!

From my personal opinion, when it comes to Services Companies!!….the use of these concepts don’t remain vague,  ambiguous and conceptual, instead it spreads three-fold times!!

Service companies believe in a multi-domain presence, not specifically concentrating on a product, feature or functionality as a generic piece.

With a blend of multi-domain exposure when the concepts like hadoop/bigdata are churned, the actual flavor and the real-time utilization of the concepts along with the value adds can be experienced.

So, whats your take on this ?

The Data Trend @ Job Levels!!

(Image: worradmu /

It was quite surprising and at the same time interesting to hear that firms have taken Data to the Next Level.

Recently, many companies are planning to have a special role/designation identified as “Data Engineers” , “Senior Data Engineers” for their employees. (Correct me if I have learnt it wrong).

On a lighter note, it really tough for some people who have spent some good time in IT to digest the role of a “Data Engineers” which sounds more like a Data Entry operator!! 🙂

Some people were even finicky that they won’t get a bride, if people start interpreting a data engineer as a data entry operator!! 🙂

Happy Learning!! 🙂



  1. Soumyo says:

    Is the reference to Data Engineers aimed at me 🙂
    Data engineers will rule the next… World 5.0 and are the new people.. the 3.0 Man..

    better all Pros start to look at data as the real thing for the next decade.. 🙂

    • @Soumyo No personal pun intended!! 😉
      Very true….the way strides are being made in the data front …no wonder!!….it will be the next big thing!!
      There has been phenomenal changes in the data arena…and it really feels good to see it from a conceptual basis.
      Though I never had a hardcore data experience….and very much eager to get groomed with it…can definitely understand your sentiments!! 🙂
      Kudos to the Data Engineers!! 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: