Big Data – surely there is more to it than ad’s?

Big data is definitely a hot topic with opinion, discussion and polarisation amongst across the community. For me the most fundamental question I am considering is who can benefit most.

I am not sure if I am being unduly pessimistic or whether I have limited exposure but the only use I see being championed is in advertising, search engine optimisation and understanding the demographics of your customer. Important? Well, yes, but surely the advances in technology, creation of new platforms and emergence of ‘data science’ takes us beyond this!

Perhaps the limiting factor has simply been the economy and limited cash within the business and government. Maybe all of the companies IT resources have been focused on eke’ing out new customers or maximising the profit from existing one; I am not sure but I believe that the time is right for companies to look at increasing their use of big data at all levels; improving skills, exploring the new technologies and looking to invest in other areas where analytics and exploration can add competitive advantage.

There are clearly valid uses in government and security that might be considered unwelcome by some. Others could have a massive reach and bring benefits to a huge audience; the options for healthcare research, materials analysis, R & D, demographics, manufacturing are huge but largely unexploited?

Enterprise Service Bus

I have heard many very smart people talk about the benefits of the enterprise service bus (ESB). I like the idea and am familiar with brokers, subscribers, queues etc.

ESB’s work for developers and for the business systems they build. But the most recent context is to use the ESB as a configuration management subsystem to build complex cloud infrastructure.
It just wont work for the following reasons:

  1. ESB’s are the domain of developers; sys admins don’t understand messages, brokers and queues – you might as well be talking another language
  2. The complexity of the ESB makes orchestrating a solution very difficult
  3. ESB’s are not real-time, declaritive.

I wait with baited breath to see whether VCE has yet delivered its’ solution uisng SonicMQ.

Now, DevOps – that is an approach that might just work but it needs a more educated IT management specialist.


Joining it up

Lots of buzz about DevOps and a number of notable proponents.

I am a believer myself but will it make it to the enterprise and at what cost?

I certainly hope that some of the most admirable aspects do make it to the enterprise. I am particularly interested in:

  1. the speed with which devops can deliver by embracing automation and the eability to remove operational noise
  2. the take-up of pattern-based computing
  3. self-documented systems

And by the same token there does need to be an amount of control an compliance as new services are introduced. A free-for-all where any technology can be delivered into the enterprise is clearly a non-starter.

I will pick on 10gen / mongoDB since I have studied it a little and it has been massively [over]-hyped recently.

How can platforms like MongoDB make it into the enterprise successfully in the next 12-18 months?

  • first; what is the real business need
  • is there any tooling to help manage the environment
  • are there already tools that can do the same but don’t have the same appeal

Again, more pondering to do.

Big Data.

Without doubt one of the hottest topics in the industry at the moment but what is Big Data?

Is it storage, analytics or is it the ability to access and query structured and unstructured data and other content?

With so many vendors jumping on the band wagon it can be hard to be specific.

Before becoming embroilled in the discussion it is important to take a step back and work out what problem you are trying to solve.

Is the problem one of storage, content or data management, data analysis or data mining?

What is it you are missing in solving your problem? Once you have worked that out it may be very obvious which direction you should focus your attention.