Most businesses want to drive success in sales and marketing by understanding their customers better.
The challenge has been to deliver continually improving analytics into customer-level behaviour across different products and channels. Using relational databases many such projects have taken years, spent millions and produced minimal results.
Launching User Orientated Architecture (UOA) inspired me to share how UOA combined with Big Data provides a paradigm shift in getting to know your customers better. The objective is getting rapid availability of high quality and commercially relevant customer-level insights.
So what was the old world solution?
Time was that modelling customers used a single schema in a central database that could be queried by users on a read-only basis. It was like many fisherman all having to use the same rod to catch the same fish that they then couldn’t eat.
The first problem was that extract, transform and load (ETL) work mushroomed, with significant effort to get data from source systems into the required format and then moved to the centre. So think of that fisherman, trying to get many fish all swimming in different directions to shoal together then stay in one spot.
The second problem was that users valued parts of the central data view but still had their own unique needs for:
- Being able to access original source data directly
- Custom data manipulation and processing
- Differing focus levels on different parts of the data
- Different data interpretation
- Being able to “write” data in bespoke calculations
- Being able to store and access large analysis results
- Wanting to use non-standard software that met their needs better
In other words, users didn’t want the first (and usually the easiest fish) that the fisherman hooked.
A locked-down central database could not fulfil these needs, so many users created their own local datamarts holding what they really needed, kind of like fishing in their own small puddle. Fragmentation and duplication of effort resulted in the messy flows and ‘puddles’ of data below:
As you can see there is significant effort in just moving the data around, and reusing outputs is problematic as the data marts are not joined together so cannot communicate easily. This all left little capacity to do the value-add analytics that were the original reason for the central database – the prized fish just got away!
The central team would make increasingly frantic efforts to evolve the central schema, but were always destined to continually lag behind with business needs evolving daily. Just as more fishermen and rods do not mean that more fish will be caught, this problem would continue to spiral with more manpower used, more money spent and dissatisfaction growing daily.
The new world of UOA democratising access to data
Giving users freedom to get their job done is a core principle of UOA. This fits perfectly with Big Data providing a relatively open system in which data is written once and read many times.
Original source system data is placed as-is into a central “Data Lake” accessed by all. Combined with an “Analytics Reservoir” where users can use the processed output and analytical results of others, collaboration and re-use is facilitated.
Users define their own “schema on read” so that they can place an interpretation on data that is specific to their own needs. Users can also pick from a wide range of open-source tools and languages to best suit their needs and skills.
Doesn’t this picture intuitively look and feel better ?!
The ethos is similar to the Open Source world, with effort and progress determined by the choices of the user community as well as commercial priorities.
The central team role becomes very pro-active and focused on enabling action by:
- Curating source system data
- Being a central knowledge hub of all the outputs available
- Helping users solve their problems
- Facilitating communication and knowledge management
- Setting common standards e.g naming conventions and documentation
- Putting the right collaboration tools in place
- Productionising business critical data ingestion and analytics
- Investigating new tools and making future-proofed architectural choices
- Making the core systems run more efficiently and cost-effectively
The growing urgency for a better solution
Personalised products are proliferating and customers increasingly self-serve using internet and mobile channels. This growing diversity of choices and behaviour is generating larger and more complex data with an urgent need for immediate decision-making.
With UOA users have immediate access to larger and more relevant data sets, and as “more data beats better algorithms” the commercial results will shine.
Don’t mess around with little puddles of data – leap forward into Data Lakes and Analytics Reservoirs instead!
Recent Comments