Boy it was fun! It we delivered some amazing capability.
Here are 5 things I learnt to help get success with Hadoop (plus a bonus 6th).
1. Don’t faff around. Get the data, land it in Hadoop. This isn’t Kimball school of data management and it’s not super expensive kit. Get the data, stick it in Hadoop, keep it forever. If it’s not there it won’t get used. Your analytical models and algorithms might just need it.
2. Take a leaf out of the agile book. Open up access to new data sources to your subject matter experts and then work together to test, learn and iterate on structuring the data so it’s useful for analysis and business outcomes. You don’t need to model the whole lot up front. You can go further and purely do schema on read but depends on your target audience.
3. If you build it they will come. It’s not field of dreams. Don’t do this.
Do however get ahead of the problem and build for yet unstated requirements. Alongside this though you must build credibility, communicate what’s possible and bring the business and analytics parts of the business along with you so the platform gets utilised.
4. Recruit a unicorn. Well actually we know that’s not going to happen so better still create a well rounded team with all the skills necessary. You don’t need and you won’t find a mythical god that can do everything needed. Mix of skills. Mix of experience. Commercial. Technical. Consultative folk. A team to be proud of and you would trust to engage with all stakeholders and make you proud.
5. Prove value. Prove you can do something different to the BI projects that you’ve tried to deliver before. Demonstrate capability. Do lunch and learns. Do show and tells. Become an internal marketer. Its possible with Hadoop to get value from data far quicker than was previously possible so find a valuable use case, work with the owner and create that beachhead example of whats possible.
Secret 6th trick is make sure you have decent data management, discovery and analytics tools to integrate with Hadoop to create a window on to your data. The easier it is to work with, the more people will come begging for more.
That’s it. Ok it’s not magic, no card tricks involved. It’s about people, culture, ways of working and some tech thrown in to boot.