TAKE A SWIM IN THE DATA LAKE who invented the technology that is normally used in the data lake space . In Australia , legislation dictates where the network data needs to be retained because law enforcement agencies want to be able to access the information if they need to . In fact , a law is currently being passed where Telco and internet service providers have to retain all of that data . Because so much of it is online and they have so much of it , financial services must do it as well , according to Gardner . He ’ s also beginning to see it a lot more data lakes in government work as well as retail .
The future of big data In the future , Gardner believes data lakes will be the best way to retain historical data . It ’ s also prevalent in real-time analytics , such as the information a bank teller has at their disposal when you give them your account number , which has been in place for a long time .
“ As we get more and more capable of managing more and more realtime analytics , the data lake will become the posterior for landing data that has been streamed through ,” Gardner said . “ The data won ’ t necessarily go into the lake first and then out to the systems , but rather , it will go through the process and then back into the lake .
“ Imagine it raining at the top of a mountain , and the water filters down into the lake , as opposed to you pumping the lake full of water at the source .”
But if analysts aren ’ t careful , they can also create what is known as a data swamp . When one person needs to access the data and copies it , then someone else comes along to access the same data for a different reason and copies it again , it creates several little “ ponds ” of data . Therefore , data lakes need an element of human touch .
“( Data lake ) will become the key use from a technology point of view ,” Gardner said . “ Unfortunately , not many people have taken a structured approach of how they started the journey . The risk is if you don ’ t try to keep it simple and not have the loose integration that you start with , then you will end up with people creating all these little ‘ ponds .’ It needs an element of being manmade , thus a data reservoir is probably the best analogy .”
23