Big Data: verbinding Click to edit Master title style wetenschap en markt
Dr. Anwar Osseyran Managing Director
Outline Ø SURFsara: ICT in service of innovation Ø What is Big Data Ø Big Data Drivers Ø Big Data Components Ø Big Data @SURFsara Ø Scientific Applications of Big Data Ø Big Data Is Big Business Ø Big Data is also Big Brother Ø Big Data: Our Value Proposition Ø To Finish with… Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
SURFsara: Onze Missie is HPC voor Innovatie q
De nationale missie van SURFsara is tweeledig: 1. 2.
q
HPC voor wetenschap en onderwijs HPC voor de kenniseconomie in Nederland
SURF Science Park
Realisatie door Ø
samenwerking tussen: § §
Ø
Universiteiten en kennisinstituten Bedrijven en overheden
Toegang tot hoogwaardige ICT systemen, -kennis en diensten
Science Park Amsterdam
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data: The Deluge
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
What is Big Data? “I cannot define it, but I know it when I see it”
My Big Bear now
My Big Bear then
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Inspired by Amsterdam…
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Defining Big Data with 3 V’s q Volume Ø Large amounts, 85% new data types Ø Massive historical archives Ø Valuable for data mining
Velocity
q Variety Ø Structured, semi-structured and unstructured Ø Variety also in Value
q Velocity
Volume
Ø At very high rates (sensors, streams, social media, …) Ø Valuable in its “fresh” state Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Variety
Big Data Drivers q “Internet of Things” q Commoditization of HPC q Human dynamics can be easily stored and queried with: Ø Distributed and non-relational Storage Ø Distributed (batch) Processing over many servers Ø Efficient and user friendly analytics
q Big data is creating a new industry and disrupting all others… Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data Eco-System @SURFsara Infrastructuur DevOps
Processing Programming algorithms
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Analytics Domain knowledge
Big Data Cluster @SURFsara
Kerberos authentication
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data and HPC (High Performance Computing) What benefits could Exascale HPC bring? It will enable discovery in many areas of science. "Aerospace engineering, astrophysics, biology, climate modeling and national security all have applications with extreme computing requirements,"
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data Requires also Sector Clouds
HPC and cloud computing simplify information access and sharing, and bridge the information gaps between science and medicine within the sector.
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
‘Collaboratorium’ Visualization and collaboration facility @SURFsara q Insight required: visualization of big data and trends q Also for improving business and Science models and computational debugging. q PowerPoint, Video Conferencing, telepresence, 3D (stereo) projection Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Also available @SURFsara
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Some Applications of Big Data
Astronomy Astron 1 Exabyte per day raw data 2 times WWW traffic per day SKA 300 - 1500 Petabytes storage per year! 20 times LHC
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Some Applications of Big Data
Healthcare Erasmus MC q Diagnostics: 4%: costs 72%: decisions q New classification of patients for better diagnostics & combined therapy
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Some Applications of Big Data
Water Management q Integration of: Sensor data + AI, Simulation results, Maps, weather, ships, roadwork, traffic, twitter, GSM, location of emergency services, ... Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Some Applications of Big Data
Infrawatch, Hollandse Brug
145 x sensors
100 Hz
x
x x x 60 60 24 seconds minutes hours
365 = big data days
(Arno Knobbe, LIACS, 2011, http://infrawatch.liacs.nl) Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Some Applications of Big Data
Ecology q Citizen Science: >20,000 users, >50M observations. q Bird radars: streaming data, many terabytes q GPS-tracking: Streaming data, world-wide. q Multi-scale information that can not be “seen” in the field.
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data and Sustainability q How to reduce power consumption? q Greening using Big Data: Ø Buildings, agriculture, materials, food production, transportation, water, waste, manufacturing, … Ø Data analytics for smart grids.
q Trust needed that the data is used without a Big Brother state. Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
BigBig Data: BigBig Brother? Data: Brother?
How should they responsibly handle it all, and how can they intelligently interpret what they find? Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data Is Big Business ü Financial Trade ü Fraud detection
United States needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more dataliterate managers, whether retrained or hired…
ü Web optimization ü Digital ads ü Online gaming ü Social network analysis ü µ-transactions ü Energy sector ü Smart agents ü Telco record mgt ü Mobile services ü M2M transactions ü Traffic management
SAS, EMC, Cloudera and others are offering product-line-specific training programs
ü Weather prediction ü Disease management ü Law enforcement ü National security ü … Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
9 Top sectors in NL 9 Top Opportunities for Big Data q Simulatie en modellering Ø Tata Steel Ø Vortech
q Visualisatie Velocity
Volume
Ø Metaphora Ø Royal Haskoning DHV
Variety
q Big Data Ø KPMG Ø Nationale Politie
q CBS q HPC Clouds “By 2015 organizations that use Big Data will outperform their peers financially by 20%” (Gartner G00212822, sep-2011)
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Ø Arcadis Ø Deltares
Verbinding kennis en markt q
q
Ons doel is het beschikbaar maken van de nieuwste technologie en expertise zodanig dat het praktisch inzetbaar is voor bedrijven. Verbinding agenda Ø Ø Ø Ø Ø Ø
Open Data e-Health Cloud Computing Digivaardig & Digiveilig Open standaarden Digitale agenda/ ICT doorbraakprojecten
Wat is er mogelijk? Ø Adviesgesprek / quickscan Ø HPC challenges (met bedrijven en studenten deze zomer) Ø Masterclasses en training Ø Toegang tot systemen, advies en services voor prototyping / pilots / showcases Ø Passepartout R&D toegang Ø HPC is in productie, services en ondersteuning Ø Sector clouds voor bedrijven en kennisinstellingen
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data Value Campaign q Goal: work with experts on exciting value cases for your company's Big Data challenges of today q Formula: you bring your data and formulate a business case with us (for rapid prototyping). q Launch end of April (duration 6 months). q Big Data Training: SURFsara provides expert training on BiG Data. Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data… Big Enough? Thank You
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Kansen voor samenwerking? q Welke behoefte signaleren de leden? q Op welke wijze zouden we de vraag verder kunnen ontdekken? Ø Organiseren van een themamiddag? Ø Distribueren van onze informatie?
q Bedrijven als launching customer, organisaties als partner? Ø Meedoen in met EZ doorbraakproject ‘MKB Innoveert’?
q Levert dit een bijdrage aan de binnen ECP platform gestelde doelen? q Tips vanuit de ECP leden? Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411
Big Data: verbinding wetenschap en markt q Aan de hand van nationale en internationale voorbeelden zal dr. Anwar Osseyran wetenschappelijke inzichten verbinden aan de economische en maatschappelijke uitdagingen op gebied van Big Data. Dr. Osseyran laat zien dat de combinatie van domeinexpertise en hoogwaardige ICT systemen de brug vormt naar nieuwe vormen van innovatie, toegepast onderzoek en samenwerking. Hij zal daarbij onder meer ingaan op de zorg als toepassingsdomein. Ook de ‘dark side’ van Big Data en hoe hiermee om te gaan, komt aan bod. dr. Anwar Osseyran, directeur SURFsara
Big Big Data Data in in de de praktijk praktijk –– VNO-NCW VNO-NCW 130411 130411 Big Data in de praktijk – VNO-NCW 130411