Computer Vision Penginderaan Visual untuk berbagai keperluan Dr. Mohammad Iqbal @ 2016 Disampaikan pada seminar nasional “Perkembangan Computer Vision dan Multimedia" yang dilaksanakan oleh Himpunan Mahasiswa Teknik Informatika Universitas Nasional pada hari Rabu, 20 Januari 2016, di Aula Universitas Nasional Blok I lantai 4
Gunadarma University
S3, S2, S1 and Proffessional Program
Faculties
1.
Computer Science and Information Technology
2.
Industrial Technology
3.
Economic
4.
Civil Engineering and Plan
5.
Psikology
6.
Literature
Research Organizations
Research Organization University and for every Faculty
Special Science Group Discussion
Lecturer Group Research: Foshema & Scimed
Penelitian Computer Vision di Gunadarma
Pusat Studi : Mikroelektronika dan Pengolahan citra – imaging system dan smart sensor Robotika dan Multimedia Sistem Multimedia dan Robotik – Implementasi robotic vision dan data set collection Informatika Kedokteran – Implementasi vision di bidang kedokteran dan kesehatan Interaksi Manusia dan Teknologi – Evaluasi Interaksi mesin dengan manusia
3
Menu Seminar kita hari ini… Apakah Computer Vision?
Penglihatan (Vision) itu Tidak Sederhana Computer Vision Anatomy Penggunaan Vision Hari Ini Kesimpulan Diskusi
Mengapa perlu belajar tentang Computer vision?
Jutaan citra di capture setiap waktu
Ada jutaan aplikasi yang bisa dibuat berdasarkan CV
Menu Seminar kita hari ini… Apakah Computer Vision? • • • •
•
Defenisi Komputer Grafik ? (transformasi 3D->2D) Defenisi Komputer grafik ? (Modeling vs. Rendering) Jadi Defenisi Komputer vision (2D->3D) Defenisi Computer Vision : • Irisan antara Computer Vision dan Computer Graphics • Menurut para ahli • Permodelan berbasiskan Citra (Image-Based Modeling) Disiplin ilmu yang terkait • Kecerdasan Buatan • Dasar Matematika yang dibutuhkan • Kaitan ilmu modern terkini untuk Computer Vision • Lingkup Kurikulum Computer Vision di Universitas
Apakah Computer Vision? •
Kebalikan dari Komputer Grafik Computer vision
World model
World model
Computer graphics
•
Pemahaman komputer terhadap Citra (Image Understanding) secara AI, atau menganalisis perilaku (behavior) / pola Citra
•
Sensor untuk robotika
•
Emulasi Komputer dari penglihatan manusia
Defenisi Komputer Grafik ? (transformasi 3D->2D)
3D geometri
proyeksi
Simulasi Sifat fisik
Grafik
Defenisi Komputer grafik ? (Modeling vs. Rendering)
Modeling
Create model Apply material ke model Tempatkan model di scene Tempatkan light di scene Tempatkan camera
Rendering Ambil “citra” dengan camera
Dua-duanya dapat selesai dengan commercial software: Autodesk MayaTM ,3D Studio MaxTM, BlenderTM, etc.
Point Light Spot Light Directional Light
Ambient Light Penggabungan pencahayaan oleh Patrick Doran (2009)
ILMU LANJUT : Grafik Komputer 9
Jadi Defenisi Komputer vision (2D->3D)
3D Geometri
Estimasi Sifat fisik
Defenisi Computer Vision : Irisan antara Computer Vision dan Computer Graphics rendering surface design animation user-interfaces Computer Graphics
modeling - shape - light - motion - optics - images IP
shape estimation motion estimation recognition 2D modeling
Computer Vision
Defenisi Computer Vision
Trucco and Verri: computing properties of the 3D world from one or more digital images Sockman and Shapiro: To make useful decisions about real physical objects and scenes based on sensed images Ballard and Brown: The construction of explicit, meaningful description of physical objects from images Forsyth and Ponce: Extracting descriptions of the world from pictures or sequences of pictures
[Trucco&Verri’98]
Defenisi Computer Vision : Permodelan berbasiskan Citra (Image-Based Modeling) image processing graphics
Images (2D)
Geometry (3D) shape
+
Photometry appearance
vision 3 Image processing
2.1 Geometric image formation
4 Feature extraction
5 Camera calibration
7 Image alignment
6 Structure from motion
2.2 Photometric image formation
8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 14 Image-based rendering
12 Photometric recovery
Disiplin Ilmu yang Terkait : Kecerdasan Buatan
Kaitan ilmu modern terkini untuk Computer Vision
Lingkup Kurikulum Computer Vision Image Processing Computer Vision
Computer Graphics
Pattern Recognition
Machine Learning
Computational Perception
Multi-Robot Systems Multi-view Geometry
Intelligent Robotics
Autonomous Robotics
Menu Seminar kita hari ini… Penglihatan (Vision) itu Tidak Sederhana
•
Karakteristik Human Vision • Ilusi Adelson Checkerboard • Warna yang konstan (Color Constancy) • Ukuran yang Konstan (Size Constancy) • Ilusi Thatcher
•
Area Fokus Komputer Grafik dan Vision – Hardware & Interaction
•
Timeline Teknologi Computer Vision
Penglihatan (Vision) itu Tidak Sederhana Mata Manusia Vs Kamera
Penglihatan itu Tidak Sederhana
Penglihatan (vision) prestasi terbesar dari kecerdasan alami (natural intelligence ) manusia Visual cortex menempati sekitar 50% dari bagian otak Macaque Seakan2 otak manusia dikhususkan utk menangani urusan vision Itu raja atau perdana menteri ya ?
Karakteristik Human Vision
Penglihatan adalah proses kontruktif
Persepsi kesadaran dari yang kita lihat adalah ILUSI yang dibuat oleh otak kita (dengan proses yang luar biasa rumit). Contoh : kecerahan (brightness), warna (color), dan ukuran yang konstan (size constancy)
22
Ilusi Adelson Checkerboard Persepsi brightness adalah fungsi rumit dari nilai piksel
(Image courtesy of Ted Adelson)
Brightness constancy problem 23
Warna yang konstan (Color Constancy) Warna Piksel sangat dipengaruhi oleh iluminasi Persepsi dari konstannya suatu warna dikelola oleh otak kita
Sunlight
Fluorescent light (Images courtesy of David Heeger)
Ukuran yang Konstan (Size Constancy) Ukuran obyek VS kedalaman obyek
(Images copyright John H. Kranz, 1999)
Karakteristik Human Vision
Penglihatan akan menyelesaikan tugas tertentu saja dalam konteks yang juga spesifik
Umumnya kemampuan visual itu terikat langsung dengan kebutuhan dan konteks seseorang (kebiasaan hidup, emosional, dll). Contoh : Thatcher illusion
Ilusi Thatcher
(Due to P. Thompson)
Ilusi Thatcher
Face processing sensitif pada orientasi citranya
Area Fokus Komputer Grafik dan Vision – Hardware & Interaction Screenless / Hologram technology Wearable Teknologi
Teknologi Display
HIGH RESOLUTION HIGH BRIGHTNESS LARGE VIEWING ANGLE HIGH WRITING SPEEDS LARGE COLOUR GAMUT HIGH CONTRAST LESS WEIGHT AND SIZE LOW POWER CONSUMPTION LOW COST
Stereoscopic
Teknologi Surface / Touch screen
Area Fokus Komputer Grafik dan Vision – Hardware & Interaction
Perangkat Input
Mouse, tablet & stylus, multi-touch, force feedback, dan game controller lainnya (seperti Wii), scanner, digital camera (images, computer vision), dsb. Semua bagian tubuh menjadi devais interaksi:
http://www.xbox.com/kinect
30
Area Fokus Komputer Grafik dan Vision – Hardware & Interaction Multi form Output
Cell Phones/PDAs (smartphones), laptop/desktops/tablets, Microsoft PPI display 3D immersive virtual reality systems such as Brown’s new Cave being built at 180 George Street Brown’s old Cave & new Cave
Apple iPhone™ Samsung Galaxy SIII (Android)
Microsoft Surface
Microsoft PPI display
ILMU LANJUT : Interaksi Manusia Komputer
Timeline Teknologi Computer Vision
# Computer Vision History graph from the book of Richard Szeliski
Menu Seminar kita hari ini… Computer Vision Anatomy
• • • • •
Langkah2 dalam Pengolahan Citra Digital Sistem Pencahayaan (Lighting system) Staging Lensa dan Kamera Aplikasi Perangkat Lunak Vision
Computer Vision Anatomy
Pada dasarnya sistem Computer atau Machine Vision dibuat untuk membantu menggantikan keahlian manusia pada bagian visual
34
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Level Pengolahan citra
Level 0: Representasi citra (akuisisi, sampling, kuantisasi, kompresi) Level 1: transformasi Image-to-image (enhancement, restoration, segmentation) Level 2: Transformasi Image-to-parameter (feature selection) Level 3: transformasi Parameter-to-decision (recognition and interpretation)
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Kedudukan DIP, ComVis
Image Processing:
Levels 0 and 1
Image Analysis:
Levels 1 and 2
Computer/Robot Vision:
Levels 2 and 3
Computer Graphics/Animation ?
Pendekatan dalam “creating images” atau membuat “visual effects” dari deksripsi yang diberikan pada level sebelumnya.
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Problem Domain Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Image Aquisition Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Image Enhancement Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Image Restoration Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Morphological Processing Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Segmentation Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Object Recognition Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Representation & Description Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Image Compression Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy : Langkah2 dalam Pengolahan Citra Digital - Colour Image Processing Image Restoration
Morphologic al Processing
Image Enhancement
Segmentation
Image Acquisition
Object Recognition
Problem Domain
Representation & Description
Colour Image Processing
Image Compression
Computer Vision Anatomy
1. 2. 3. 4.
Lighting Staging Lenses Cameras
Computer Vision Anatomy : Lighting
Computer Vision Anatomy : Staging
Parameter-parameter penting dalam sistem pencitraan (imaging system).
49
Computer Vision Anatomy : Kamera dan Lensa
Kamera dan Lensa :
Jenis Sensor : CCD Vs CMOS (complimentary metal-oxide semiconductor) Ukuran Sensor :
CCD/CMOS Size. (Image copyright of Edmund Optics).
Cara Pembacaan : area scanning and line scanning.
50
Computer Vision Anatomy : Kamera dan Lensa
Sistem Lensa :
Relationship between focal length, object and camera plane. (Image copyright of Edmund Industrial Optics).
Wide area lens (catadioptric, fisheye) Vs Basic Lens (zoom, macro, telesentric) Sistem Filter Lensa : Polarization, IR, UV, …
Computer Vision Anatomy : Kamera dan Lensa
Resolution : Resolusi citra B lebih baik dari A. (Image copyright of Edmund Industrial Optic).
Focus :
52
Computer Vision Anatomy : Kamera dan Lensa – Model dan Geometri Kamera Pinhole camera
or Geometric transformations in 2D and 3D
53
Computer Vision Anatomy : Kamera dan Lensa – Camera Calibration
Know 2D/3D correspondences, compute projection matrix
also radial distortion (non-linear)
Aplikasi Perangkat Lunak Vision
HALCON dari MVTEC http://www.mvtec.com/halcon/ HALCON is the comprehensive standard software with an integrated development environment (IDE) for machine vision that is used worldwide. It leads to cost savings and improved time to market: HALCON's flexible architecture facilitates rapid development of machine vision, medical imaging, and image analysis applications. HALCON provides outstanding performance and a comprehensive support of multi-core platforms, MMX, and SSE2. It serves all industries by a library of more than 1400 operators for blob analysis, morphology, pattern matching, measuring, identification, and 3D vision, to name just a few.
Aplikasi Perangkat Lunak Vision COGNEX (http://www.cognex.com/Main.aspx)
Vision Systems : All-in-one systems that combine camera, processor and vision software into a single rugged package, with a simple and flexible user interface for configuring your application. Vision Software : Vision software gives you the most flexibility for combining the full library of powerful Cognex vision tools with the cameras, frame grabbers and peripherals of your choice, and enables easy integration with PC-based data and control programs. Vision Sensors : Easy, affordable sensors that can be used in place of photoelectric sensors for more reliable inspection, error-proofing and part detection. Industrial ID : Fast, reliable 1D and 2D code reading and verification for direct part mark or high-contrast applications. Industry-Specific Products: A result of over 25 years of vision experience solving the most difficult vision applications, these products include wafer identification, surface mount device placement guidance, cylindrical product inspection and more. Web and Surface Inspection : Industry-leading technology for detecting and classifying defects during the continuous production of metals, paper, nonwovens, plastics and glass.
Menu Seminar kita hari ini…
Penggunaan Vision Hari Ini
Penggunaan vision Hari Ini
Contoh state-of-the-art Animal
Vehicle
head
Four-legged Mammal
wheel
Move on road Facing right
leg
Can run, jump Is herbivorous Facing right
Penggunaan vision Hari Ini Industrial Vision
Damar Darbito, 2013 - Inspeksi Produksi Kartu Seluler
Penggunaan vision Hari Ini Industrial Vision
Deteksi kecacatan pada mulut botol
Deteksi kecacatan dalam botol
Deteksi kecacatan pinggir botol
Benyamin, 2013 - Inspeksi Produksi Botol Susu plastik
Penggunaan vision Hari Ini Recovery 3D layout dan context
BED
61
Penggunaan vision Hari Ini Editing images as if they were 3D scenes
Penggunaan vision Hari Ini Earth viewers (3D modeling)
Image from Microsoft’s Virtual Earth (see also: Google Earth)
Penggunaan vision Hari Ini 3D from thousands of images
3D from one image
Building Rome in a Day: Agarwal et al. 2009
65
Hoiem Efros Hebert SIGGRAPH 2005
Penggunaan vision Hari Ini Optical character recognition (OCR) Technology to convert scanned docs to text •
If you have a scanner, it probably came with OCR software
Digit recognition, AT&T labs http://www.research.att.com/~yann/ 66
License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Penggunaan vision Hari Ini Face detection
Many new digital cameras now detect faces Canon, Sony, Nikon …
Penggunaan vision Hari Ini Smile detection?
Sony Cyber-shot® T70 Digital Still Camera
Penggunaan vision Hari Ini
Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk, you are assured to get paid for it… “
69
Penggunaan vision Hari Ini Vision-based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns” Read the story wikipedia
70
Penggunaan vision Hari Ini Login without a password…
Fingerprint scanners on many new laptops, other devices 71
Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/
Penggunaan vision Hari Ini Object recognition (in mobile phones)
Point & Find, Nokia Google Goggles
Penggunaan vision Hari Ini Special effects: shape capture
The Matrix movies, ESC Entertainment, XYZRGB, NRC
Penggunaan vision Hari Ini Special effects: motion capture
Pirates of the Carribean, Industrial Light and Magic
Penggunaan vision Hari Ini Special effects: motion capture
Based-on Ega Hegarini 2015 - Motion Analysis for sport science
Penggunaan vision Hari Ini Sports
Sportvision first down line Nice explanation on www.howstuffworks.com http://www.sportvision.com/video.html 76
Penggunaan vision Hari Ini Smart cars
Mobileye
Vision systems currently in high-end BMW, GM, Volvo models By 2010: 70% of car manufacturers. Slide content courtesy of Amnon Shashua
Penggunaan vision Hari Ini Smart Vision Drone
Penggunaan vision Hari Ini Google cars
http://www.nytimes.com/2010/10/10/science/10google.html?ref=artificialintelligence
Penggunaan vision Hari Ini Interactive Games: Kinect
Object Recognition: http://www.youtube.com/watch?feature=iv&v=fQ59dXOo63o
Mario: http://www.youtube.com/watch?v=8CTJL5lUjHg
3D: http://www.youtube.com/watch?v=7QrnwoO1-8A
Robot: http://www.youtube.com/watch?v=w8BmgtMKFbY
Penggunaan vision Hari Ini Vision in space
NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007.
Vision systems (JPL) used for several tasks • • • •
Panorama stitching 3D terrain modeling Obstacle detection, position tracking For more, read “Computer Vision on Mars” by Matthies et al.
Penggunaan vision Hari Ini Industrial robots
Vision-guided robots position nut runners on wheels
Penggunaan vision Hari Ini Mobile robots
http://www.robocup.org/
NASA’s Mars Spirit Rover http://en.wikipedia.org/wiki/Spirit_rover
Saxena et al. 2008 STAIR at Stanford
Penggunaan Vision Hari Ini
Medical imaging
3D imaging MRI, CT
Image guided surgery Grimson et al., MIT
Penggunaan vision Hari Ini Entertainment : Video Mapping 1.
2. 3.
Uses projection to place videographics on a physical object. Creates an optical illusion using light. Transforms ordinary objects into magical living entities.
86
www.artisuniversalis.com/educational
Kesimpulan Hari ini sudah sama-sama kita bicarakan : Definisi Dasar Ilmu yang harus dikuasai Tantangannya Anatominya Implementasi Computer Vision dalam kehidupan Selanjutnya ? Terserah anda… (mau jadi player? Atau mau jadi penonton saja?)
merci
Thank You
Terima Kasih
Question?
δ
Itot
