Industriële 3D visie gebaseerd op actieve stereo.
Phaer geeft u voorsprong in uw business met ‘visie’ en vision
Koenraad Van de Veere
[email protected]
Onze vision expert klanten & hun toepassingen
Marktleidende systeem- en machinebouwers passen in hun producten meestal een combinatie van vision technieken en 2D / 3D data-fusion toe.
Componenten van technologie-leidende fabrikanten
Technologie leidende fabrikanten van vision componenten die voorsprong in business geven door voorsprong in techniek en door de kennis die tot hun technologieleiderschap heeft geleid te delen
Toegevoegde waarde - expertise levert resultaat
Betere resultaten, tijdswinst en betrouwbaarheid door betere techniek, kennis, expertise, netwerk, transparantie, oriëntatie baken, lange termiin denken en korte termijn handelen, global supply & -support !
3D vision, sleutel tot nieuwe markten
2004-2006
2011
2009
2010
2012-2015
Phaer’s diensten en technologie aanbod, haar ervaring, wijze van aanpak en netwerk geven u voorsprong in business met 3D visie
3D vision, just do it !
2D vision, 3D vision wat is het verschil ? •
2D
één camera meet lichtintensiteit •
3D
één of meerdere “camera’s” meten afstand / hoogte o o o o
Twee camera’s Camera plus laser Stereometrie Time of flight
3D beeldvorming, welke is de beste methode Hangt af van de toepassing en het optisch gedrag van uw product ! • • •
•
•
Wat wil u als eindresultaat ? Hoe nauwkeurig dient de meting te zijn ? Welke zijn de mechanische en optische beperkingen van het product ? Welke zijn de beperkingen van uw setup / machine ? Welke zijn de beperkende omgevingsfactoren (licht, trilling, etc.) ?
Maximum waarborg = kennis & ervaring Don’t buy smart, be smart ! Bouw component gebaseerde vision systemen !
Actieve stereo vision - Snel, Gemakkelijk, Precies
3D using Active Stereo Vision The Ensenso Model Range SDK & API Conclusion
3D using Active Stereo Vision
3D Acquisition Techniques
Laser Time-of-Flight Stereo Vision Triangulation (passive) Moving Objects Multi-View Requirements on Objects
!
Fringe Projection
Kinect
Stereo Vision (+Projector)
needs controlled movement integrated multi-view stitching
Texture needed
3D using Active Stereo Vision
Stereo Vision Principle Object is observed from different angles If angle 𝛼,𝛽 and 𝑏 are known we can compute the position of 𝑝 Triangulation principle
“Base line” (camera distance)
3D using Active Stereo Vision
Stereo Vision Principle Objects appear shifted depending on their distance Shift in pixels (“Disparity”) can be converted to mm if…
Distance and mounting angle of cameras are known
Focal length and distortions are known
Disparity becomes mm after one time, easy calibration
3D using Active Stereo Vision
Stereo Vision - Without projector Image comparison (‘Stereo matching’) calculates the disparity for each pixel Problem: on uni-colored surfaces the disparity can’t be clearly determined
?? Fragmented depth image
3D using Active Stereo Vision
Stereo Vision - With projector Solution: Projection of additional texture Result: (almost) full depth image Issue of all triangulation methods: not all parts of the object can be seen in both cameras ‘Shadow’
Shadow Full depth image
3D using Active Stereo Vision - ENSENSO
Why Ensenso? Live metric 3D data out of the system Use multiple Ensenso‘s to enlarge volume of 3D measurement Use additional color camera to combine height and color in a single dataset for measurement Easy set up and integration of the system, pre-calibrated
3D using Active Stereo Vision - ENSENSO
Multi Ensenso setup - increase field, suppress shadows & occlusion Ensenso projects fixed random pattern and acquires stereo images → Allows overlapping of several patterns without interference
Camera 1
Camera 2
Camera 1 + 2
3D using Active Stereo Vision - ENSENSO
Ensenso + color camera - fuse 3D & color Color channels are pixel to pixel mapped to the 3D data Data reduction: Segment / recognise in 2D color, get 3D data for obtained ROI
Visualisation
+
= Color uEye camera 3D + pixel mapped color
Ensenso = 3D
3D using Active Stereo Vision The Ensenso Model Range SDK & API Conclusion
The Ensenso Model Range
For every purpose the right 3D solution!
Ensenso N10
Ensenso N20
Ensenso N30 & N35
The Ensenso Model Range
Hardware N20
N10
Sensors:
Sensors: CMOS 1.3MP sensor
Pattern projector: NIR 850nm or Blue at 465nm
CMOS 0.36MP sensor
Pattern projector: NIR 850nm
NIR use cases: Synchronized color images from uEyes Undisturbed human working environment Objects that require NIR for optimal contrast (fruit, vegetables, meat, plants etc.)
The Ensenso Model Range
Hardware - Ensenso N30 IP65/67 housing
Harting Push/Pull RJ45 connector 7 Pin GPIO connector Identical sensors, electronics, and object lenses as N20 → Identical models
Dimensions 52 x 50 x 175 mm: + 2 mm in Z compared to N20 Fixation threaded holes like N20 → same adapter plate
The Ensenso Model Range
Ensenso N35 IP65/67 housing, electronics, sensors, lenses like N30 FlexView projector added: Piezo-mechanical translation of projector pattern
Increased resolution in XYZ (factor of ~2 compared to N20/N30)
Useful for static scenes only
The Ensenso Model Range
FlexView performance & results Increased Z resolution by factor 2 Effective XY resolution improved by factor 2 More robust data on difficult surfaces Increased duration of data acquisition
Image acquisition duration ca. 45ms x number of image pairs
Increased duration of processing to 3D
3 image pairs can already give 80% of the max. result
The Ensenso Model Range
FlexView example Finer details More robust 3D data for slanted surfaces
Single-shot acquisition using N20/N30
More exact object contours Significantly less noise
4 image pairs
ONLY useful for static scenes! FlexView with 8 image pairs using N35
The Ensenso Model Range
View Field Size Focal Length
Vergence Angle
Working Distance
Configurations
3D using Active Stereo Vision The Ensenso Model Range SDK & API Conclusion
Ensenso SDK & API
Ensenso Software NxView
Preview live data: raw-images, depth-image, 3D-render-view
Find best settings: camera-, matching- and post processing-settings
Link multiple cameras
Calibrate your system
Ensenso SDK & API
Ensenso Software C++ / C# Interface Halcon Interface
Ensenso NxLib
NxView OS support for Windows and Linux
NxTreeEdit
Software interface to C, C++, C#/.NET, HALCON including examples IDS sensor experience, availability & support Full integration in HALCON 3D processing environment
Ensenso SDK & API
Hand Eye calibration
Ensenso SDK & API
Simple point cloud processing functions
Primitive fitting Find primitive 3D shapes Plane, Sphere, Cylinder
Found sphere Found cylinder
Found plane
3D Rendering Z data
Rendered 3D data
Conclusion on active stereo vision & Ensenso
Active stereo vision, eenvoudig méér • • • • • • • • • • • • • •
Snelle ‘one shot’ 3D meting in metrische data (mm) compacte industriële sensor zonder bewegende delen Gemakkelijk op te stellen & te kalibreren (incl. hand eye) Werkt bij bewegende objecten Gebaseerd op industriële componenten en software met multi generatie top reputatie Competitief geprijsd Flexibel werkvolume (van een paar cm2 tot 2.5m2 ) en flexibele werkafstand (10 cm - 280 cm) Synchrone multi view opname & digitale i/o Gemakkelijke, snelle ‘non expert’ calibratie (stereo pairs + color + hyper spectral +…) Hoge x-y resolutie up to 1.3 Megapixel Betrouwbare nauwkeurige Z data (mm) : 1-3mm op een WD van 2.8 m in a FOV van 1,7 x 1,4m Kleuronafhankelijk werken ; Eenvoudige volumebepaling Verschillende data kanalen en “sensor fusie” leidt tot hoge systeem robuustheid & precisie (3D, 2D zwart-wit, 2D kleur etc.) Krachtige, toegankelijke en zeer goed gedocumenteerde SDK
Actieve stereo visie, eenvoudig zeer betrouwbaar Stereo visie wordt toegepast door Daimler Mercedes om aanrijding met voetgangers te voorkomen. Ook goed genoeg voor uw 3D toepassing ? Film: © Daimler AG
Active stereo visie breekt hard door… • Stereo Vision has been around in research since decades! • New algorithms & accurate robust data • Moore’s law: Increased computing power boost at lower $$$ • Applied for pedestrian collision avoidance in cars by Daimler, Audi etc. • so why not for picking and scanning your product reliably ?
Applications from the recent past
Food picking, placing, sorting volume measurement
(de-) paletisation & staple inspection
Precise volume measurement
Fruit defect inspection Co bot applications (random picking, completeness inspection, movement tracking,…)
Color independent object segmentation and localization Bin Picking
+
=
Fruit ripeness inspection 3D fused with hyperspectral
Active stereo vision, how to turn it into new business ? • Get the tool • get an R&D kit special condition until 15/11
• Get the knowledge • get a case based training • Get the experience • build a demonstrator • Get the business, go fast to market • your demonstrator convinces your customer • seeing = believing
Active stereo vision, simply right LAAT DE GEREALISEERDE REFERENTIE CASES VOOR ZICHZELF SPREKEN. Don’t buy smart, be smart
[email protected] www.phaer.eu
Koenraad Van de Veere, November 2015
