“Artificial Intelligence”: het stellen van de diagnose van echte of valse kunst
WCN Congres, Beurs van Berlage, Amsterdam, 23 november 2012
Dr.ir. Jan C.A. van der Lubbe
Gabriele Bronzetti et al. (Inst. of Cardiology, Univ. Bologna): Van Gogh, Doctor Gachet, and Digitalis: A Self-diagnostic Portrait? Cardiovascular Drug Reviews, 2002, Vol. 20, No 3, pp. 233-236,
with respect to reasoning: cardiology = art history
Echt versus Vals?
Echt versus Vals?
ECHT
VALS?
Echt versus vals?
Portrait of Doctor Gachet
Echt versus vals?
ECHT
ECHT
Portrait of Doctor Gachet
Upper layer Hidden layers Underpainting
Altink Carriers: canvas and paper (Rembrandt) Visual characteristics (De Ploeg, Mondriaan, Van Gogh)
Hidden layers (Vermeer), Underpainting (Van Gogh)
Carrier
AUTHENTIC PROJECT DATERING EN AUTHENTICITEITS-BEPALING OP BASIS VAN VISUELE KENMERKEN PICTOLOGY: • Checklist voor authenticiteit • Van visuele kenmerken naar meetbare grootheden DATA MINING: • Zoeken naar relaties tussen kenmerken • Schilder’s karakteristieken Groninger Museum; Haags Gemeentemuseum; Van Gogh Museum; Rijksmuseum; Paleis ‘t Loo; Teylers Museum; Museum Kranenburgh
IMAGE
= {(R1,G1,B1), (R2,G2,B2)……} = {(12,245,114), (43,67,78)...} PAINTINGS ARE NUMBERS……!
Spatial Kernel Filtering
g ( x, y )
a
W1
W2
W3
W4
W5
W6
W7
W8
W7
b
w(s, t ) f ( x s, y t )
s at b
s, t : coordinates of filter with kernel size (2a 1) (2b 1) x, y : coordinates of image with size M N w(s, t) : weight factor at kernel cell (s, t)
Spatial Kernel Filtering
g ( x, y )
a
W1
W2
W3
W4
W5
W6
W7
W8
W7
b
w(s, t ) f ( x s, y t )
s at b
s, t : coordinates of filter with kernel size (2a 1) (2b 1) x, y : coordinates of image with size M N w(s, t) : weight factor at kernel cell (s, t)
Co-Occurrence Matrix Measures cx y k
c
ij
i j k
– Difference average
cx y k cij i j
k 0,1,2,
1 1 2 3 2 0
0 2 3 2 1 2
2 3 2 1 1 2
– Sum average
1 1 1 2 3 2
kc
k 0
x y
(k)
K 1
t7 kcxy (k) k 0
– Sum entropy
,2K 1
3 2 0 0 2 3
t6
2K 2
2K 2
ij
x y
t8 cx y k log cx y k k 0
2 1 2 2 2 1
– Difference entropy
K 1
t9 cxy k log cxy k k 0
– Correlation
K 1 K 1
t5
ij c i 0 j 0
x y
Learning the artist’s characteristics Problem Small number of authentic and false paintings; small training set Features Search for artistic meaningful features (brushstrokes): Gabor features, Cooccurrence matrices Important feature families Color features, Edge features, Texture features
Color
Edges
Texture
De Ploeg
+
+
+
Bergen
-
+
-
The Hague
-
-
+
Authentic Approach Feature extraction for indisputably authentic works • Colour • Texture
RF1
• Style
RF2
• Contrast
RF3
• Edges and Planes
…
• Composition
…
• ...
RFN 244-dim feature vector (N=244)
“Painter signature”
Database
Given RF(1,1),…..,RF(m,n), is RF(x,y) authentic?
EACH PAINTING IS A POINT IN A MULTIDIMENSIONAL SPACE
Feature Space
Case-based reasoning Comparing a patient with former patients/ cases Difference: In art history features are not known a priori
Feature Space
Classifier
KNOWLEDGE RULES Strong variation in the used warm colours ?
NO
YES Strong brush strokes?
NO
YES
YES
NO
NO
Transformed
V =(v1,v2,…,v131 ,v132,…,v244) Not strong warm colours variation (131) & wide range of brightness (13) & large constant colour patches (180) & average used of green palette high (128)
NO
NO
YES
DECISION TREES
NO
YES
YES
YES
NO
YES
NO
YES
(0.12, 0.80, 0.08) – probability to be part of a certain class
KnowledgeKnowledge-based reasoning based reasoning Deduction A -> B A --------B Abduction A -> B B --------A
“Appel – Geert Jan Jansen” Study
Learning Set
Texture
Texture
ATTRIBUTION AND DATING No signature: Has this etching been made by Rembrandt? If Rembrandt has made this etching: When he has made it? If he has made it in 1638: Has it been printed in 1638?
PAPER MANUFACTURING
(From: “De Papiermaaker”, P.J. Kasteleyn, 1792)
THE MOULD
• • • •
zeven gingen circa 5 jaar mee 1 riem: 20 boeken 1 boek: 24 vel (alles van dezelfde zeef) hypothese:
papier 1 = papier 2 print uit dezelfde tijd
Making X-Rays
THE MATCHING PROCESS: FINDING SHEETS FROM THE SAME MOULD B233
MADE AND PRINTED IN 1641 B020
PLATE MADE IN 1638, PRINT FROM 1641
TOESCHRIJVING Kupferstichkabinett Dresden: Portret van een oude man met een muts (275 x 205 mm) Portret van een jonge man (250 x 206 mm) Portret van een vrouw met hoed en sluier (270 x 203 mm Noord- of Zuid-Nederlands, krijt in zwart
Op basis van kettinglijnen: dezelfde anonieme kunstenaar; ca. 1500
HET IS ALLEMAAL WISKUNDE…….
HET REVIGO-PROJECT
WAT WE NU ZIEN, IS NIET WAT HET OOIT WAS!
BEKEND ALS “SNEEUWLANDSCHAP MET EG”, MAAR….
zo is het nu 2012
zo was het waarschijnlijk in 1888
HIJ MAAKTE VEEL TEKENINGEN IN BLAUW EN PAARS
DEZE WAS OOK EERST BLAUW-PAARS…….
EN SOMS GAAT DE ACHTERUITGANG WEL ERG SNEL………
DIT HADDEN WE
DIT HEBBEN WE NOG…
ECHT OF VALS?
THANKS FOR YOUR ATTENTION!
0172-617485
ANALYSE VAN TUSSEN- EN ONDERLAGEN Christie’s London 2004 (24 miljoen Euro) Recent toegeschreven aan Vermeer Twijfel over authenticiteit op basis van gele manteltje
Focused Ion Beam assisted Electron Microscopy
EnergyDispersive Spectrometry (EDS):
Verdeling van elementen Pb
Ca
KLEUREN TRANSFORMATIE faded colour
original colour
250
200
150
100
50
0
R
G
B
R
G
B
250
200
150
100
50
0
VIRTUELE KLEUREN-RECONSTRUCTIE
Vincent van Gogh, Patch of Grass, Paris, 1887, oil on canvas, 30 cm × 40 cm, Kröller-Möller Museum, Otterlo
HASY LAB (HAmburger SYnchrotronstrahlungslabor) Deutsches Elektronen-Synchrotron DESY, Hamburg
X-RAY Fluorescence Spectroscopy (XRF)
HASYLAB, Hamburg
Energy spectrum at targeted area
A: patch of grass, B: Mercury (Hg), C: Antimony (Sb), D: Lead (Pb), E: Zinc (Zn)
X-Radiography
XRF-image
Infrared Reflectography
5 slices: pixel values normalized to zero mean and unit variance; minimum value mapped to zero, maximum to one
Haralick & Shapiro: > 800 objects
Objects in surface painting
Black objects are related to hidden painting
INPAINTING
Example-based colourization
RGB Lab colour space
a-channel (mercury)
l—channel (antimony)
SECURITY AND PRIVACY OF IMD Dr.ir. Jan C.A. van der Lubbe
Delft University of Technology Faculty EEMC S Dept. Intelligent Systems Information Security and Privacy Lab (ISPL)
ISPL
