Automatisch Testen
Netherlands Germany Switzerland Serbia
Customer Business Lunch 6 november 2014
enterprise integration • where complex business challenges meet agile solutions
enterprise integration • where complex business challenges meet agile solutions
3
Vraag?
Doen wij al aan automatisch testen?
enterprise integration • where complex business challenges meet agile solutions
4
Agenda Automatisch testen Waarom? Mogelijkheden Demo Conclusie
enterprise integration • where complex business challenges meet agile solutions
5
Herkenbaar?
enterprise integration • where complex business challenges meet agile solutions
6
Realiteit
enterprise integration • where complex business challenges meet agile solutions
7
Delivery Rationalization Framework DRF Document Generation
Monitoring
Managed
Configuration Management
Defined Quantitatively Managed Reporting and Notification
OPTIMIZED
Code Delivery
Vandaag Automatisch Testen Deployment Management
Test Delivery
Sonic Deployment Manager/SDP (laatste business lunch)
Code Review
enterprise integration • where complex business challenges meet agile solutions
8
Waar staan wij?
Test Delivery is the process of conducting tests, through manual or automated means, following defined test methodologies and best practices.
Level 3: Defined Level 2: Managed
Level 4: Quantitatively Managed
Level 5: Optimizing Continuous improvement
Automated and Measurable
Standardized test methodologies
Clearly defined test cases
Level 1: Initial Manual, Freestyle
enterprise integration • where complex business challenges meet agile solutions
9
Waarom automatisch testen? Dynamiek (veel veranderingen) en eisen (SLA’s) Snelle, kwalitatieve en consistente oplevering Eerste stap in het process van continuous delivery.
enterprise integration • where complex business challenges meet agile solutions
12
Voor- en nadelen van automatisch testen? Voordelen Bespaart tijd en geld Vebetert kwaliteit en vertouwen Vergroot snelheid, efficientie en dekking (scope) Nadelen Schrijven en debuggen testscripts is tijdrovend Onderhoud testscripts en testdata
enterprise integration • where complex business challenges meet agile solutions
13
Wat zijn de mogelijkheden ?
TMNS ervaringen van smalle tot brede scope: JMS UnitTest SoapUI (mogelijk in combinatie met Hermes en/of Jenkins) Lisa Virtual testing
enterprise integration • where complex business challenges meet agile solutions
14
JMS Unittest (waarom?) Snel inzetbaar (minimale investering) Herbruikbaar en betrouwbaar Simpel in gebruik Snel resultaat (analyse van testuitkomsten) Nadeel: statisch, beperkt in mogelijkheden Residu in JMS Message Manager enterprise integration • where complex business challenges meet agile solutions
15
Concept JMS Unittest Input
Input Input 7.6 = Input 2013
7.6
Result
Output 7.6
Sonic 2013
Sonic 7.6
Test
2013
Output Compare
2013
enterprise integration • where complex business challenges meet agile solutions
17
Demo JMS Unittest
enterprise integration • where complex business challenges meet agile solutions
18
SoapUI (waarom?)
Freeware en goede support Gebruikersvriendelijke grafische Interface MockService (isolatie, simulatie, virtualisatie) Sterk in het testen van webservices (request/reply) Jenkins mogelijk als orchestrator I.c.m. Hermes Synchroon testen ESB mogelijk Mogelijk nadeel: Vereist WSDL
enterprise integration • where complex business challenges meet agile solutions
22
Lisa Wat? LISA voorziet in één simpel te gebruiken geintegreerde omgeving voor het ontwikkelen, plaatsen en monitoren van tests. Waarom? Geen code benodigd om test uit te voeren Batch uitvoering op basis van datasets Verschillende omgevingen zonder scriptwijziging Uitgebreide grafische rapportagemogelijkheden
enterprise integration • where complex business challenges meet agile solutions
24
LISA Context
enterprise integration • where complex business challenges meet agile solutions
27
LISA Scope
Web/Web Services HTTP/HTML Request REST Step Web Service Execution (XML) WSDL Validation Raw SOAP Request Base64 Encoder Step Multipart Mime Step SAML Assertion Query Web Service Execution (Legacy) Start or Stop Web Server (Legacy) Java/J2EE Dynamic Java Execution RMI Server Execution Enterprise Java Bean Execution Other Transactions SQL Database Execution (JDBC) Corba Execution Utilities Save Property as Last Response Output Log Message Write Properties to File Read Properties to File Do-Nothing Step Parse Text as Response Audit Step Base64 Encoder Step Checksum Step Convert XML to Element Object Compare Strings for Response Lookup Compare Strings for Next Step Lookup External/Sub Process Execute External Command File System Snapshot
Execute Sub Process Execute JUnit Test Case/Suite Read a File (Disk, URL or Class Path) FTP Step JMS Messaging JMS Messaging (JNDI) Message Consumer bea Weblogic JMS (JNDI) Message Consumer Read a File (Disk, URL, or Classpath) Web Service Execution (XML) Raw SOAP Request FTP Step Web Service Execution (Legacy) Sun JCAPS JCaps Messaging (Native) JCaps Messaging (JNDI) Message Consumer Read a File (Disk, URL, or Classpath) Web Service Execution (XML) Raw SOAP Request SQL Database Execution (JDBC) FTP Step Web Service Execution (Legacy) Oracle Oracle OC4J (JNDI) Oracle AQ (JMS) Oracle AQ (JPub) Message Consumer Read a File (Disk. URL, or Classpath) Web Service Execution (XML) Raw SOAP Request
SQL Database Execution (JDBC) FTP Step Web Service Execution (Legacy) TIBCO TIBCO Rendezvous Messaging TIBCO EMS Messaging TIBCO Direct JMS Message Consumer Read a File (Disk. URL, or Classpath) Web Service Execution (XML) Raw SOAP Request SQL Database Execution (JDBC) FTP Step Web Service Execution (Legacy) Sonic SonicMQ Messaging (Native) SonicMQ Messaging (JNDI) Message Consumer Read a File (Disk, URL, or Classpath) Web Service Execution (XML) Raw SOAP Request SQL Database Execution (JDBC) FTP Step Web Service Execution (Legacy) WebMethods webMethods Broker webMethods Integration Server Services Message Consumer Read a File (Disk, URL or Classpath) Web Service Execution (XML) Raw SOAP Request SQL Database Execution (JDBC) HTTP/HTML Request
REST Step FTP Step Web Service Execution (Legacy) IBM IBM Websphere MQ Message Consumer Virtual Service Environment Virtual Service Router Virtual Service Tracker Virtual Conversational/Stateless Response Selector Virtual HTTP/S Listener Virtual HTTP/S Live Invocation Virtual HTTP/S Responder Virtual JDBC Listener Virtual JDBC Responder Socket Server Emulator Messaging Virtualization Marker Compare Strings for Response Lookup Compare Strings for Next Step Lookup Custom Extensions Custom Test Step Execution Java Script Step Pathfinder Agent Script step Swing Test Step Create a Virtual Web service Java Protocol Request Listener Java Protocol Responder Java Pass Through
enterprise integration • where complex business challenges meet agile solutions
28
enterprise integration • where complex business challenges meet agile solutions
29
Lisa demo
enterprise integration • where complex business challenges meet agile solutions
30
Best Practices 1. Zorg dat je juiste mensen de juiste dingen doen 2. Genoeg tijd en resources 3. Denk aan onderhoudbaarheid 4. Hou het simpel 5. Automatiseer op basis van de werkelijke omgeving 6. Maak en hou verwachtingen helder 7. Volledige kennis en begrip van de applicatie 8. Ontwikkel strategie en voer deze stap voor stap uit 9. Beheers gedrag Testcase en Script Reviews 10. Behandel test scripts als een ontwikkelproject enterprise integration • where complex business challenges meet agile solutions
31
Samenvatting
Level 3: Defined Level 2: Managed
Level 4: Quantitatively Managed
Level 5: Optimizing Continuous improvement
Automated and Measurable
Standardized test methodologies
Clearly defined test cases
Level 1: Initial Manual, Freestyle
enterprise integration • where complex business challenges meet agile solutions
32
Conclusie Altijd profijt: Het is nooit te laat om met automatisch testen te beginnen Groeipad: Eenmaal ingericht is er altijd ruimte voor verbetering
enterprise integration • where complex business challenges meet agile solutions
33
