Gnorasi

A Modular Knowledge-Based Platform for Object-Based Image Analysis

Angelos Tzotsos, Gnorasi Developer, OTB Developer, NTUA

OSGeo Charter Member

OGC Member


GEOBIA 2014 Thessaloniki


Introduction

  • Gnorasi is a software platform for advanced remote sensing image processing
  • The project's goal is to provide:
    • knowledge-based image analysis methods
    • state of the art image processing algorithms
    • easy to use GUI
    • predefined knowledge bases for various remote sensing applications

Gnorasi origin

  • The word gnorasi is a combination of the Greek words:
    • ΓΝΩΣΗ (Knowledge)
    • + ΟΡΑΣΗ (Vision)
    • = ΓΝΩΡΑΣΗ

What lead us to Gnorasi

  • NOT Yet Another Remote Sensing Application
  • Existing approaches tend to have poor extensibility and modularity properties
  • Gnorasi addresses the formalization of the semantic interpretation of remote sensing data through a modular, extensible system that enables the effective combination of:
    • Knowledge
    • Reasoning
    • Imaging processing algorithms

Gnorasi goals

  • An integrated, modular framework for the seamless and effective development of remote sensing applications
  • Investigation and research into novel algorithms for remote sensing data analysis
  • Research in knowledge modeling and inferencing methodologies
  • Platform for Object-Based Image Analysis

Added value

  • Gnorasi enables the expert user to effectively put into practice its scientific expertise in an graphical, intuitive fashion
  • Through the gradual use of Gnorasi platform experts’ knowledge can be accumulated and made available to new users
  • Open Science!

Target Group

  • Gnorasi can be used/extended on
    • Agricultural planning
    • Environmental applications
    • Hazards management
    • Decision support systems
    • Land use/Land Cover applications
    • Research on climate changes
  • Wherever satellite and aerial images could be used to extract knowledge
  • ...and by remote sensing experts and researchers to build and test their algorithms, visually...
  • Open Source Community!

Gnorasi Users

users

Project Partners

Gnorasi is:

  • Free (as in Freedom) Software
  • Released under GPL v2 license
  • Written in C++
  • Cross platform (Linux, Windows, OS X)

Free and Open Source Software (FOSS)


  • It's a prerequisite for the Web, for Big Data and Cloud Computing
  • Everybody uses FOSS today, even without knowing it


GNU OSS

Open Source Geospatial
Foundation (OSGeo)

Since 2006 A Non Profit Umbrella for:

  • GeoSpatial Free and Open Source Software
  • Education
  • Open Data

GNU

OGC Standards

  • Interoperability
  • Future Proof
  • Return on Investment

OSGeoLive

  • GNU/Linux distribution
  • 60+ Open Source Geospatial Applications
  • Sample Datasets
  • Consistent Overviews & Quickstarts
  • Translations

Remote Sensing


Desktop GIS

General GIS viewing, editing, and analysis on the desktop


Browser Facing GIS

General GIS viewing, editing and analysis in the browser


Web Services

Publishing spatial data to the internet


Data Stores

Storing spatial data


Navigation and Maps



Spatial Tools

Specific analysis tools


Domain Specific GIS

Applications targeted at a specific domain


Geospatial Libraries



Data-flow network in Gnorasi

  • Close to Unix Pipelining: Successive actions one after the other
  • Graph Theory
  • Smarter than simple pipe-lining:
    • Parallel processing
    • Connection validation
    • Multiple Input / Output (Ν:Ν relation)
    • Ability to support heterogeneous inputs on the same processor
    • Multi-instance per processor (theoretically infinite number of same processors in singe data-flow)

Data-flow network in Gnorasi

  • Data-flow networks consist of modular units, called processors, which encapsulate rendering and data processing algorithms.
  • A processor operates on input data it receives from its inports and outputs the processed results via its outports.
  • Processor ports are typed (Image, Vector, LabelMap etc.)
  • Processors furthermore have properties for the parameterization of the encapsulated algorithms
  • and widgets (QT based)

Graphical representation of data-flow networks

users

Graphical representation of data-flow networks

users

Why we need data-flow networks

users

Why we need data-flow networks

  • User friendly
  • Advances experimentation for remote sensing experts
  • Rapid development and direct results
  • Possibility to overview intermediate results
  • Easy adaptation
  • Reduces needed resources and time

Gnorasi Implementation

Software Dependencies

  • Qt Framework
  • Voreen
  • Insight Toolkit (ITK)
  • Orfeo Toolbox (OTB)
  • GDAL

Voreen

  • open source VOlume REndering ENgine
  • used mostly on medical applications
  • allows interactive visualization of volumetric data sets
  • http://voreen.uni-muenster.de/

Insight Toolkit (ITK)

  • ITK is an open-source, cross-platform system for image analysis.
  • used for medical applications
  • ITK employs leading-edge algorithms for registering and segmenting multidimensional data
  • http://www.itk.org/

Orfeo Toolbox (OTB)

  • an open source library of image processing algorithms, adapted to large remote sensing images
  • Targeted state-of-the-art algorithms for high resolution remote sensing imagery, hyperspectral sensors or SAR
  • CNES ORFEO Accompaniment Program
  • Orfeo Toolbox is not a black box
  • http://www.orfeo-toolbox.org/otb/

Orfeo Toolbox

Multi-purpose library for
image processing

  • Orthorectification, cartographic projections
  • Image fusion, pan-sharpening
  • Radiometric correction
  • Registration
  • Filtering
  • OBIA: segmentation, classification
  • Feature extraction (texture, lines, vegetation indexes)
  • Change detection
  • Feature extraction
  • Supervised learning
  • Spatial reasoning
  • etc...

Orfeo Toolbox

Main characteristics

  • Extension of the ITK API formalism
  • Open Source (CeCILL licence)
  • Multi-platforms (GNU/Linux, UNIX, MS-Windows, MAC OS)
  • C++ language
  • Open source libraries (ITK, GDAL, OSSIM, VTK, FLTK, LibSVM, Python, Qt etc)
  • Streaming and multi-threading process
  • Synchronized pipeline (processing on demand)

OTB-QGIS Integration

Through Sextante, now known as QGIS Processing

Gnorasi UI Demonstration

Gnorasi UI Demonstration

http://www.youtube.com/watch?v=Pqf3_-QrXZg

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Gnorasi UI Demonstration

Object-Based Image Analysis with Ontologies

Ontology definitions

  • An ontology defines a set of representational primitives with which to model a domain of knowledge or discourse (Gruber, 1995)
    • Classes: which represent a set of objects
    • Properties: which express attributes and relations between classes and objects
    • Constraints: for expressing logical consistency
    • Individuals: atoms (objects) which are members of a class
  • Abstraction level of data models
    • Analogous to hierarchical and relational models
  • Ontologies, through inference, can provide us with implicitly defined information
    • Reasoning engines

Ontology definitions

Chalenges

  • Object-based image analysis produces a large number of objects (many thousands)
  • Probabilistic inference for class membership
  • Classification is based on user-defined rules
    • Feature-based
    • Spatial analysis
  • Classes can appear as premises in rules

Proposed solutions

  • Numerical computations are executed outside the ontological framework
  • Fuzzy membership values are computed using membership functions
  • Ontological inference is used for the assignment of objects to classes
  • User defined rules, through UI
  • SPARQL Update and GeoSPARQL are used to define the rules in ontological terms
  • Development decision: The ontological classification is implemented as an external web service

Ontology UI

Ontology UI

Fuzzy Values

  • The outcome of the rule definition processor are the fuzzy values of all objects for the features present in the rules
  • These arithmetic values have to be assigned to semantic entities, i.e. the defined classes

Ontology Data

  • The fuzzy values along with the class hierarchy and rule definitions are sent to the ontology classification service
  • The following are performed by the service
    • The class hierarchy is added to the core ontology
    • The fuzzy values are transformed to ontological data properties
    • Rules are translated to SPARQL Update queries
    • Rules are iteratively executed

Example SPARQL Update

  • Example queries for assigning an object to classes Sidewalk and Vegetation with confidences ?conf1 and ?conf2

Sidewalk
INSERT {?object gn:depicts gn:depiction1.
            gn:depiction1 gn:depictsClass gn:Sidewalk. 
            gn:depiction1 gn:withConfidence ?conf.
} WHERE {?object rdf:type gn:Object. 
    ?object gn:fuzzyNDVIMean ?conf.
}
                

Vegetation
INSERT {?object gn:depicts gn:depiction2.
            gn:depiction2 gn:depictsClass gn:Vegetation. 
            gn:depiction2 gn:withConfidence ?conf2.
} WHERE {?object rdf:type gn:Object. 
               ?object gn:fuzzyNDVIMean ?conf2.
}
                

Example SPARQL Update

  • Sidewalk rule with geospatial restriction (adjacent to Road)

INSERT {?object gn:depicts gn:depiction1.
            gn:depiction1 gn:depictsClass gn:Sidewalk. 
            gn:depiction1 gn:withConfidence ?conf.
} WHERE {?object rdf:type gn:Object . 
               ?object gn:fuzzyNDVIMean ?conf.
               ?filterObject rdf:type gn:Object.
               ?filterObject gn:depicts ?gn:filterDepiction.
               ?filterDepiction gn:depictsClass gn:Road
               FILTER (geof:sfTouches(?object, ?filterObject))
}
                

Classification

  • In the example, Sidewalk depends on Road definition. Objects assigned to Road must exist
  • Iterative rule execution until convergence (no changes in the repository)
  • In the end, every object will be assigned to classes with different membership values

INSERT {?object gn:depicts gn:depiction1.
            gn:depiction1 gn:depictsClass gn:Sidewalk. 
            gn:depiction1 gn:withConfidence ?conf.
} WHERE {   ?object rdf:type gn:Object. 
    ?object gn:fuzzyNDVIMean ?conf.
    ?filterObject rdf:type gn:Object.
    ?filterObject gn:depicts ?gn:filterDepiction.
    ?filterDepiction gn:depictsClass gn:Road
    FILTER (geof:touches(?object, ?filterObject))
}
                

Classification

  • In a hierarchy, objects will try to match the deepest classes

Classification

  • Object are assigned to classes with the highest membership value
  • A minimum threshold is applied
  • The service returns a list:
    
    [object id],[class id],[confidence]
                            
  • This is the classification result

Ontology development

  • Java based
  • REST communication
  • Server:
    • Jetty
  • Reasoner:
    • OpenRDF Sesame2 backend
    • OWLIM-lite3 reasoner
  • Geospatial capabilities:
    • PostgreSQL
    • PostGIS
    • uSeekM IndexingSail

Concluding

  • Efficient ontology-based classification
  • Employ both feature-based classification and geospatial restrictions
  • Handling of fuzzy membership values
  • Built as a web service

Getting Involved

Web Site

http://www.gnorasi.gr/


Source Code, Wiki, Issues on GitHub

https://github.com/kalxas/gnorasi


Mercurial repository

http://aiolos.survey.ntua.gr/hg/gnorasi


Live Demo DVD

http://www.gnorasi.gr/downloads/gnorasi-live.i686-0.5.0.iso

The Gnorasi Team

  • Charalampos Doulaverakis (ITI)
  • Angelos Tzotsos (NTUA)
  • Ioannis Tsampoulatidis (ITI)
  • Nikolaos Gerontidis (ITI)
  • Argyros Argyridis (NTUA)
  • Konstantinos Karantzalos (NTUA)
  • Ioannis Kompatsiaris (ITI)
  • Demetre Argialas (NTUA)
  • Argyris Margaritis (Draxis)
  • Stavros Tekes (Draxis)
  • Panagiotis Symeonidis (Draxis)
  • Dimitrios Tektonidis (Altec)

Thank you

Questions?